* [PATCH v3 00/24] Convert avocado tests to normal Python unittests
@ 2024-07-30 17:03 Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 01/24] python: Install pycotap in our venv if necessary Daniel P. Berrangé
` (24 more replies)
0 siblings, 25 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P. Berrangé
This series is an update to Thomas' v2:
https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg05805.html
wherein Thomas suggested someone could do the asset caching updates
while he is on vacation:
https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg06228.html
hence this posting what I'm calling a v3.
The Avocado v88 that we use in QEMU is already on a life support
system: It is not supported by upstream anymore, and with the latest
versions of Python, it won't work anymore since it depends on the
"imp" module that has been removed in Python 3.12.
There have been several attempts to update the test suite in QEMU
to a newer version of Avocado, but so far no attempt has successfully
been merged yet.
Additionally, the whole "make check" test suite in QEMU is using the
meson test runner nowadays, so running the python-based tests via the
Avocodo test runner looks and feels quite like an oddball, requiring
the users to deal with the knowledge of multiple test runners in
parallel (e.g. the timeout settings work completely differently).
So instead of trying to update the python-based test suite in QEMU
to a newer version of Avocado, we should try to better integrate
it with the meson test runner instead. Indeed most tests work quite
nicely without the Avocado framework already, as you can see with
this patch series - it does not convert all tests, just a subset so
far, but this already proves that many tests only need small modifi-
cations to work without Avocado.
Only tests that use the LinuxTest / LinuxDistro and LinuxSSHMixIn
classes (e.g. based on cloud-init images or using SSH) really depend
on the Avocado framework, so we'd need a solution for those if we
want to continue using them. One solution might be to simply use the
required functions from avocado.utils for these tests, and still run
them via the meson test runner instead, but that needs some further
investigation that will be done later.
Now if you want to try out these patches: Apply the patches, then
recompile and then run:
make check-functional
You can also run single targets e.g. with:
make check-functional-ppc
You can also run the tests without any test runner now by
setting the PYTHONPATH environment variable to the "python" folder
of your source tree, and by specifying the build directory via
QEMU_BUILD_ROOT (if autodetection fails) and by specifying the
QEMU binary via QEMU_TEST_QEMU_BINARY. For example:
export PYTHONPATH=$HOME/qemu/python
export QEMU_TEST_QEMU_BINARY=qemu-system-x86_64
export QEMU_BUILD_ROOT=$HOME/qemu/build
~/qemu/tests/functional/test_virtio_version.py
The logs of the tests can be found in the build directory under
tests/functional/<arch>/<testname> - console log and general logs will
be put in separate files there.
Still to be done: Update the documentation for this new test framework.
v3:
- Split tests/functional/__init__.py into multiple files
- Introduce an 'Asset' class to handle downloading & caching
- Convert all tests to use new Asset class instead of 'fetch_asset'
- Ensure 'make check-functional' pre-caches all assets before
running the tests, to avoid meson test timeouts due to slow
downloads
- Fix bios bits test logging in verbose mode to not print subprocess
output to stdout, since it confuses TAP parsing
v2:
- Addressed review feedback from v1
- Add pycotap as a wheel instead of trying to install it on demand
when running "make check-functional" (works much better now!)
- Converted much more tests
- Lots of other small improvements here and there
RFC -> v1:
- Now using pycotap for running the tests instead of "pytest"
- Change the name from "tests/pytest" to "tests/functional" accordingly
- Make it possible to run the tests directly
- Use Python's urllib instead of wget for downloading
- Lots of makefile / meson integration improvements
- Converted more tests
- Update MAINTAINERS file accordingly
- Added a patch to run check-functional in the gitlab-CI
- ... lots of other changes I forgot about ... in fact, I changed so
many things that I also did not dare to pick up the Reviewed-bys
from the RFC
Daniel P. Berrangé (2):
tests/functional: add a module for handling asset download & caching
tests/functional: enable pre-emptive caching of assets
Thomas Huth (22):
python: Install pycotap in our venv if necessary
tests/functional: Add base classes for the upcoming pytest-based tests
tests/functional: Set up logging
tests/Makefile.include: Increase the level of indentation in the help
text
tests/functional: Prepare the meson build system for the functional
tests
tests/functional: Convert simple avocado tests into standalone python
tests
tests/functional: Convert avocado tests that just need a small
adjustment
tests/functional: Convert some tests that download files via
fetch_asset()
tests/functional: Add a function for extracting files from an archive
tests/functional: Convert some avocado tests that needed
avocado.utils.archive
tests/functional: Convert the s390x avocado tests into standalone
tests
tests/functional: Convert the x86_cpu_model_versions test
tests/functional: Convert the microblaze avocado tests into standalone
tests
tests/functional: Convert the riscv_opensbi avocado test into a
standalone test
tests/functional: Convert the virtio_gpu avocado test into a
standalone test
tests/functional: Convert most ppc avocado tests into standalone tests
tests/functional: Convert the ppc_amiga avocado test into a standalone
test
tests/functional: Convert the ppc_hv avocado test into a standalone
test
tests/functional: Convert the m68k nextcube test with tesseract
tests/functional: Convert the acpi-bits test into a standalone test
tests/functional: Convert the rx_gdbsim avocado test into a standalone
test
gitlab-ci: Add "check-functional" to the build tests
.gitlab-ci.d/buildtest-template.yml | 3 +-
.gitlab-ci.d/buildtest.yml | 60 +++---
MAINTAINERS | 32 +--
python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
pythondeps.toml | 1 +
tests/Makefile.include | 42 ++--
tests/avocado/machine_microblaze.py | 61 ------
tests/avocado/machine_mips_loongson3v.py | 39 ----
tests/avocado/machine_sparc64_sun4u.py | 36 ----
tests/avocado/ppc_amiga.py | 38 ----
tests/avocado/riscv_opensbi.py | 63 ------
tests/avocado/tesseract_utils.py | 46 -----
.../acpi-bits/bits-config/bits-cfg.txt | 0
.../acpi-bits/bits-tests/smbios.py2 | 0
.../acpi-bits/bits-tests/smilatency.py2 | 0
.../acpi-bits/bits-tests/testacpi.py2 | 0
.../acpi-bits/bits-tests/testcpuid.py2 | 0
tests/functional/meson.build | 180 +++++++++++++++++
tests/functional/qemu_test/__init__.py | 14 ++
tests/functional/qemu_test/asset.py | 130 ++++++++++++
tests/functional/qemu_test/cmd.py | 171 ++++++++++++++++
tests/functional/qemu_test/config.py | 36 ++++
tests/functional/qemu_test/tesseract.py | 35 ++++
tests/functional/qemu_test/testcase.py | 187 ++++++++++++++++++
tests/functional/qemu_test/utils.py | 47 +++++
.../test_acpi_bits.py} | 81 ++++----
.../test_arm_canona1100.py} | 30 +--
.../test_arm_n8x0.py} | 36 ++--
.../test_avr_mega2560.py} | 22 ++-
.../test_cpu_queries.py} | 7 +-
.../test_empty_cpu_model.py} | 7 +-
.../test_info_usernet.py} | 11 +-
.../test_loongarch64_virt.py} | 46 +++--
.../test_m68k_nextcube.py} | 29 +--
.../test_mem_addr_space.py} | 52 +----
.../functional/test_microblaze_s3adsp1800.py | 39 ++++
.../test_microblazeel_s3adsp1800.py | 42 ++++
tests/functional/test_mips64el_loongson3v.py | 39 ++++
.../test_netdev_ethtool.py} | 50 ++---
.../test_pc_cpu_hotplug_props.py} | 11 +-
.../test_ppc64_hv.py} | 48 ++---
.../test_ppc64_powernv.py} | 56 ++----
.../test_ppc64_pseries.py} | 56 ++----
.../ppc_405.py => functional/test_ppc_405.py} | 28 +--
.../test_ppc_40p.py} | 63 +++---
.../test_ppc_74xx.py} | 74 +++----
tests/functional/test_ppc_amiga.py | 42 ++++
.../test_ppc_bamboo.py} | 32 +--
.../test_ppc_mpc8544ds.py} | 28 +--
.../test_ppc_virtex_ml507.py} | 28 +--
tests/functional/test_riscv_opensbi.py | 36 ++++
.../test_rx_gdbsim.py} | 57 +++---
.../test_s390x_ccw_virtio.py} | 79 ++++----
.../test_s390x_topology.py} | 86 ++++----
tests/functional/test_sparc64_sun4u.py | 40 ++++
.../version.py => functional/test_version.py} | 13 +-
.../test_virtio_gpu.py} | 64 +++---
.../test_virtio_version.py} | 8 +-
.../test_x86_cpu_model_versions.py} | 63 ++----
tests/meson.build | 1 +
60 files changed, 1631 insertions(+), 994 deletions(-)
create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
delete mode 100644 tests/avocado/machine_microblaze.py
delete mode 100644 tests/avocado/machine_mips_loongson3v.py
delete mode 100644 tests/avocado/machine_sparc64_sun4u.py
delete mode 100644 tests/avocado/ppc_amiga.py
delete mode 100644 tests/avocado/riscv_opensbi.py
delete mode 100644 tests/avocado/tesseract_utils.py
rename tests/{avocado => functional}/acpi-bits/bits-config/bits-cfg.txt (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%)
create mode 100644 tests/functional/meson.build
create mode 100644 tests/functional/qemu_test/__init__.py
create mode 100644 tests/functional/qemu_test/asset.py
create mode 100644 tests/functional/qemu_test/cmd.py
create mode 100644 tests/functional/qemu_test/config.py
create mode 100644 tests/functional/qemu_test/tesseract.py
create mode 100644 tests/functional/qemu_test/testcase.py
create mode 100644 tests/functional/qemu_test/utils.py
rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py} (86%)
mode change 100644 => 100755
rename tests/{avocado/machine_arm_canona1100.py => functional/test_arm_canona1100.py} (52%)
mode change 100644 => 100755
rename tests/{avocado/machine_arm_n8x0.py => functional/test_arm_n8x0.py} (51%)
mode change 100644 => 100755
rename tests/{avocado/machine_avr6.py => functional/test_avr_mega2560.py} (73%)
mode change 100644 => 100755
rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%)
mode change 100644 => 100755
rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
mode change 100644 => 100755
rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
mode change 100644 => 100755
rename tests/{avocado/machine_loongarch.py => functional/test_loongarch64_virt.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/machine_m68k_nextcube.py => functional/test_m68k_nextcube.py} (76%)
mode change 100644 => 100755
rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_microblaze_s3adsp1800.py
create mode 100755 tests/functional/test_microblazeel_s3adsp1800.py
create mode 100755 tests/functional/test_mips64el_loongson3v.py
rename tests/{avocado/netdev-ethtool.py => functional/test_netdev_ethtool.py} (66%)
mode change 100644 => 100755
rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
mode change 100644 => 100755
rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (88%)
mode change 100644 => 100755
rename tests/{avocado/ppc_powernv.py => functional/test_ppc64_powernv.py} (71%)
mode change 100644 => 100755
rename tests/{avocado/ppc_pseries.py => functional/test_ppc64_pseries.py} (76%)
mode change 100644 => 100755
rename tests/{avocado/ppc_405.py => functional/test_ppc_405.py} (53%)
mode change 100644 => 100755
rename tests/{avocado/ppc_prep_40p.py => functional/test_ppc_40p.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_ppc_amiga.py
rename tests/{avocado/ppc_bamboo.py => functional/test_ppc_bamboo.py} (60%)
mode change 100644 => 100755
rename tests/{avocado/ppc_mpc8544ds.py => functional/test_ppc_mpc8544ds.py} (55%)
mode change 100644 => 100755
rename tests/{avocado/ppc_virtex_ml507.py => functional/test_ppc_virtex_ml507.py} (60%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_riscv_opensbi.py
rename tests/{avocado/machine_rx_gdbsim.py => functional/test_rx_gdbsim.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
mode change 100644 => 100755
rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_sparc64_sun4u.py
rename tests/{avocado/version.py => functional/test_version.py} (78%)
mode change 100644 => 100755
rename tests/{avocado/virtio-gpu.py => functional/test_virtio_gpu.py} (73%)
mode change 100644 => 100755
rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
mode change 100644 => 100755
rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
mode change 100644 => 100755
--
2.45.2
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v3 01/24] python: Install pycotap in our venv if necessary
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-31 12:49 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests Daniel P. Berrangé
` (23 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
The upcoming functional tests will require pycotap for providing
TAP output from the python-based tests. Since we want to be able
to run some of the tests offline by default, too, let's install
it along with meson in our venv if necessary (it's size is only
5 kB, so adding the wheel here should not really be a problem).
The wheel file has been obtained with:
pip download --only-binary :all: --dest . --no-cache pycotap
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
pythondeps.toml | 1 +
2 files changed, 1 insertion(+)
create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
diff --git a/python/wheels/pycotap-1.3.1-py3-none-any.whl b/python/wheels/pycotap-1.3.1-py3-none-any.whl
new file mode 100644
index 0000000000000000000000000000000000000000..9c2c7d259365c8a0487228c6878e2c0b10ba6bdd
GIT binary patch
literal 5119
zcmeI0c{r49-^VX&3}X-3cO#7@+t|t4j6saOFm|%Vlq5p-b*v*b+!#WUElXPLTgIN!
zWLF9yOO}-P=zd?_J-zqc^Y8P=^F5C1I?g|?>vNoc{Jy{6`8`b$6qIZL0MG#GGI+~H
zES3$R0sv1|0AK|Gz(3g84}<oXc5v|Y^~5+hNcjgpv?lq^D>8-dSoQbO2%udjK9nR1
zX@Mw7zRZ{mFv|`)4NPytMe=eP5h(K}-@36c__dlrZxYgrbFtHmU$`UJ-juerPK0GR
zshPauoDlZTdy!Dv>PW-C`_4w>c^0|S#!PJQr&Rx&%f}eZTPHMc1X8cU%aYZ2Q)PK2
zm@9*23{tU5IZhDSn50gKRt1xkbO~0B`E<O9e<BUPhQp1f#(gmRd99?AMrO*KBFkV7
z(7ULx8)hjBwWl=YvLVrCX$xltk4av?rYp!=^KMZtM-4fJ+?f6VI127^u`p15wv3O0
zN^3C6WLa0>YJDWa1^0xIIV#bxv{|)IIpNc11RFU$I6~(T)|;Gf1>iao+q>y8Qu&AR
zX&ZGcxw#t-3T;Z$&qQX^FO{VmvdcLPNNXO?UxY^wdnFsSo`}Ojht)qBW)<Y;Ihc5E
zDnV*dO~ulY*Q8!@&Xc><Mp?ScSvfsgk8t<OD7%FsZ-2<RM_oS5RZNeAA@)X&Az_jo
zo<ic3*a_<Bi)B07<{_<8$UL&qGmga3sr#3p<8?`d+;(aXS5ahFhxTc!JLsTfEdHTW
z5q_^LuK`^r?NK4S6wTpOaSLis^E6(!U>R#y&_6pL<Zx@$pJe1DA$0*x$vO!xF6|41
zzw?zYA~5g0Hj0`)zX+pAahkhX^x8Byqgw1)i#<4dsvbl-d;}so+OdIXxfWO+1*lf4
zOI{Q&rRyqPp?r`5X|lE%QxC5pTo@dgjwji3ujncdF>sGnx#HyZ>EF1+wb6y9RGk}|
zm|OSaNVkTPnmS*=!%K8^w1vs;6Ns#%7*MoHUtf9N>mJwo_){ZBm9}~XK?Na2=L8ds
ziyM~BIXK2LGrDWYR?Js<&cYIA4VTCR^oo`(vRq1Uid<2D0I!6n9<Q<oW4d7L_4?kE
z7X|&4V)3gpj9K|RK2sZa+WGG=lsdWI*(m9KjE~hVnCqjBQI19xnC|m<*3t*@%PUk0
z$z!gMcGt(=15T7%u?vO50~YZl>V5Vs#*|i-snEN6nLM3LEgwhU2dxN<boG2XU~iSW
znD9U-3rMoq=hGjN3(fx|*!@9}?j;TFb1*bn%pog%-OcJHWvMM2S)^!rw+wM;yZkaT
zx?jP`KDjol&aU>Utb%K^JAaFZe0iU1Gur}{wa*q5c6a@yHteQ+%(2J>nTbYw0soAc
zGzp7<Z7a*TYq|TtlmLybS<dQ)ZYa9$EAyq2e(1=X;+uP4SY6JADJzI^2#cF14x6Mb
zQpwt~9^yh6F{|Aoox{f?j8)kpu9-=rZ)y0&>E(+$gN)nkBfmNxcqBpa`}?Nj(bK71
zeA%e>r%@xv#hCfu#H-)c?JC5kq@H~t-wu09ER5T4Nqp@_VX0&s$fo}4!Yz?aeMh`D
z<^;?zR(m{Wz~?cHVVN%HF*on@H1D*XMlUXr;-1eNgKNHS<PNU(@#;e@(ablh;~N;+
zzLbPw`Va#QiRoRorN*U@RNouV8V5EjGCe8RJaxo1!*Kbpc|yM}P3-W~)6Gbl*CB@B
zma2xvP?HYjmI)5Rc!-*F(u$|reuel|J*NSWFM1Qo<4LAFNKabDL!F+tq`~F=0pD@E
zHLqp8`fa`g{NUE%!iV#)3|ng@a6d9b>+_+Io!?~G%d5|vy>`kiZmn1pNZ3=@w<rCv
z2a)CrBDWB;7L?iLF#XXmXQl9#pQc5+9RUCc0e)?mK4?$hKO3gGC2B|-Dm7x#*FS1P
zlr)Ewp6cxxlzTVV*AMNHK6ws8e11yeEL19jjtwQYh!bJ=ztrlo6~HRDDIc(D4i7Xt
z?tT1VW7F(1n-+_9ad2>%o|a%K`xB<_aHhsaF%ICb4Jqcy>+>z7lj6t$07eM_{C{<D
zsH~Knl&q8s8iSVp^QB{AYID}uz~UUymn1{~^ovAXo^1=FfvkC}<P}`FKMic-nO)>8
zS*_R&5^>ee_h78y;Qml@k&aVFH|!v+ryrm9M170dm!!J8zxQ0p^xf|Bh|j(UU%He1
zN!?dLj5|ASyKJF>m|A=jBE4_72;3^vM*Tinso5R2>y}i4K(8f}HN@8Y9kdddc;)Bi
z?j$k7X0B9M7x6L<X(duK-Rj`;j3jj<=E0Gw9%uY3e6WlvFY{8}l8_=#p70ullU{K*
zmOWrq3d%5a&m~*QF<pB|#lamClrn>io2u_f6Ds8$;0?a&!}!_b0Z3lMM^wNwfFURR
zdWk;<KRG&WRBgOC>8O)`#y$R8P>MA@%A06to-;ocOykBG*IKwHv3$FkR%hI#Zs_U-
z<_~SfK?b{6N~=1}FU+Du`BCNbHo1SynT+a}GhJVcX}tikUG5q%D_qBlaM=+DzABn$
z<BC$H3SV&0VZujA77r?hl-`HBSJLTRP$kb!e+kPYkBy6MX#A3~nZd_<_+<OUlC<*J
z+cSPSa{^was6CN@yms)tPoWuY`tMzv#=~hk2~%g90I`Pk3q5L7--^s0R~De+tu3Nw
ztXWS>jyGXbWxADZydWKL50Mw5*qJQ&dTx*9c~`{snkGopR7n++t&n>S<hV>}&WMm5
zhmrO}OADL8%)vBEP?KqCo7MTH6Xj0eT~>`h>}v{h2(P#%@AW!ZTH1;7;kdG0(*jQo
zD62ScMz)JY7t}<?TY!}9stp3}{-MjR&|0~wh%hdu+~y3s#a>L{w2^q&=AIyt^-c2;
zvw}&4#JSNk?B%-GY{iWQbPel7MQ@-BEZSoe1)`>wbDstGDobMTm2|AN)-Le>Rm3bN
zc*}oN#GZi|sHd-+-+znvzeW6?Tg1coH;<zz0N^7v00{j5PW+>Yk$Na?U2T+hv;`@J
zTZuk2|AD{|4(XVBo1H2_+jMO5kB{{eF6Wx5HpbFKUl*+;LEhJ_ZpH2k?p0%RtI-tP
zTb=c~X=-X}gf&@-qIBDD(So)C;Opkk%BRZ6xYvtH2-hzwJ}H0ZIxog}6XZ;B(QrH&
zD<eiTp-hie$qEnHP}X}@$uHp8uYPDuDJ$WQlBs=C8CFb&v*(3b;L=lxUYD!No;TZ$
zR)Jt@BTG?eytzR^gysT!eZ-r)hdsoX2%gN3+`IcB#Nlk)?e<j5oy=t2@+sS7setS+
z%|6?zDFz@h#TK-n>TOWiUe&7U<mZ$6%?&+GQ!}ie7mi_Q70(v)6nM_4TDR#4WL$PK
zv-Ezoef4e7B8$pR>hTF{3|FTQLl?^a$z*zi?aB=?63OM$Yc(C*u{EWjs?QojtOmYt
zS@F$HW~Dk4gX8PvDf&*Uk)S(g85o6G-_PVNfA~Z<^pwp*HBhn0`kQx%*xu)_?Xk^4
z&rlXG28b71^-Y;R&V;DBK1LDq$m>3qarlHli^{WftFEl{e26;G&AoONT8<u$%gC)z
zh;C>6IPE@pg4^_N*mcYB7tp4T^K6pEDFtjqc2sps?vNDcR9O!DQtk!Z29<`jlH}Kz
zo9&2zue)Xg(s{i!<DF3nsUg*;Mc-SP(xL~)peKPBJD5x%gT9RpyHv+fixcVn$BaDh
zjL@Nai6cXt!k^NFCp+$|nyLl!k}I!X@l#e_yyETiHv677cJFph=AJ5F>ch?UZfpG^
z$*(={veXW|;7#=JB@B(-_VmD-$vnyDKyF>Z@6Kf;2G6)Bnk85=jIr97&%j)G&V<X7
zD_k49uN+zyUnBcU&Ip%FKVDUR+K6S==^@R<`^0#VoRwNlcT4}J#f7~SNU{L2(O2(7
z6!JO)RvN*PrB&_0&a}k10Wj_xyphm|d(%qWquWa^6fs{?#)No#5W;^oYojgrP73vt
z`_y-X57=>M=4fY2LqdAAKc5YBLk8(S22@nC?-k8WsJ5#1!XIf~vI5PLCs`abP|JJ7
z)T=BQQsP{Q(i*H(@-kbCZYS5czc$@gDGn_Sriq%%%v^QZ5Dp4`4$!Y!ppg)v{1wMT
z*Ke*KY<15HLfB%A<@PVBNIz6ZzBksB=LsH6aOaxB4zSqwT0)P{YQovO#%g&7X<WrO
z@ExkD^MvW8?ZdAT{#}pu>Hl>M6%@;^o(BNn`@{Pi2U{8H=^4=t8X}3qU<CB!K>uK^
znar5nOn?ay0fnFHm98-}g!i73ikCiCc5X;!NUnZJZg>IIHz+;4eQF#Gfx?NsMm1&-
ziId2;ptD8|W{^_(apLnq#0zBqy@fgQpN@Fnr{e`JAp_8#ksSa;f3q!&pTC2*>lIgT
zDa=)jWU0}B^x0E0zrr-`+Nrpc0f0X_0Py_=W~QfOVy63WsL8iT3kaXp7>uNq<B1XK
z%n?>(cqhYPaMMk;8^*&b*M+^C$`)@pVbSOFZ9_6lWl_C-OZK^;fpzvL%)1ya9h=~5
z_mCtLnrQEvj%hK;#R_gS4g$VjVv4tW^Ok}W4ia4g(n~xTw@6;HAHu9vo03mezljm*
zGf;hI_f>z#@&)=v9zsIHKGE#3g<yy$;VDedmmZ(d<6C#{U!S+ayhL1yU5uvkMvd(Z
z6zSxaRp5QL7`h+eC@avR%y@W=l6+q3lg=wVOsH8u9KE;Rqnbm%O6Qm!EkWCv#%9mn
zWqay0PMFK%NA;6>bBb!6>@pMwh09$sLhZa%5fsbYn!-<KJuc=-CxX_fE^F<GoiHe?
z@(4)Q;(Qtse0UwwaQUN~u5a-h(U19gqcac0mOaPWoQK!Fof3V(ZA%*bqf3~{qO{DX
zujfBd?26||uoO1zqM`$BCG0c(mmXjomwX-l-?WRHBFM<uDE|3c=17Ey@2kJLr1=@}
zcQNHiga}!xp8-EY%g=znixo#AM6~?{_@|KZGwkn<cSj;b#M1oO%@OR!oczDU@+<7u
zF?%FJgc038!;XgTue4t+e<VT#1LHr_{%iohqJFjMkq8mRzd`+rX@5ojYStswk!}12
a`M=gRMNm<H|DFb*0Z`-skj4D{@BaZhvADJX
literal 0
HcmV?d00001
diff --git a/pythondeps.toml b/pythondeps.toml
index f6e590fdd8..0bc4144d4b 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -20,6 +20,7 @@
[meson]
# The install key should match the version in python/wheels/
meson = { accepted = ">=1.1.0", installed = "1.2.3", canary = "meson" }
+pycotap = { accepted = ">=1.1.0", installed = "1.3.1" }
[docs]
# Please keep the installed versions in sync with docs/requirements.txt
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 01/24] python: Install pycotap in our venv if necessary Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-31 12:53 ` Philippe Mathieu-Daudé
` (2 more replies)
2024-07-30 17:03 ` [PATCH v3 03/24] tests/functional: Set up logging Daniel P. Berrangé
` (22 subsequent siblings)
24 siblings, 3 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P . Berrangé
From: Thomas Huth <thuth@redhat.com>
The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py
file with some adjustments to get rid of the Avocado dependencies (i.e.
we also have to drop the LinuxSSHMixIn and LinuxTest for now).
The emulator binary and build directory are now passed via
environment variables that will be set via meson.build later.
Signed-off-by: Thomas Huth <thuth@redhat.com>
[DB: split __init__.py into multiple files]
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/functional/qemu_test/__init__.py | 13 ++
tests/functional/qemu_test/cmd.py | 171 +++++++++++++++++++++++++
tests/functional/qemu_test/config.py | 36 ++++++
tests/functional/qemu_test/testcase.py | 154 ++++++++++++++++++++++
4 files changed, 374 insertions(+)
create mode 100644 tests/functional/qemu_test/__init__.py
create mode 100644 tests/functional/qemu_test/cmd.py
create mode 100644 tests/functional/qemu_test/config.py
create mode 100644 tests/functional/qemu_test/testcase.py
diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
new file mode 100644
index 0000000000..2f1e0bc70d
--- /dev/null
+++ b/tests/functional/qemu_test/__init__.py
@@ -0,0 +1,13 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2024 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+
+from .config import BUILD_DIR
+from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
+ interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
+ exec_command, exec_command_and_wait_for_pattern
+from .testcase import QemuSystemTest, QemuBaseTest
diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test/cmd.py
new file mode 100644
index 0000000000..67e860e2c8
--- /dev/null
+++ b/tests/functional/qemu_test/cmd.py
@@ -0,0 +1,171 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2018, 2024 Red Hat, Inc.
+#
+# Original Author (Avocado-based tests):
+# Cleber Rosa <crosa@redhat.com>
+#
+# Adaption for standalone version:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import logging
+import os
+import os.path
+import subprocess
+
+
+def has_cmd(name, args=None):
+ """
+ This function is for use in a @skipUnless decorator, e.g.:
+
+ @skipUnless(*has_cmd('sudo -n', ('sudo', '-n', 'true')))
+ def test_something_that_needs_sudo(self):
+ ...
+ """
+
+ if args is None:
+ args = ('which', name)
+
+ try:
+ _, stderr, exitcode = run_cmd(args)
+ except Exception as e:
+ exitcode = -1
+ stderr = str(e)
+
+ if exitcode != 0:
+ cmd_line = ' '.join(args)
+ err = f'{name} required, but "{cmd_line}" failed: {stderr.strip()}'
+ return (False, err)
+ else:
+ return (True, '')
+
+def has_cmds(*cmds):
+ """
+ This function is for use in a @skipUnless decorator and
+ allows checking for the availability of multiple commands, e.g.:
+
+ @skipUnless(*has_cmds(('cmd1', ('cmd1', '--some-parameter')),
+ 'cmd2', 'cmd3'))
+ def test_something_that_needs_cmd1_and_cmd2(self):
+ ...
+ """
+
+ for cmd in cmds:
+ if isinstance(cmd, str):
+ cmd = (cmd,)
+
+ ok, errstr = has_cmd(*cmd)
+ if not ok:
+ return (False, errstr)
+
+ return (True, '')
+
+def run_cmd(args):
+ subp = subprocess.Popen(args,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ universal_newlines=True)
+ stdout, stderr = subp.communicate()
+ ret = subp.returncode
+
+ return (stdout, stderr, ret)
+
+def is_readable_executable_file(path):
+ return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
+
+def _console_interaction(test, success_message, failure_message,
+ send_string, keep_sending=False, vm=None):
+ assert not keep_sending or send_string
+ if vm is None:
+ vm = test.vm
+ console = vm.console_file
+ console_logger = logging.getLogger('console')
+ while True:
+ if send_string:
+ vm.console_socket.sendall(send_string.encode())
+ if not keep_sending:
+ send_string = None # send only once
+ try:
+ msg = console.readline().decode().strip()
+ except UnicodeDecodeError:
+ msg = None
+ if not msg:
+ continue
+ console_logger.debug(msg)
+ if success_message is None or success_message in msg:
+ break
+ if failure_message and failure_message in msg:
+ console.close()
+ fail = 'Failure message found in console: "%s". Expected: "%s"' % \
+ (failure_message, success_message)
+ test.fail(fail)
+
+def interrupt_interactive_console_until_pattern(test, success_message,
+ failure_message=None,
+ interrupt_string='\r'):
+ """
+ Keep sending a string to interrupt a console prompt, while logging the
+ console output. Typical use case is to break a boot loader prompt, such:
+
+ Press a key within 5 seconds to interrupt boot process.
+ 5
+ 4
+ 3
+ 2
+ 1
+ Booting default image...
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ :param interrupt_string: a string to send to the console before trying
+ to read a new line
+ """
+ _console_interaction(test, success_message, failure_message,
+ interrupt_string, True)
+
+def wait_for_console_pattern(test, success_message, failure_message=None,
+ vm=None):
+ """
+ Waits for messages to appear on the console, while logging the content
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ """
+ _console_interaction(test, success_message, failure_message, None, vm=vm)
+
+def exec_command(test, command):
+ """
+ Send a command to a console (appending CRLF characters), while logging
+ the content.
+
+ :param test: a test containing a VM.
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param command: the command to send
+ :type command: str
+ """
+ _console_interaction(test, None, None, command + '\r')
+
+def exec_command_and_wait_for_pattern(test, command,
+ success_message, failure_message=None):
+ """
+ Send a command to a console (appending CRLF characters), then wait
+ for success_message to appear on the console, while logging the.
+ content. Mark the test as failed if failure_message is found instead.
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param command: the command to send
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ """
+ _console_interaction(test, success_message, failure_message, command + '\r')
diff --git a/tests/functional/qemu_test/config.py b/tests/functional/qemu_test/config.py
new file mode 100644
index 0000000000..edd75b7fd0
--- /dev/null
+++ b/tests/functional/qemu_test/config.py
@@ -0,0 +1,36 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2018, 2024 Red Hat, Inc.
+#
+# Original Author (Avocado-based tests):
+# Cleber Rosa <crosa@redhat.com>
+#
+# Adaption for standalone version:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+from pathlib import Path
+
+
+def _source_dir():
+ # Determine top-level directory of the QEMU sources
+ return Path(__file__).parent.parent.parent.parent
+
+def _build_dir():
+ root = os.getenv('QEMU_BUILD_ROOT')
+ if root is not None:
+ return Path(root)
+ # Makefile.mtest only exists in build dir, so if it is available, use CWD
+ if os.path.exists('Makefile.mtest'):
+ return Path(os.getcwd())
+
+ root = os.path.join(_source_dir(), 'build')
+ if os.path.exists(root):
+ return Path(root)
+
+ raise Exception("Cannot identify build dir, set QEMU_BUILD_ROOT")
+
+BUILD_DIR = _build_dir()
diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
new file mode 100644
index 0000000000..82cc1d454f
--- /dev/null
+++ b/tests/functional/qemu_test/testcase.py
@@ -0,0 +1,154 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2018, 2024 Red Hat, Inc.
+#
+# Original Author (Avocado-based tests):
+# Cleber Rosa <crosa@redhat.com>
+#
+# Adaption for standalone version:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import logging
+import os
+import pycotap
+import sys
+import unittest
+import uuid
+
+from qemu.machine import QEMUMachine
+from qemu.utils import kvm_available, tcg_available
+
+from .cmd import run_cmd
+from .config import BUILD_DIR
+
+
+class QemuBaseTest(unittest.TestCase):
+
+ qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
+ arch = None
+
+ workdir = None
+ log = logging.getLogger('qemu-test')
+
+ def setUp(self, bin_prefix):
+ self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
+ self.arch = self.qemu_bin.split('-')[-1]
+
+ self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch,
+ self.id())
+ if not os.path.exists(self.workdir):
+ os.makedirs(self.workdir)
+
+ def main():
+ tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
+ test_output_log = pycotap.LogMode.LogToError)
+ path = os.path.basename(sys.argv[0])[:-3]
+ unittest.main(module = None, testRunner = tr, argv=["__dummy__", path])
+
+
+class QemuSystemTest(QemuBaseTest):
+ """Facilitates system emulation tests."""
+
+ cpu = None
+ machine = None
+ _machinehelp = None
+
+ def setUp(self):
+ self._vms = {}
+
+ super().setUp('qemu-system-')
+
+ def set_machine(self, machinename):
+ # TODO: We should use QMP to get the list of available machines
+ if not self._machinehelp:
+ self._machinehelp = run_cmd([self.qemu_bin, '-M', 'help'])[0];
+ if self._machinehelp.find(machinename) < 0:
+ self.skipTest('no support for machine ' + machinename)
+ self.machine = machinename
+
+ def require_accelerator(self, accelerator):
+ """
+ Requires an accelerator to be available for the test to continue
+
+ It takes into account the currently set qemu binary.
+
+ If the check fails, the test is canceled. If the check itself
+ for the given accelerator is not available, the test is also
+ canceled.
+
+ :param accelerator: name of the accelerator, such as "kvm" or "tcg"
+ :type accelerator: str
+ """
+ checker = {'tcg': tcg_available,
+ 'kvm': kvm_available}.get(accelerator)
+ if checker is None:
+ self.skipTest("Don't know how to check for the presence "
+ "of accelerator %s" % accelerator)
+ if not checker(qemu_bin=self.qemu_bin):
+ self.skipTest("%s accelerator does not seem to be "
+ "available" % accelerator)
+
+ def require_netdev(self, netdevname):
+ netdevhelp = run_cmd([self.qemu_bin,
+ '-M', 'none', '-netdev', 'help'])[0];
+ if netdevhelp.find('\n' + netdevname + '\n') < 0:
+ self.skipTest('no support for " + netdevname + " networking')
+
+ def require_device(self, devicename):
+ devhelp = run_cmd([self.qemu_bin,
+ '-M', 'none', '-device', 'help'])[0];
+ if devhelp.find(devicename) < 0:
+ self.skipTest('no support for device ' + devicename)
+
+ def _new_vm(self, name, *args):
+ vm = QEMUMachine(self.qemu_bin, base_temp_dir=self.workdir)
+ self.log.debug('QEMUMachine "%s" created', name)
+ self.log.debug('QEMUMachine "%s" temp_dir: %s', name, vm.temp_dir)
+ self.log.debug('QEMUMachine "%s" log_dir: %s', name, vm.log_dir)
+ if args:
+ vm.add_args(*args)
+ return vm
+
+ @property
+ def vm(self):
+ return self.get_vm(name='default')
+
+ def get_vm(self, *args, name=None):
+ if not name:
+ name = str(uuid.uuid4())
+ if self._vms.get(name) is None:
+ self._vms[name] = self._new_vm(name, *args)
+ if self.cpu is not None:
+ self._vms[name].add_args('-cpu', self.cpu)
+ if self.machine is not None:
+ self._vms[name].set_machine(self.machine)
+ return self._vms[name]
+
+ def set_vm_arg(self, arg, value):
+ """
+ Set an argument to list of extra arguments to be given to the QEMU
+ binary. If the argument already exists then its value is replaced.
+
+ :param arg: the QEMU argument, such as "-cpu" in "-cpu host"
+ :type arg: str
+ :param value: the argument value, such as "host" in "-cpu host"
+ :type value: str
+ """
+ if not arg or not value:
+ return
+ if arg not in self.vm.args:
+ self.vm.args.extend([arg, value])
+ else:
+ idx = self.vm.args.index(arg) + 1
+ if idx < len(self.vm.args):
+ self.vm.args[idx] = value
+ else:
+ self.vm.args.append(value)
+
+ def tearDown(self):
+ for vm in self._vms.values():
+ vm.shutdown()
+ super().tearDown()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 03/24] tests/functional: Set up logging
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 01/24] python: Install pycotap in our venv if necessary Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 10:10 ` Alex Bennée
2024-07-30 17:03 ` [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text Daniel P. Berrangé
` (21 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P . Berrangé
From: Thomas Huth <thuth@redhat.com>
Create log files for each test separately, one file that contains
the basic logging and one that contains the console output.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/qemu_test/testcase.py | 27 +++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
index 82cc1d454f..27bbf4a0af 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -31,7 +31,8 @@ class QemuBaseTest(unittest.TestCase):
arch = None
workdir = None
- log = logging.getLogger('qemu-test')
+ log = None
+ logdir = None
def setUp(self, bin_prefix):
self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
@@ -42,6 +43,20 @@ def setUp(self, bin_prefix):
if not os.path.exists(self.workdir):
os.makedirs(self.workdir)
+ self.logdir = self.workdir
+ self.log = logging.getLogger('qemu-test')
+ self.log.setLevel(logging.DEBUG)
+ self._log_fh = logging.FileHandler(os.path.join(self.logdir,
+ 'base.log'), mode='w')
+ self._log_fh.setLevel(logging.DEBUG)
+ fileFormatter = logging.Formatter(
+ '%(asctime)s - %(levelname)s: %(message)s')
+ self._log_fh.setFormatter(fileFormatter)
+ self.log.addHandler(self._log_fh)
+
+ def tearDown(self):
+ self.log.removeHandler(self._log_fh)
+
def main():
tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
test_output_log = pycotap.LogMode.LogToError)
@@ -61,6 +76,15 @@ def setUp(self):
super().setUp('qemu-system-')
+ console_log = logging.getLogger('console')
+ console_log.setLevel(logging.DEBUG)
+ self._console_log_fh = logging.FileHandler(os.path.join(self.workdir,
+ 'console.log'), mode='w')
+ self._console_log_fh.setLevel(logging.DEBUG)
+ fileFormatter = logging.Formatter('%(asctime)s: %(message)s')
+ self._console_log_fh.setFormatter(fileFormatter)
+ console_log.addHandler(self._console_log_fh)
+
def set_machine(self, machinename):
# TODO: We should use QMP to get the list of available machines
if not self._machinehelp:
@@ -151,4 +175,5 @@ def set_vm_arg(self, arg, value):
def tearDown(self):
for vm in self._vms.values():
vm.shutdown()
+ logging.getLogger('console').removeHandler(self._console_log_fh)
super().tearDown()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (2 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 03/24] tests/functional: Set up logging Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-31 12:49 ` Philippe Mathieu-Daudé
2024-08-01 10:00 ` Alex Bennée
2024-07-30 17:03 ` [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests Daniel P. Berrangé
` (20 subsequent siblings)
24 siblings, 2 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
The next patch is going to add some entries that need more space between
the command and the help text, so let's increase the indentation here
first.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/Makefile.include | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6618bfed70..167d8f29a4 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -3,28 +3,28 @@
.PHONY: check-help
check-help:
@echo "Regression testing targets:"
- @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
- @echo " $(MAKE) bench Run speed tests"
+ @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
+ @echo " $(MAKE) bench Run speed tests"
@echo
@echo "Individual test suites:"
- @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
- @echo " $(MAKE) check-qtest Run qtest tests"
- @echo " $(MAKE) check-unit Run qobject tests"
- @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
- @echo " $(MAKE) check-block Run block tests"
+ @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
+ @echo " $(MAKE) check-qtest Run qtest tests"
+ @echo " $(MAKE) check-unit Run qobject tests"
+ @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
+ @echo " $(MAKE) check-block Run block tests"
ifneq ($(filter $(all-check-targets), check-softfloat),)
- @echo " $(MAKE) check-tcg Run TCG tests"
- @echo " $(MAKE) check-softfloat Run FPU emulation tests"
+ @echo " $(MAKE) check-tcg Run TCG tests"
+ @echo " $(MAKE) check-softfloat Run FPU emulation tests"
endif
- @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets"
+ @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets"
@echo
- @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
- @echo " $(MAKE) check-venv Creates a Python venv for tests"
- @echo " $(MAKE) check-clean Clean the tests and related data"
+ @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
+ @echo " $(MAKE) check-venv Creates a Python venv for tests"
+ @echo " $(MAKE) check-clean Clean the tests and related data"
@echo
@echo "The following are useful for CI builds"
- @echo " $(MAKE) check-build Build most test binaries"
- @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
+ @echo " $(MAKE) check-build Build most test binaries"
+ @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
@echo
@echo
@echo "The variable SPEED can be set to control the gtester speed setting."
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (3 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:12 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 06/24] tests/functional: Convert simple avocado tests into standalone python tests Daniel P. Berrangé
` (19 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Provide a meson.build file for the upcoming python-based functional
tests, and add some wrapper glue targets to the tests/Makefile.include
file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"),
while the "thorough" tests should only be run when running a
"make check-functional" test run (since these tests might download
additional assets from the internet).
The changes to the meson.build files are partly based on an earlier
patch by Ani Sinha.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/Makefile.include | 11 ++++++
tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
tests/meson.build | 1 +
3 files changed, 78 insertions(+)
create mode 100644 tests/functional/meson.build
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 167d8f29a4..d0c2ae30b7 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -9,6 +9,8 @@ check-help:
@echo "Individual test suites:"
@echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
@echo " $(MAKE) check-qtest Run qtest tests"
+ @echo " $(MAKE) check-functional Run python-based functional tests"
+ @echo " $(MAKE) check-functional-TARGET Run functional tests for a given target"
@echo " $(MAKE) check-unit Run qobject tests"
@echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
@echo " $(MAKE) check-block Run block tests"
@@ -152,6 +154,15 @@ check-acceptance-deprecated-warning:
check-acceptance: check-acceptance-deprecated-warning | check-avocado
+FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
+.PHONY: $(FUNCTIONAL_TARGETS)
+$(FUNCTIONAL_TARGETS):
+ @$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
+
+.PHONY: check-functional
+check-functional:
+ @$(MAKE) SPEED=thorough check-func check-func-quick
+
# Consolidated targets
.PHONY: check check-clean get-vm-images
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
new file mode 100644
index 0000000000..7fc9c4e513
--- /dev/null
+++ b/tests/functional/meson.build
@@ -0,0 +1,66 @@
+# QEMU functional tests:
+# Tests that are put in the 'quick' category are run by default during
+# 'make check'. Everything that should not be run during 'make check'
+# (e.g. tests that fetch assets from the internet) should be put into
+# the 'thorough' category instead.
+
+# Most tests run too slow with TCI enabled, so skip the functional tests there
+if get_option('tcg_interpreter')
+ subdir_done()
+endif
+
+# Timeouts for individual tests that can be slow e.g. with debugging enabled
+test_timeouts = {
+}
+
+tests_generic = [
+]
+
+tests_x86_64_quick = [
+]
+
+tests_x86_64_thorough = [
+]
+
+foreach speed : ['quick', 'thorough']
+ foreach dir : target_dirs
+ if not dir.endswith('-softmmu')
+ continue
+ endif
+
+ target_base = dir.split('-')[0]
+ test_emulator = emulators['qemu-system-' + target_base]
+
+ if speed == 'quick'
+ suites = ['func-quick', 'func-' + target_base]
+ target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic
+ else
+ suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
+ target_tests = get_variable('tests_' + target_base + '_' + speed, [])
+ endif
+
+ test_deps = roms
+ test_env = environment()
+ if have_tools
+ test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img')
+ test_deps += [qemu_img]
+ endif
+ test_env.set('QEMU_TEST_QEMU_BINARY',
+ meson.global_build_root() / 'qemu-system-' + target_base)
+ test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())
+ test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
+ meson.current_source_dir())
+
+ foreach test : target_tests
+ test('func-@0@/@1@'.format(target_base, test),
+ python,
+ depends: [test_deps, test_emulator, emulator_modules],
+ env: test_env,
+ args: [meson.current_source_dir() / 'test_' + test + '.py'],
+ protocol: 'tap',
+ timeout: test_timeouts.get(test, 60),
+ priority: test_timeouts.get(test, 60),
+ suite: suites)
+ endforeach
+ endforeach
+endforeach
diff --git a/tests/meson.build b/tests/meson.build
index acb6807094..3345ad2098 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -85,3 +85,4 @@ subdir('unit')
subdir('qapi-schema')
subdir('qtest')
subdir('migration')
+subdir('functional')
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 06/24] tests/functional: Convert simple avocado tests into standalone python tests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (4 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-31 12:52 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment Daniel P. Berrangé
` (18 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P . Berrangé
From: Thomas Huth <thuth@redhat.com>
These test are rather simple and don't need any modifications apart
from adjusting the "from avocado_qemu" line. To ease debugging, make
the files executable and add a shebang line and Python '__main__'
handling, too, so that these tests can now be run by executing them
directly.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/meson.build | 5 ++
.../test_cpu_queries.py} | 7 ++-
.../test_empty_cpu_model.py} | 7 ++-
.../test_mem_addr_space.py} | 52 +++----------------
.../test_pc_cpu_hotplug_props.py} | 11 ++--
.../test_virtio_version.py} | 8 +--
6 files changed, 34 insertions(+), 56 deletions(-)
rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%)
mode change 100644 => 100755
rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
mode change 100644 => 100755
rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
mode change 100644 => 100755
rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
mode change 100644 => 100755
rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
mode change 100644 => 100755
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 7fc9c4e513..a8cc0e6330 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -14,9 +14,14 @@ test_timeouts = {
}
tests_generic = [
+ 'empty_cpu_model',
]
tests_x86_64_quick = [
+ 'cpu_queries',
+ 'mem_addr_space',
+ 'pc_cpu_hotplug_props',
+ 'virtio_version',
]
tests_x86_64_thorough = [
diff --git a/tests/avocado/cpu_queries.py b/tests/functional/test_cpu_queries.py
old mode 100644
new mode 100755
similarity index 89%
rename from tests/avocado/cpu_queries.py
rename to tests/functional/test_cpu_queries.py
index d3faa14720..97dd600c01
--- a/tests/avocado/cpu_queries.py
+++ b/tests/functional/test_cpu_queries.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Sanity check of query-cpu-* results
#
# Copyright (c) 2019 Red Hat, Inc.
@@ -8,7 +10,7 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class QueryCPUModelExpansion(QemuSystemTest):
"""
@@ -33,3 +35,6 @@ def test(self):
e = self.vm.cmd('query-cpu-model-expansion', model=model,
type='full')
self.assertEqual(e['model']['name'], c['name'])
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/empty_cpu_model.py b/tests/functional/test_empty_cpu_model.py
old mode 100644
new mode 100755
similarity index 84%
rename from tests/avocado/empty_cpu_model.py
rename to tests/functional/test_empty_cpu_model.py
index d906ef3d3c..0081b06d85
--- a/tests/avocado/empty_cpu_model.py
+++ b/tests/functional/test_empty_cpu_model.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Check for crash when using empty -cpu option
#
# Copyright (c) 2019 Red Hat, Inc.
@@ -7,7 +9,7 @@
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class EmptyCPUModel(QemuSystemTest):
def test(self):
@@ -17,3 +19,6 @@ def test(self):
self.vm.wait()
self.assertEqual(self.vm.exitcode(), 1, "QEMU exit code should be 1")
self.assertRegex(self.vm.get_log(), r'-cpu option cannot be empty')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/mem-addr-space-check.py b/tests/functional/test_mem_addr_space.py
old mode 100644
new mode 100755
similarity index 93%
rename from tests/avocado/mem-addr-space-check.py
rename to tests/functional/test_mem_addr_space.py
index d3974599f4..bb0cf062ca
--- a/tests/avocado/mem-addr-space-check.py
+++ b/tests/functional/test_mem_addr_space.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Check for crash when using memory beyond the available guest processor
# address space.
#
@@ -8,7 +10,7 @@
#
# SPDX-License-Identifier: GPL-2.0-or-later
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
import time
class MemAddrCheck(QemuSystemTest):
@@ -22,9 +24,6 @@ class MemAddrCheck(QemuSystemTest):
# for all 32-bit cases, pci64_hole_size is 0.
def test_phybits_low_pse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
With pse36 feature ON, a processor has 36 bits of addressing. So it can
access up to a maximum of 64GiB of memory. Memory hotplug region begins
at 4 GiB boundary when "above_4g_mem_size" is 0 (this would be true when
@@ -52,9 +51,6 @@ def test_phybits_low_pse36(self):
def test_phybits_low_pae(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
With pae feature ON, a processor has 36 bits of addressing. So it can
access up to a maximum of 64GiB of memory. Rest is the same as the case
with pse36 above.
@@ -72,9 +68,6 @@ def test_phybits_low_pae(self):
def test_phybits_ok_pentium_pse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Setting maxmem to 59.5G and making sure that QEMU can start with the
same options as the failing case above with pse36 cpu feature.
"""
@@ -91,9 +84,6 @@ def test_phybits_ok_pentium_pse36(self):
def test_phybits_ok_pentium_pae(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Test is same as above but now with pae cpu feature turned on.
Setting maxmem to 59.5G and making sure that QEMU can start fine
with the same options as the case above.
@@ -111,9 +101,6 @@ def test_phybits_ok_pentium_pae(self):
def test_phybits_ok_pentium2(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Pentium2 has 36 bits of addressing, so its same as pentium
with pse36 ON.
"""
@@ -130,9 +117,6 @@ def test_phybits_ok_pentium2(self):
def test_phybits_low_nonpse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Pentium processor has 32 bits of addressing without pse36 or pae
so it can access physical address up to 4 GiB. Setting maxmem to
4 GiB should make QEMU fail to start with "phys-bits too low"
@@ -153,9 +137,6 @@ def test_phybits_low_nonpse36(self):
# now lets test some 64-bit CPU cases.
def test_phybits_low_tcg_q35_70_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
For q35 7.1 machines and above, there is a HT window that starts at
1024 GiB and ends at 1 TiB - 1. If the max GPA falls in this range,
"above_4G" memory is adjusted to start at 1 TiB boundary for AMD cpus
@@ -182,9 +163,6 @@ def test_phybits_low_tcg_q35_70_amd(self):
def test_phybits_low_tcg_q35_71_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD_HT_START is defined to be at 1012 GiB. So for q35 machines
version > 7.0 and AMD cpus, instead of 1024 GiB limit for 40 bit
processor address space, it has to be 1012 GiB , that is 12 GiB
@@ -205,9 +183,6 @@ def test_phybits_low_tcg_q35_71_amd(self):
def test_phybits_ok_tcg_q35_70_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as q35-7.0 AMD case except that here we check that QEMU can
successfully start when maxmem is < 988G.
"""
@@ -224,9 +199,6 @@ def test_phybits_ok_tcg_q35_70_amd(self):
def test_phybits_ok_tcg_q35_71_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as q35-7.1 AMD case except that here we check that QEMU can
successfully start when maxmem is < 976G.
"""
@@ -243,9 +215,6 @@ def test_phybits_ok_tcg_q35_71_amd(self):
def test_phybits_ok_tcg_q35_71_intel(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same parameters as test_phybits_low_tcg_q35_71_amd() but use
Intel cpu instead. QEMU should start fine in this case as
"above_4G" memory starts at 4G.
@@ -264,9 +233,6 @@ def test_phybits_ok_tcg_q35_71_intel(self):
def test_phybits_low_tcg_q35_71_amd_41bits(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD processor with 41 bits. Max cpu hw address = 2 TiB.
By setting maxram above 1012 GiB - 32 GiB - 4 GiB = 976 GiB, we can
force "above_4G" memory to start at 1 TiB for q35-7.1 machines
@@ -291,9 +257,6 @@ def test_phybits_low_tcg_q35_71_amd_41bits(self):
def test_phybits_ok_tcg_q35_71_amd_41bits(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD processor with 41 bits. Max cpu hw address = 2 TiB.
Same as above but by setting maxram between 976 GiB and 992 Gib,
QEMU should start fine.
@@ -312,9 +275,6 @@ def test_phybits_ok_tcg_q35_71_amd_41bits(self):
def test_phybits_low_tcg_q35_intel_cxl(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
cxl memory window starts after memory device range. Here, we use 1 GiB
of cxl window memory. 4G_mem end aligns at 4G. pci64_hole is 32 GiB and
starts after the cxl memory window.
@@ -335,9 +295,6 @@ def test_phybits_low_tcg_q35_intel_cxl(self):
def test_phybits_ok_tcg_q35_intel_cxl(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as above but here we do not reserve any cxl memory window. Hence,
with the exact same parameters as above, QEMU should start fine even
with cxl enabled.
@@ -352,3 +309,6 @@ def test_phybits_ok_tcg_q35_intel_cxl(self):
time.sleep(self.DELAY_Q35_BOOT_SEQUENCE)
self.vm.shutdown()
self.assertNotRegex(self.vm.get_log(), r'phys-bits too low')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/pc_cpu_hotplug_props.py b/tests/functional/test_pc_cpu_hotplug_props.py
old mode 100644
new mode 100755
similarity index 90%
rename from tests/avocado/pc_cpu_hotplug_props.py
rename to tests/functional/test_pc_cpu_hotplug_props.py
index 4bd3e02665..9d5a37cb17
--- a/tests/avocado/pc_cpu_hotplug_props.py
+++ b/tests/functional/test_pc_cpu_hotplug_props.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# Ensure CPU die-id can be omitted on -device
#
@@ -20,16 +21,16 @@
# License along with this library; if not, see <http://www.gnu.org/licenses/>.
#
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class OmittedCPUProps(QemuSystemTest):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=cpu:qemu64
- """
+
def test_no_die_id(self):
self.vm.add_args('-nodefaults', '-S')
self.vm.add_args('-smp', '1,sockets=2,cores=2,threads=2,maxcpus=8')
self.vm.add_args('-device', 'qemu64-x86_64-cpu,socket-id=1,core-id=0,thread-id=0')
self.vm.launch()
self.assertEqual(len(self.vm.cmd('query-cpus-fast')), 2)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/virtio_version.py b/tests/functional/test_virtio_version.py
old mode 100644
new mode 100755
similarity index 98%
rename from tests/avocado/virtio_version.py
rename to tests/functional/test_virtio_version.py
index afe5e828b5..eb23060564
--- a/tests/avocado/virtio_version.py
+++ b/tests/functional/test_virtio_version.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
"""
Check compatibility of virtio device types
"""
@@ -12,7 +13,7 @@
import os
from qemu.machine import QEMUMachine
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
# Virtio Device IDs:
VIRTIO_NET = 1
@@ -60,8 +61,6 @@ class VirtioVersionCheck(QemuSystemTest):
Check if virtio-version-specific device types result in the
same device tree created by `disable-modern` and
`disable-legacy`.
-
- :avocado: tags=arch:x86_64
"""
# just in case there are failures, show larger diff:
@@ -173,3 +172,6 @@ def test_modern_only_devs(self):
self.check_modern_only('virtio-mouse-pci', VIRTIO_INPUT)
self.check_modern_only('virtio-tablet-pci', VIRTIO_INPUT)
self.check_modern_only('virtio-keyboard-pci', VIRTIO_INPUT)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (5 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 06/24] tests/functional: Convert simple avocado tests into standalone python tests Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-31 12:55 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching Daniel P. Berrangé
` (17 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P . Berrangé
From: Thomas Huth <thuth@redhat.com>
These simple tests can be converted to stand-alone tests quite easily,
e.g. by just setting the machine to 'none' now manually or by adding
"-cpu" command line parameters, since we don't support the corresponding
avocado tags in the new python test framework.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/meson.build | 6 ++
.../test_info_usernet.py} | 11 ++-
.../test_ppc_74xx.py} | 74 ++++++++-----------
.../version.py => functional/test_version.py} | 13 ++--
4 files changed, 53 insertions(+), 51 deletions(-)
rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
mode change 100644 => 100755
rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
mode change 100644 => 100755
rename tests/{avocado/version.py => functional/test_version.py} (78%)
mode change 100644 => 100755
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index a8cc0e6330..8a8fa0ab99 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -15,6 +15,12 @@ test_timeouts = {
tests_generic = [
'empty_cpu_model',
+ 'info_usernet',
+ 'version',
+]
+
+tests_ppc_quick = [
+ 'ppc_74xx',
]
tests_x86_64_quick = [
diff --git a/tests/avocado/info_usernet.py b/tests/functional/test_info_usernet.py
old mode 100644
new mode 100755
similarity index 87%
rename from tests/avocado/info_usernet.py
rename to tests/functional/test_info_usernet.py
index e1aa7a6e0a..ded973253a
--- a/tests/avocado/info_usernet.py
+++ b/tests/functional/test_info_usernet.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test for the hmp command "info usernet"
#
# Copyright (c) 2021 Red Hat, Inc.
@@ -8,18 +10,16 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
from qemu.utils import get_info_usernet_hostfwd_port
class InfoUsernet(QemuSystemTest):
- """
- :avocado: tags=machine:none
- """
def test_hostfwd(self):
self.require_netdev('user')
+ self.machine = 'none'
self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
@@ -31,3 +31,6 @@ def test_hostfwd(self):
self.assertGreater(port, 0,
('Found a redirected port that is not greater than'
' zero'))
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_74xx.py b/tests/functional/test_ppc_74xx.py
old mode 100644
new mode 100755
similarity index 74%
rename from tests/avocado/ppc_74xx.py
rename to tests/functional/test_ppc_74xx.py
index f54757c243..5386016f26
--- a/tests/avocado/ppc_74xx.py
+++ b/tests/functional/test_ppc_74xx.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Smoke tests for 74xx cpus (aka G4).
#
# Copyright (c) 2021, IBM Corp.
@@ -5,132 +7,120 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest
+from qemu_test import wait_for_console_pattern
class ppc74xxCpu(QemuSystemTest):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=accel:tcg
- """
+
timeout = 5
def test_ppc_7400(self):
- """
- :avocado: tags=cpu:7400
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7400')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7410(self):
- """
- :avocado: tags=cpu:7410
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7410')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,74xx')
def test_ppc_7441(self):
- """
- :avocado: tags=cpu:7441
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7441')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7445(self):
- """
- :avocado: tags=cpu:7445
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7445')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7447(self):
- """
- :avocado: tags=cpu:7447
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7447')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7447a(self):
- """
- :avocado: tags=cpu:7447a
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7447a')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7448(self):
- """
- :avocado: tags=cpu:7448
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7448')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,MPC86xx')
def test_ppc_7450(self):
- """
- :avocado: tags=cpu:7450
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7450')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7451(self):
- """
- :avocado: tags=cpu:7451
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7451')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7455(self):
- """
- :avocado: tags=cpu:7455
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7455')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7457(self):
- """
- :avocado: tags=cpu:7457
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7457')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7457a(self):
- """
- :avocado: tags=cpu:7457a
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7457a')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/version.py b/tests/functional/test_version.py
old mode 100644
new mode 100755
similarity index 78%
rename from tests/avocado/version.py
rename to tests/functional/test_version.py
index c6139568a1..5e566d76b1
--- a/tests/avocado/version.py
+++ b/tests/functional/test_version.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Version check example test
#
# Copyright (c) 2018 Red Hat, Inc.
@@ -9,17 +11,18 @@
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class Version(QemuSystemTest):
- """
- :avocado: tags=quick
- :avocado: tags=machine:none
- """
+
def test_qmp_human_info_version(self):
+ self.machine = 'none'
self.vm.add_args('-nodefaults')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
command_line='info version')
self.assertRegex(res, r'^(\d+\.\d+\.\d)')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (6 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:20 ` Philippe Mathieu-Daudé
2024-08-01 21:51 ` Richard Henderson
2024-07-30 17:03 ` [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets Daniel P. Berrangé
` (16 subsequent siblings)
24 siblings, 2 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P. Berrangé
The 'Asset' class is a simple module that declares a downloadable
asset that can be cached locally. Downloads are stored in the user's
home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/functional/qemu_test/__init__.py | 1 +
tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++
2 files changed, 97 insertions(+)
create mode 100644 tests/functional/qemu_test/asset.py
diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
index 2f1e0bc70d..db05c8f412 100644
--- a/tests/functional/qemu_test/__init__.py
+++ b/tests/functional/qemu_test/__init__.py
@@ -6,6 +6,7 @@
# later. See the COPYING file in the top-level directory.
+from .asset import Asset
from .config import BUILD_DIR
from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
new file mode 100644
index 0000000000..6432da2e0b
--- /dev/null
+++ b/tests/functional/qemu_test/asset.py
@@ -0,0 +1,96 @@
+# Test utilities for fetching & caching assets
+#
+# Copyright 2024 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import hashlib
+import logging
+import os
+import subprocess
+import urllib.request
+from pathlib import Path
+from shutil import copyfileobj
+
+
+# Instances of this class must be declared as class level variables
+# starting with a name "ASSET_". This enables the pre-caching logic
+# to easily find all referenced assets and download them prior to
+# execution of the tests.
+class Asset:
+
+ def __init__(self, url, hash):
+ self.url = url
+ self.hash = hash
+ self.cache_dir = Path(Path("~").expanduser(),
+ ".cache", "qemu", "download")
+ self.cache_file = Path(self.cache_dir,
+ hashlib.sha256(url.encode("utf-8")).hexdigest())
+ self.log = logging.getLogger('qemu-test')
+
+ def __repr__(self):
+ return "Asset: url=%s hash=%s cache=%s" % (
+ self.url, self.hash, self.cache_file)
+
+ def _check(self, cache_file):
+ if self.hash is None:
+ return True
+ if len(self.hash) == 40:
+ sum_prog = 'sha1sum'
+ elif len(self.hash) == 64:
+ sum_prog = 'sha256sum'
+ elif len(self.hash) == 128:
+ sum_prog = 'sha512sum'
+ else:
+ raise Exception("unknown hash type")
+
+ checksum = subprocess.check_output(
+ [sum_prog, str(cache_file)]).split()[0]
+ return self.hash == checksum.decode("utf-8")
+
+ def valid(self):
+ return self.cache_file.exists() and self._check(self.cache_file)
+
+ def fetch(self):
+ if not self.cache_dir.exists():
+ self.cache_dir.mkdir(parents=True, exist_ok=True)
+
+ if self.valid():
+ self.log.debug("Using cached asset %s for %s",
+ self.cache_file, self.url)
+ return str(self.cache_file)
+
+ self.log.info("Downloading %s to %s...", self.url, self.cache_file)
+ tmp_cache_file = self.cache_file.with_suffix(".download")
+
+ try:
+ resp = urllib.request.urlopen(self.url)
+ except Exception as e:
+ self.log.error("Unable to download %s: %s", self.url, e)
+ raise
+
+ try:
+ with tmp_cache_file.open("wb+") as dst:
+ copyfileobj(resp, dst)
+ except:
+ tmp_cache_file.unlink()
+ raise
+ try:
+ # Set these just for informational purposes
+ os.setxattr(str(tmp_cache_file), "user.qemu-asset-url",
+ self.url.encode('utf8'))
+ os.setxattr(str(tmp_cache_file), "user.qemu-asset-hash",
+ self.hash.encode('utf8'))
+ except Exception as e:
+ self.log.info("Unable to set xattr on %s: %s", tmp_cache_file, e)
+ pass
+
+ if not self._check(tmp_cache_file):
+ tmp_cache_file.unlink()
+ raise Exception("Hash of %s does not match %s" %
+ (self.url, self.hash))
+ tmp_cache_file.replace(self.cache_file)
+
+ self.log.info("Cached %s at %s" % (self.url, self.cache_file))
+ return str(self.cache_file)
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (7 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-31 5:01 ` Richard Henderson
2024-08-01 16:01 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 10/24] tests/functional: Convert some tests that download files via fetch_asset() Daniel P. Berrangé
` (15 subsequent siblings)
24 siblings, 2 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P. Berrangé
Many tests need to access assets stored on remote sites. We don't want
to download these during test execution when run by meson, since this
risks hitting test timeouts when data transfers are slow.
Add support for pre-emptive caching of assets by setting the env var
QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
instead of running the test, the assets will be downloaded and saved
to the cache, then the timestamp file created.
A meson custom target is created as a dependency of each test suite
to trigger the pre-emptive caching logic before the test runs.
When run in caching mode, it will locate assets by looking for class
level variables with a name prefix "ASSET_", and type "Asset".
At the ninja level
ninja test --suite functional
will speculatively download any assets that are not already cached,
so it is advisable to set a timeout multiplier.
QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
will fail the test if a required asset is not already cached
nina check-func-precache
will download and cache all assets required by the functional
tests
At the make level, precaching is always done by
make check-functional
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/Makefile.include | 3 ++-
tests/functional/meson.build | 33 +++++++++++++++++++++++--
tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++
tests/functional/qemu_test/testcase.py | 8 ++++++
4 files changed, 75 insertions(+), 3 deletions(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index d0c2ae30b7..dbc8fc4678 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -161,7 +161,8 @@ $(FUNCTIONAL_TARGETS):
.PHONY: check-functional
check-functional:
- @$(MAKE) SPEED=thorough check-func check-func-quick
+ @$(MAKE) precache-functional
+ @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
# Consolidated targets
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 8a8fa0ab99..cef74b82a9 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -33,6 +33,7 @@ tests_x86_64_quick = [
tests_x86_64_thorough = [
]
+precache_all = []
foreach speed : ['quick', 'thorough']
foreach dir : target_dirs
if not dir.endswith('-softmmu')
@@ -63,11 +64,35 @@ foreach speed : ['quick', 'thorough']
meson.current_source_dir())
foreach test : target_tests
- test('func-@0@/@1@'.format(target_base, test),
+ testname = '@0@-@1@'.format(target_base, test)
+ testfile = 'test_' + test + '.py'
+ testpath = meson.current_source_dir() / testfile
+ teststamp = testname + '.tstamp'
+ test_precache_env = environment()
+ test_precache_env.set('QEMU_TEST_PRECACHE', meson.current_build_dir() / teststamp)
+ test_precache_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
+ meson.current_source_dir())
+ precache = custom_target('func-precache-' + testname,
+ output: teststamp,
+ command: [python, testpath],
+ depend_files: files(testpath),
+ build_by_default: false,
+ env: test_precache_env)
+ precache_all += precache
+
+ # Ideally we would add 'precache' to 'depends' here, such that
+ # 'build_by_default: false' lets the pre-caching automatically
+ # run immediately before the test runs. In practice this is
+ # broken in meson, with it running the pre-caching in the normal
+ # compile phase https://github.com/mesonbuild/meson/issues/2518
+ # If the above bug ever gets fixed, when QEMU changes the min
+ # meson version, add the 'depends' and remove the custom
+ # 'run_target' logic below & in Makefile.include
+ test('func-' + testname,
python,
depends: [test_deps, test_emulator, emulator_modules],
env: test_env,
- args: [meson.current_source_dir() / 'test_' + test + '.py'],
+ args: [testpath],
protocol: 'tap',
timeout: test_timeouts.get(test, 60),
priority: test_timeouts.get(test, 60),
@@ -75,3 +100,7 @@ foreach speed : ['quick', 'thorough']
endforeach
endforeach
endforeach
+
+run_target('precache-functional',
+ depends: precache_all,
+ command: ['true'])
diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index 6432da2e0b..6197d907eb 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -9,6 +9,8 @@
import logging
import os
import subprocess
+import sys
+import unittest
import urllib.request
from pathlib import Path
from shutil import copyfileobj
@@ -61,6 +63,9 @@ def fetch(self):
self.cache_file, self.url)
return str(self.cache_file)
+ if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
+ raise Exception("Asset cache is invalid and downloads disabled")
+
self.log.info("Downloading %s to %s...", self.url, self.cache_file)
tmp_cache_file = self.cache_file.with_suffix(".download")
@@ -94,3 +99,32 @@ def fetch(self):
self.log.info("Cached %s at %s" % (self.url, self.cache_file))
return str(self.cache_file)
+
+ def precache_test(test):
+ log = logging.getLogger('qemu-test')
+ log.setLevel(logging.DEBUG)
+ handler = logging.StreamHandler(sys.stdout)
+ handler.setLevel(logging.DEBUG)
+ formatter = logging.Formatter(
+ '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+ handler.setFormatter(formatter)
+ log.addHandler(handler)
+ for name, asset in vars(test.__class__).items():
+ if name.startswith("ASSET_") and type(asset) == Asset:
+ log.info("Attempting to cache '%s'" % asset)
+ asset.fetch()
+ log.removeHandler(handler)
+
+ def precache_suite(suite):
+ for test in suite:
+ if isinstance(test, unittest.TestSuite):
+ Asset.precache_suite(test)
+ elif isinstance(test, unittest.TestCase):
+ Asset.precache_test(test)
+
+ def precache_suites(path, cacheTstamp):
+ loader = unittest.loader.defaultTestLoader
+ tests = loader.loadTestsFromNames([path], None)
+
+ with open(cacheTstamp, "w") as fh:
+ Asset.precache_suite(tests)
diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
index 27bbf4a0af..21dbd90613 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -21,6 +21,7 @@
from qemu.machine import QEMUMachine
from qemu.utils import kvm_available, tcg_available
+from .asset import Asset
from .cmd import run_cmd
from .config import BUILD_DIR
@@ -58,6 +59,13 @@ def tearDown(self):
self.log.removeHandler(self._log_fh)
def main():
+ path = os.path.basename(sys.argv[0])[:-3]
+
+ cache = os.environ.get("QEMU_TEST_PRECACHE", None)
+ if cache is not None:
+ Asset.precache_suites(path, cache)
+ return
+
tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
test_output_log = pycotap.LogMode.LogToError)
path = os.path.basename(sys.argv[0])[:-3]
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 10/24] tests/functional: Convert some tests that download files via fetch_asset()
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (8 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 11/24] tests/functional: Add a function for extracting files from an archive Daniel P. Berrangé
` (14 subsequent siblings)
24 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Now that we've got the Asset class with pre-caching, we can convert
some Avocado tests that use fetch_asset() for downloading their
required files.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 12 ++---
tests/avocado/machine_mips_loongson3v.py | 39 ---------------
tests/functional/meson.build | 22 ++++++++
.../test_arm_n8x0.py} | 36 ++++++-------
.../test_avr_mega2560.py} | 22 ++++----
.../test_loongarch64_virt.py} | 46 +++++++++--------
tests/functional/test_mips64el_loongson3v.py | 39 +++++++++++++++
.../test_netdev_ethtool.py} | 50 +++++++------------
.../ppc_405.py => functional/test_ppc_405.py} | 28 +++++------
9 files changed, 154 insertions(+), 140 deletions(-)
delete mode 100644 tests/avocado/machine_mips_loongson3v.py
rename tests/{avocado/machine_arm_n8x0.py => functional/test_arm_n8x0.py} (51%)
mode change 100644 => 100755
rename tests/{avocado/machine_avr6.py => functional/test_avr_mega2560.py} (73%)
mode change 100644 => 100755
rename tests/{avocado/machine_loongarch.py => functional/test_loongarch64_virt.py} (54%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_mips64el_loongson3v.py
rename tests/{avocado/netdev-ethtool.py => functional/test_netdev_ethtool.py} (66%)
mode change 100644 => 100755
rename tests/{avocado/ppc_405.py => functional/test_ppc_405.py} (53%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 98eddf7ae1..c2b4352edd 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -222,7 +222,7 @@ S: Maintained
F: docs/system/target-avr.rst
F: gdb-xml/avr-cpu.xml
F: target/avr/
-F: tests/avocado/machine_avr6.py
+F: tests/functional/test_avr_mega2560.py
CRIS TCG CPUs
M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
@@ -265,7 +265,7 @@ M: Song Gao <gaosong@loongson.cn>
S: Maintained
F: target/loongarch/
F: tests/tcg/loongarch64/
-F: tests/avocado/machine_loongarch.py
+F: tests/functional/test_loongarch64_virt.py
M68K TCG CPUs
M: Laurent Vivier <laurent@vivier.eu>
@@ -884,7 +884,7 @@ F: include/hw/display/blizzard.h
F: include/hw/input/lm832x.h
F: include/hw/input/tsc2xxx.h
F: include/hw/misc/cbus.h
-F: tests/avocado/machine_arm_n8x0.py
+F: tests/functional/test_arm_n8x0.py
F: docs/system/arm/nseries.rst
Palm
@@ -1393,7 +1393,7 @@ F: hw/mips/loongson3_bootp.h
F: hw/mips/loongson3_virt.c
F: include/hw/intc/loongson_ipi.h
F: include/hw/intc/loongson_liointc.h
-F: tests/avocado/machine_mips_loongson3v.py
+F: tests/functional/test_mips64el_loongson3v.py
Boston
M: Paul Burton <paulburton@kernel.org>
@@ -1419,7 +1419,7 @@ PowerPC Machines
L: qemu-ppc@nongnu.org
S: Orphan
F: hw/ppc/ppc405*
-F: tests/avocado/ppc_405.py
+F: tests/functional/test_ppc_405.py
Bamboo
L: qemu-ppc@nongnu.org
@@ -2486,7 +2486,7 @@ R: Sriram Yagnaraman <sriram.yagnaraman@ericsson.com>
S: Maintained
F: docs/system/devices/igb.rst
F: hw/net/igb*
-F: tests/avocado/netdev-ethtool.py
+F: tests/functional/test_netdev_ethtool.py
F: tests/qtest/igb-test.c
F: tests/qtest/libqos/igb.c
diff --git a/tests/avocado/machine_mips_loongson3v.py b/tests/avocado/machine_mips_loongson3v.py
deleted file mode 100644
index 5194cf18c9..0000000000
--- a/tests/avocado/machine_mips_loongson3v.py
+++ /dev/null
@@ -1,39 +0,0 @@
-# Functional tests for the Generic Loongson-3 Platform.
-#
-# Copyright (c) 2021 Jiaxun Yang <jiaxun.yang@flygoat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later.
-# See the COPYING file in the top-level directory.
-#
-# SPDX-License-Identifier: GPL-2.0-or-later
-
-import os
-import time
-
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-
-class MipsLoongson3v(QemuSystemTest):
- timeout = 60
-
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
- def test_pmon_serial_console(self):
- """
- :avocado: tags=arch:mips64el
- :avocado: tags=endian:little
- :avocado: tags=machine:loongson3-virt
- :avocado: tags=cpu:Loongson-3A1000
- :avocado: tags=device:liointc
- :avocado: tags=device:goldfish_rtc
- """
-
- pmon_hash = '7c8b45dd81ccfc55ff28f5aa267a41c3'
- pmon_path = self.fetch_asset('https://github.com/loongson-community/pmon/'
- 'releases/download/20210112/pmon-3avirt.bin',
- asset_hash=pmon_hash, algorithm='md5')
-
- self.vm.set_console()
- self.vm.add_args('-bios', pmon_path)
- self.vm.launch()
- wait_for_console_pattern(self, 'CPU GODSON3 BogoMIPS:')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index cef74b82a9..32c7cc2d3c 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -11,6 +11,7 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
+ 'netdev_ethtool' : 180,
}
tests_generic = [
@@ -19,10 +20,30 @@ tests_generic = [
'version',
]
+tests_arm_thorough = [
+ 'arm_n8x0',
+]
+
+tests_avr_thorough = [
+ 'avr_mega2560',
+]
+
+tests_loongarch64_thorough = [
+ 'loongarch64_virt',
+]
+
+tests_mips64el_thorough = [
+ 'mips64el_loongson3v',
+]
+
tests_ppc_quick = [
'ppc_74xx',
]
+tests_ppc_thorough = [
+ 'ppc_405',
+]
+
tests_x86_64_quick = [
'cpu_queries',
'mem_addr_space',
@@ -31,6 +52,7 @@ tests_x86_64_quick = [
]
tests_x86_64_thorough = [
+ 'netdev_ethtool',
]
precache_all = []
diff --git a/tests/avocado/machine_arm_n8x0.py b/tests/functional/test_arm_n8x0.py
old mode 100644
new mode 100755
similarity index 51%
rename from tests/avocado/machine_arm_n8x0.py
rename to tests/functional/test_arm_n8x0.py
index 12e9a6803b..5dc473f66f
--- a/tests/avocado/machine_arm_n8x0.py
+++ b/tests/functional/test_arm_n8x0.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a Linux kernel and checks the console
#
# Copyright (c) 2020 Red Hat, Inc.
@@ -10,21 +12,22 @@
import os
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class N8x0Machine(QemuSystemTest):
"""Boots the Linux kernel and checks that the console is operational"""
timeout = 90
+ ASSET_KERNEL = Asset(('http://stskeeps.subnetmask.net/meego-n8x0/'
+ 'meego-arm-n8x0-1.0.80.20100712.1431-'
+ 'vmlinuz-2.6.35~rc4-129.1-n8x0'),
+ 'e9d5ab8d7548923a0061b6fbf601465e479ed269')
+
def __do_test_n8x0(self):
- kernel_url = ('http://stskeeps.subnetmask.net/meego-n8x0/'
- 'meego-arm-n8x0-1.0.80.20100712.1431-'
- 'vmlinuz-2.6.35~rc4-129.1-n8x0')
- kernel_hash = 'e9d5ab8d7548923a0061b6fbf601465e479ed269'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console(console_index=1)
self.vm.add_args('-kernel', kernel_path,
@@ -32,18 +35,15 @@ def __do_test_n8x0(self):
self.vm.launch()
wait_for_console_pattern(self, 'TSC2005 driver initializing')
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+ @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
def test_n800(self):
- """
- :avocado: tags=arch:arm
- :avocado: tags=machine:n800
- """
+ self.set_machine('n800')
self.__do_test_n8x0()
- @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+ @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
def test_n810(self):
- """
- :avocado: tags=arch:arm
- :avocado: tags=machine:n810
- """
+ self.set_machine('n810')
self.__do_test_n8x0()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/machine_avr6.py b/tests/functional/test_avr_mega2560.py
old mode 100644
new mode 100755
similarity index 73%
rename from tests/avocado/machine_avr6.py
rename to tests/functional/test_avr_mega2560.py
index 5485db79c6..a2d026e03a
--- a/tests/avocado/machine_avr6.py
+++ b/tests/functional/test_avr_mega2560.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# QEMU AVR integration tests
#
@@ -19,26 +20,24 @@
import time
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest, Asset
class AVR6Machine(QemuSystemTest):
timeout = 5
+ ASSET_ROM = Asset(('https://github.com/seharris/qemu-avr-tests'
+ '/raw/36c3e67b8755dcf/free-rtos/Demo'
+ '/AVR_ATMega2560_GCC/demo.elf'),
+ '7eb521f511ca8f2622e0a3c5e8dd686efbb911d4')
+
def test_freertos(self):
- """
- :avocado: tags=arch:avr
- :avocado: tags=machine:arduino-mega-2560-v3
- """
"""
https://github.com/seharris/qemu-avr-tests/raw/master/free-rtos/Demo/AVR_ATMega2560_GCC/demo.elf
constantly prints out 'ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX'
"""
- rom_url = ('https://github.com/seharris/qemu-avr-tests'
- '/raw/36c3e67b8755dcf/free-rtos/Demo'
- '/AVR_ATMega2560_GCC/demo.elf')
- rom_hash = '7eb521f511ca8f2622e0a3c5e8dd686efbb911d4'
- rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash)
+ rom_path = self.ASSET_ROM.fetch()
+ self.set_machine('arduino-mega-2560-v3')
self.vm.add_args('-bios', rom_path)
self.vm.add_args('-nographic')
self.vm.launch()
@@ -48,3 +47,6 @@ def test_freertos(self):
self.assertIn('ABCDEFGHIJKLMNOPQRSTUVWXABCDEFGHIJKLMNOPQRSTUVWX',
self.vm.get_log())
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/machine_loongarch.py b/tests/functional/test_loongarch64_virt.py
old mode 100644
new mode 100755
similarity index 54%
rename from tests/avocado/machine_loongarch.py
rename to tests/functional/test_loongarch64_virt.py
index 8de308f2d6..fb9329e486
--- a/tests/avocado/machine_loongarch.py
+++ b/tests/functional/test_loongarch64_virt.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# SPDX-License-Identifier: GPL-2.0-or-later
#
# LoongArch virt test.
@@ -5,15 +7,28 @@
# Copyright (c) 2023 Loongson Technology Corporation Limited
#
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
class LoongArchMachine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
timeout = 120
+ ASSET_KERNEL = Asset(
+ ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
+ 'releases/download/2024-05-30/vmlinuz.efi'),
+ '951b485b16e3788b6db03a3e1793c067009e31a2')
+ ASSET_INITRD = Asset(
+ ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
+ 'releases/download/2024-05-30/ramdisk'),
+ 'c67658d9b2a447ce7db2f73ba3d373c9b2b90ab2')
+ ASSET_BIOS = Asset(
+ ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
+ 'releases/download/2024-05-30/QEMU_EFI.fd'),
+ 'f4d0966b5117d4cd82327c050dd668741046be69')
+
def wait_for_console_pattern(self, success_message, vm=None):
wait_for_console_pattern(self, success_message,
failure_message='Kernel panic - not syncing',
@@ -21,25 +36,11 @@ def wait_for_console_pattern(self, success_message, vm=None):
def test_loongarch64_devices(self):
- """
- :avocado: tags=arch:loongarch64
- :avocado: tags=machine:virt
- """
-
- kernel_url = ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
- 'releases/download/2024-05-30/vmlinuz.efi')
- kernel_hash = '951b485b16e3788b6db03a3e1793c067009e31a2'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.set_machine('virt')
- initrd_url = ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
- 'releases/download/2024-05-30/ramdisk')
- initrd_hash = 'c67658d9b2a447ce7db2f73ba3d373c9b2b90ab2'
- initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
-
- bios_url = ('https://github.com/yangxiaojuan-loongson/qemu-binary/'
- 'releases/download/2024-05-30/QEMU_EFI.fd')
- bios_hash = ('f4d0966b5117d4cd82327c050dd668741046be69')
- bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
+ initrd_path = self.ASSET_INITRD.fetch()
+ bios_path = self.ASSET_BIOS.fetch()
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -56,3 +57,6 @@ def test_loongarch64_devices(self):
self.wait_for_console_pattern('Run /sbin/init as init process')
exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
'processor : 3')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/functional/test_mips64el_loongson3v.py b/tests/functional/test_mips64el_loongson3v.py
new file mode 100755
index 0000000000..55d62928c7
--- /dev/null
+++ b/tests/functional/test_mips64el_loongson3v.py
@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+#
+# Functional tests for the Generic Loongson-3 Platform.
+#
+# Copyright (c) 2021 Jiaxun Yang <jiaxun.yang@flygoat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import time
+
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+
+class MipsLoongson3v(QemuSystemTest):
+ timeout = 60
+
+ ASSET_PMON = Asset(
+ ('https://github.com/loongson-community/pmon/'
+ 'releases/download/20210112/pmon-3avirt.bin'),
+ 'fcdf6bb2cb7885a4a62f31fcb0d5e368bac7b6cea28f40c6dfa678af22fea20a')
+
+ @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+ def test_pmon_serial_console(self):
+ self.set_machine('loongson3-virt')
+
+ pmon_path = self.ASSET_PMON.fetch()
+
+ self.vm.set_console()
+ self.vm.add_args('-bios', pmon_path)
+ self.vm.launch()
+ wait_for_console_pattern(self, 'CPU GODSON3 BogoMIPS:')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/netdev-ethtool.py b/tests/functional/test_netdev_ethtool.py
old mode 100644
new mode 100755
similarity index 66%
rename from tests/avocado/netdev-ethtool.py
rename to tests/functional/test_netdev_ethtool.py
index 5f33288f81..b159a43c4f
--- a/tests/avocado/netdev-ethtool.py
+++ b/tests/functional/test_netdev_ethtool.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# ethtool tests for emulated network devices
#
# This test leverages ethtool's --test sequence to validate network
@@ -5,39 +7,32 @@
#
# SPDX-License-Identifier: GPL-2.0-or-late
-from avocado import skip
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from unittest import skip
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class NetDevEthtool(QemuSystemTest):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=machine:q35
- """
# Runs in about 17s under KVM, 19s under TCG, 25s under GCOV
timeout = 45
# Fetch assets from the netdev-ethtool subdir of my shared test
# images directory on fileserver.linaro.org.
- def get_asset(self, name, sha1):
- base_url = ('https://fileserver.linaro.org/s/'
- 'kE4nCFLdQcoBF9t/download?'
- 'path=%2Fnetdev-ethtool&files=' )
- url = base_url + name
- # use explicit name rather than failing to neatly parse the
- # URL into a unique one
- return self.fetch_asset(name=name, locations=(url), asset_hash=sha1)
+ ASSET_BASEURL = ('https://fileserver.linaro.org/s/'
+ 'kE4nCFLdQcoBF9t/download?'
+ 'path=%2Fnetdev-ethtool&files=')
+ ASSET_BZIMAGE = Asset(ASSET_BASEURL + "bzImage",
+ "33469d7802732d5815226166581442395cb289e2")
+ ASSET_ROOTFS = Asset(ASSET_BASEURL + "rootfs.squashfs",
+ "9793cea7021414ae844bda51f558bd6565b50cdc")
def common_test_code(self, netdev, extra_args=None):
+ self.set_machine('q35')
# This custom kernel has drivers for all the supported network
# devices we can emulate in QEMU
- kernel = self.get_asset("bzImage",
- "33469d7802732d5815226166581442395cb289e2")
-
- rootfs = self.get_asset("rootfs.squashfs",
- "9793cea7021414ae844bda51f558bd6565b50cdc")
+ kernel = self.ASSET_BZIMAGE.fetch()
+ rootfs = self.ASSET_ROOTFS.fetch()
append = 'printk.time=0 console=ttyS0 '
append += 'root=/dev/sr0 rootfstype=squashfs '
@@ -68,15 +63,9 @@ def common_test_code(self, netdev, extra_args=None):
self.vm.kill()
def test_igb(self):
- """
- :avocado: tags=device:igb
- """
self.common_test_code("igb")
def test_igb_nomsi(self):
- """
- :avocado: tags=device:igb
- """
self.common_test_code("igb", "pci=nomsi")
# It seems the other popular cards we model in QEMU currently fail
@@ -88,14 +77,11 @@ def test_igb_nomsi(self):
@skip("Incomplete reg 0x00178 support")
def test_e1000(self):
- """
- :avocado: tags=device:e1000
- """
self.common_test_code("e1000")
@skip("Incomplete reg 0x00178 support")
def test_i82550(self):
- """
- :avocado: tags=device:i82550
- """
self.common_test_code("i82550")
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_405.py b/tests/functional/test_ppc_405.py
old mode 100644
new mode 100755
similarity index 53%
rename from tests/avocado/ppc_405.py
rename to tests/functional/test_ppc_405.py
index 4e7e01aa76..5e200eab95
--- a/tests/avocado/ppc_405.py
+++ b/tests/functional/test_ppc_405.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that the U-Boot firmware boots on ppc 405 machines and check the console
#
# Copyright (c) 2021 Red Hat, Inc.
@@ -5,20 +7,20 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado_qemu import exec_command_and_wait_for_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
class Ppc405Machine(QemuSystemTest):
timeout = 90
+ ASSET_UBOOT = Asset(('https://gitlab.com/huth/u-boot/-/raw/'
+ 'taihu-2021-10-09/u-boot-taihu.bin'),
+ '3208940e908a5edc7c03eab072c60f0dcfadc2ab')
+
def do_test_ppc405(self):
- uboot_url = ('https://gitlab.com/huth/u-boot/-/raw/'
- 'taihu-2021-10-09/u-boot-taihu.bin')
- uboot_hash = ('3208940e908a5edc7c03eab072c60f0dcfadc2ab');
- file_path = self.fetch_asset(uboot_url, asset_hash=uboot_hash)
+ file_path = self.ASSET_UBOOT.fetch()
self.vm.set_console(console_index=1)
self.vm.add_args('-bios', file_path)
self.vm.launch()
@@ -26,11 +28,9 @@ def do_test_ppc405(self):
exec_command_and_wait_for_pattern(self, 'reset', 'AMCC PowerPC 405EP')
def test_ppc_ref405ep(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:ref405ep
- :avocado: tags=cpu:405ep
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
+ self.set_machine('ref405ep')
self.do_test_ppc405()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 11/24] tests/functional: Add a function for extracting files from an archive
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (9 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 10/24] tests/functional: Convert some tests that download files via fetch_asset() Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:22 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 12/24] tests/functional: Convert some avocado tests that needed avocado.utils.archive Daniel P. Berrangé
` (13 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P . Berrangé
From: Thomas Huth <thuth@redhat.com>
Some Avocado-based tests use the "archive" module from avocado.utils
to extract files from an archive. To be able to use these tests
without Avocado, we have to provide our own function for extracting
files. Fortunately, there is already the tarfile module that will
provide us with this functionality, so let's just add a nice wrapper
function around that.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/qemu_test/utils.py | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
create mode 100644 tests/functional/qemu_test/utils.py
diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py
new file mode 100644
index 0000000000..4eb5e5d5e5
--- /dev/null
+++ b/tests/functional/qemu_test/utils.py
@@ -0,0 +1,21 @@
+# Utilities for python-based QEMU tests
+#
+# Copyright 2024 Red Hat, Inc.
+#
+# Authors:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import tarfile
+
+def archive_extract(archive, dest_dir, member=None):
+ with tarfile.open(archive) as tf:
+ if hasattr(tarfile, 'data_filter'):
+ tf.extraction_filter = getattr(tarfile, 'data_filter',
+ (lambda member, path: member))
+ if member:
+ tf.extract(member=member, path=dest_dir)
+ else:
+ tf.extractall(path=dest_dir)
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 12/24] tests/functional: Convert some avocado tests that needed avocado.utils.archive
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (10 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 11/24] tests/functional: Add a function for extracting files from an archive Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:27 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests Daniel P. Berrangé
` (12 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P . Berrangé
From: Thomas Huth <thuth@redhat.com>
Instead of using the "archive" module from avocado.utils, switch
these tests to use the new wrapper function that is based on the
"tarfile" module instead.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 6 +--
tests/avocado/machine_sparc64_sun4u.py | 36 -----------------
tests/functional/meson.build | 6 +++
.../test_arm_canona1100.py} | 30 ++++++++------
.../test_ppc_bamboo.py} | 32 +++++++--------
tests/functional/test_sparc64_sun4u.py | 40 +++++++++++++++++++
6 files changed, 82 insertions(+), 68 deletions(-)
delete mode 100644 tests/avocado/machine_sparc64_sun4u.py
rename tests/{avocado/machine_arm_canona1100.py => functional/test_arm_canona1100.py} (52%)
mode change 100644 => 100755
rename tests/{avocado/ppc_bamboo.py => functional/test_ppc_bamboo.py} (60%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_sparc64_sun4u.py
diff --git a/MAINTAINERS b/MAINTAINERS
index c2b4352edd..ff952d683b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -731,7 +731,7 @@ S: Odd Fixes
F: include/hw/arm/digic.h
F: hw/*/digic*
F: include/hw/*/digic*
-F: tests/avocado/machine_arm_canona1100.py
+F: tests/functional/test_arm_canona1100.py
F: docs/system/arm/digic.rst
Goldfish RTC
@@ -1426,7 +1426,7 @@ L: qemu-ppc@nongnu.org
S: Orphan
F: hw/ppc/ppc440_bamboo.c
F: hw/pci-host/ppc4xx_pci.c
-F: tests/avocado/ppc_bamboo.py
+F: tests/functional/test_ppc_bamboo.py
e500
L: qemu-ppc@nongnu.org
@@ -1712,7 +1712,7 @@ F: include/hw/pci-host/sabre.h
F: hw/pci-bridge/simba.c
F: include/hw/pci-bridge/simba.h
F: pc-bios/openbios-sparc64
-F: tests/avocado/machine_sparc64_sun4u.py
+F: tests/functional/test_sparc64_sun4u.py
Sun4v
M: Artyom Tarasenko <atar4qemu@gmail.com>
diff --git a/tests/avocado/machine_sparc64_sun4u.py b/tests/avocado/machine_sparc64_sun4u.py
deleted file mode 100644
index d333c0ae91..0000000000
--- a/tests/avocado/machine_sparc64_sun4u.py
+++ /dev/null
@@ -1,36 +0,0 @@
-# Functional test that boots a Linux kernel and checks the console
-#
-# Copyright (c) 2020 Red Hat, Inc.
-#
-# Author:
-# Thomas Huth <thuth@redhat.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import os
-
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
-from boot_linux_console import LinuxKernelTest
-
-class Sun4uMachine(LinuxKernelTest):
- """Boots the Linux kernel and checks that the console is operational"""
-
- timeout = 90
-
- def test_sparc64_sun4u(self):
- """
- :avocado: tags=arch:sparc64
- :avocado: tags=machine:sun4u
- """
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day23.tar.xz')
- tar_hash = '142db83cd974ffadc4f75c8a5cad5bcc5722c240'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
- self.vm.set_console()
- self.vm.add_args('-kernel', self.workdir + '/day23/vmlinux',
- '-append', self.KERNEL_COMMON_COMMAND_LINE)
- self.vm.launch()
- wait_for_console_pattern(self, 'Starting logging: OK')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 32c7cc2d3c..91bf80f0f4 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -21,6 +21,7 @@ tests_generic = [
]
tests_arm_thorough = [
+ 'arm_canona1100',
'arm_n8x0',
]
@@ -42,6 +43,11 @@ tests_ppc_quick = [
tests_ppc_thorough = [
'ppc_405',
+ 'ppc_bamboo',
+]
+
+tests_sparc64_thorough = [
+ 'sparc64_sun4u',
]
tests_x86_64_quick = [
diff --git a/tests/avocado/machine_arm_canona1100.py b/tests/functional/test_arm_canona1100.py
old mode 100644
new mode 100755
similarity index 52%
rename from tests/avocado/machine_arm_canona1100.py
rename to tests/functional/test_arm_canona1100.py
index a42d8b0f2b..57cecbbb36
--- a/tests/avocado/machine_arm_canona1100.py
+++ b/tests/functional/test_arm_canona1100.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots the canon-a1100 machine with firmware
#
# Copyright (c) 2020 Red Hat, Inc.
@@ -8,28 +10,30 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
class CanonA1100Machine(QemuSystemTest):
"""Boots the barebox firmware and checks that the console is operational"""
timeout = 90
+ ASSET_BIOS = Asset(('https://qemu-advcal.gitlab.io'
+ '/qac-best-of-multiarch/download/day18.tar.xz'),
+ '068b5fc4242b29381acee94713509f8a876e9db6')
+
def test_arm_canona1100(self):
- """
- :avocado: tags=arch:arm
- :avocado: tags=machine:canon-a1100
- :avocado: tags=device:pflash_cfi02
- """
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day18.tar.xz')
- tar_hash = '068b5fc4242b29381acee94713509f8a876e9db6'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ self.set_machine('canon-a1100')
+
+ file_path = self.ASSET_BIOS.fetch()
+ archive_extract(file_path, dest_dir=self.workdir,
+ member="day18/barebox.canon-a1100.bin")
self.vm.set_console()
self.vm.add_args('-bios',
self.workdir + '/day18/barebox.canon-a1100.bin')
self.vm.launch()
wait_for_console_pattern(self, 'running /env/bin/init')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_bamboo.py b/tests/functional/test_ppc_bamboo.py
old mode 100644
new mode 100755
similarity index 60%
rename from tests/avocado/ppc_bamboo.py
rename to tests/functional/test_ppc_bamboo.py
index a81be3d608..8a1f7eca4d
--- a/tests/avocado/ppc_bamboo.py
+++ b/tests/functional/test_ppc_bamboo.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on the ppc bamboo board and check the console
#
# Copyright (c) 2021 Red Hat
@@ -5,30 +7,25 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado_qemu import exec_command_and_wait_for_pattern
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
class BambooMachine(QemuSystemTest):
timeout = 90
+ ASSET_IMAGE = Asset(('http://landley.net/aboriginal/downloads/binaries/'
+ 'system-image-powerpc-440fp.tar.gz'),
+ '53e5f16414b195b82d2c70272f81c2eedb39bad9')
+
def test_ppc_bamboo(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:bamboo
- :avocado: tags=cpu:440epb
- :avocado: tags=device:rtl8139
- :avocado: tags=accel:tcg
- """
+ self.set_machine('bamboo')
self.require_accelerator("tcg")
self.require_netdev('user')
- tar_url = ('http://landley.net/aboriginal/downloads/binaries/'
- 'system-image-powerpc-440fp.tar.gz')
- tar_hash = '53e5f16414b195b82d2c70272f81c2eedb39bad9'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
self.vm.set_console()
self.vm.add_args('-kernel', self.workdir +
'/system-image-powerpc-440fp/linux',
@@ -40,3 +37,6 @@ def test_ppc_bamboo(self):
exec_command_and_wait_for_pattern(self, 'ping 10.0.2.2',
'10.0.2.2 is alive!')
exec_command_and_wait_for_pattern(self, 'halt', 'System Halted')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/functional/test_sparc64_sun4u.py b/tests/functional/test_sparc64_sun4u.py
new file mode 100755
index 0000000000..3ac18aebd1
--- /dev/null
+++ b/tests/functional/test_sparc64_sun4u.py
@@ -0,0 +1,40 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a Linux kernel and checks the console
+#
+# Copyright (c) 2020 Red Hat, Inc.
+#
+# Author:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+
+class Sun4uMachine(QemuSystemTest):
+ """Boots the Linux kernel and checks that the console is operational"""
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io'
+ '/qac-best-of-multiarch/download/day23.tar.xz'),
+ '142db83cd974ffadc4f75c8a5cad5bcc5722c240')
+
+ def test_sparc64_sun4u(self):
+ self.set_machine('sun4u')
+ file_path = self.ASSET_IMAGE.fetch()
+ kernel_name = 'day23/vmlinux'
+ archive_extract(file_path, self.workdir, kernel_name)
+ self.vm.set_console()
+ self.vm.add_args('-kernel', os.path.join(self.workdir, kernel_name),
+ '-append', 'printk.time=0')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'Starting logging: OK')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (11 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 12/24] tests/functional: Convert some avocado tests that needed avocado.utils.archive Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 17:11 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 14/24] tests/functional: Convert the x86_cpu_model_versions test Daniel P. Berrangé
` (11 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
These tests use archive.lzma_uncompress() from the Avocado utils,
so provide a small helper function for this, based on the
standard lzma module from Python instead.
And while we're at it, replace the MD5 hashes in the topology test
with proper SHA256 hashes, since MD5 should not be used anymore
nowadays.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 4 +-
tests/functional/meson.build | 6 ++
tests/functional/qemu_test/utils.py | 14 +++
.../test_s390x_ccw_virtio.py} | 79 ++++++++---------
.../test_s390x_topology.py} | 86 ++++++++-----------
5 files changed, 94 insertions(+), 95 deletions(-)
rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
mode change 100644 => 100755
rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index ff952d683b..c78edb078a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1739,7 +1739,7 @@ S: Supported
F: hw/s390x/
F: include/hw/s390x/
F: configs/devices/s390x-softmmu/default.mak
-F: tests/avocado/machine_s390_ccw_virtio.py
+F: tests/functional/test_s390x_ccw_virtio.py
T: git https://github.com/borntraeger/qemu.git s390-next
L: qemu-s390x@nongnu.org
@@ -1802,7 +1802,7 @@ F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
F: docs/devel/s390-cpu-topology.rst
F: docs/system/s390x/cpu-topology.rst
-F: tests/avocado/s390_topology.py
+F: tests/functional/test_s390x_topology.py
X86 Machines
------------
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 91bf80f0f4..ce5e5be2f3 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -12,6 +12,7 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
'netdev_ethtool' : 180,
+ 's390x_ccw_virtio' : 180,
}
tests_generic = [
@@ -46,6 +47,11 @@ tests_ppc_thorough = [
'ppc_bamboo',
]
+tests_s390x_thorough = [
+ 's390x_ccw_virtio',
+ 's390x_topology',
+]
+
tests_sparc64_thorough = [
'sparc64_sun4u',
]
diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py
index 4eb5e5d5e5..a12dac51b6 100644
--- a/tests/functional/qemu_test/utils.py
+++ b/tests/functional/qemu_test/utils.py
@@ -8,6 +8,9 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
+import lzma
+import os
+import shutil
import tarfile
def archive_extract(archive, dest_dir, member=None):
@@ -19,3 +22,14 @@ def archive_extract(archive, dest_dir, member=None):
tf.extract(member=member, path=dest_dir)
else:
tf.extractall(path=dest_dir)
+
+def lzma_uncompress(xz_path, output_path):
+ if os.path.exists(output_path):
+ return
+ with lzma.open(xz_path, 'rb') as lzma_in:
+ try:
+ with open(output_path, 'wb') as raw_out:
+ shutil.copyfileobj(lzma_in, raw_out)
+ except:
+ os.remove(output_path)
+ raise
diff --git a/tests/avocado/machine_s390_ccw_virtio.py b/tests/functional/test_s390x_ccw_virtio.py
old mode 100644
new mode 100755
similarity index 85%
rename from tests/avocado/machine_s390_ccw_virtio.py
rename to tests/functional/test_s390x_ccw_virtio.py
index 26e938c9e9..f110f3b462
--- a/tests/avocado/machine_s390_ccw_virtio.py
+++ b/tests/functional/test_s390x_ccw_virtio.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots an s390x Linux guest with ccw and PCI devices
# attached and checks whether the devices are recognized by Linux
#
@@ -12,17 +14,38 @@
import os
import tempfile
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import lzma_uncompress
class S390CCWVirtioMachine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
timeout = 120
+ ASSET_BUSTER_KERNEL = Asset(
+ ('https://snapshot.debian.org/archive/debian/'
+ '20201126T092837Z/dists/buster/main/installer-s390x/'
+ '20190702+deb10u6/images/generic/kernel.debian'),
+ '5821fbee57d6220a067a8b967d24595621aa1eb6')
+ ASSET_BUSTER_INITRD = Asset(
+ ('https://snapshot.debian.org/archive/debian/'
+ '20201126T092837Z/dists/buster/main/installer-s390x/'
+ '20190702+deb10u6/images/generic/initrd.debian'),
+ '81ba09c97bef46e8f4660ac25b4ac0a5be3a94d6')
+
+ ASSET_F31_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/31/Server/s390x/os'
+ '/images/kernel.img'),
+ 'b93d1efcafcf29c1673a4ce371a1f8b43941cfeb')
+ ASSET_F31_INITRD = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/31/Server/s390x/os'
+ '/images/initrd.img'),
+ '3de45d411df5624b8d8ef21cd0b44419ab59b12f')
+
def wait_for_console_pattern(self, success_message, vm=None):
wait_for_console_pattern(self, success_message,
failure_message='Kernel panic - not syncing',
@@ -41,23 +64,10 @@ def clear_guest_dmesg(self):
self.dmesg_clear_count += 1
def test_s390x_devices(self):
+ self.set_machine('s390-ccw-virtio')
- """
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
- """
-
- kernel_url = ('https://snapshot.debian.org/archive/debian/'
- '20201126T092837Z/dists/buster/main/installer-s390x/'
- '20190702+deb10u6/images/generic/kernel.debian')
- kernel_hash = '5821fbee57d6220a067a8b967d24595621aa1eb6'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-
- initrd_url = ('https://snapshot.debian.org/archive/debian/'
- '20201126T092837Z/dists/buster/main/installer-s390x/'
- '20190702+deb10u6/images/generic/initrd.debian')
- initrd_hash = '81ba09c97bef46e8f4660ac25b4ac0a5be3a94d6'
- initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+ kernel_path = self.ASSET_BUSTER_KERNEL.fetch()
+ initrd_path = self.ASSET_BUSTER_INITRD.fetch()
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -160,29 +170,13 @@ def test_s390x_devices(self):
def test_s390x_fedora(self):
+ self.set_machine('s390-ccw-virtio')
- """
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
- :avocado: tags=device:virtio-gpu
- :avocado: tags=device:virtio-crypto
- :avocado: tags=device:virtio-net
- :avocado: tags=flaky
- """
-
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/31/Server/s390x/os'
- '/images/kernel.img')
- kernel_hash = 'b93d1efcafcf29c1673a4ce371a1f8b43941cfeb'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_F31_KERNEL.fetch()
- initrd_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/31/Server/s390x/os'
- '/images/initrd.img')
- initrd_hash = '3de45d411df5624b8d8ef21cd0b44419ab59b12f'
- initrd_path_xz = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+ initrd_path_xz = self.ASSET_F31_INITRD.fetch()
initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
- archive.lzma_uncompress(initrd_path_xz, initrd_path)
+ lzma_uncompress(initrd_path_xz, initrd_path)
self.vm.set_console()
kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + ' audit=0 '
@@ -275,3 +269,6 @@ def test_s390x_fedora(self):
exec_command_and_wait_for_pattern(self,
'while ! (dmesg -c | grep Start.virtcrypto_remove) ; do'
' sleep 1 ; done', 'Start virtcrypto_remove.')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/s390_topology.py b/tests/functional/test_s390x_topology.py
old mode 100644
new mode 100755
similarity index 88%
rename from tests/avocado/s390_topology.py
rename to tests/functional/test_s390x_topology.py
index 9154ac8776..20727f6bdf
--- a/tests/avocado/s390_topology.py
+++ b/tests/functional/test_s390x_topology.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a Linux kernel and checks the console
#
# Copyright IBM Corp. 2023
@@ -9,16 +11,13 @@
# later. See the COPYING file in the top-level directory.
import os
-import shutil
import time
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import interrupt_interactive_console_until_pattern
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import process
-from avocado.utils import archive
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import lzma_uncompress
class S390CPUTopology(QemuSystemTest):
@@ -47,6 +46,17 @@ class S390CPUTopology(QemuSystemTest):
'root=/dev/ram '
'selinux=0 '
'rdinit=/bin/sh')
+ ASSET_F35_KERNEL = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img'),
+ '1f2dddfd11bb1393dd2eb2e784036fbf6fc11057a6d7d27f9eb12d3edc67ef73')
+
+ ASSET_F35_INITRD = Asset(
+ ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/initrd.img'),
+ '1100145fbca00240c8c372ae4b89b48c99844bc189b3dfbc3f481dc60055ca46')
def wait_until_booted(self):
wait_for_console_pattern(self, 'no job control',
@@ -78,21 +88,10 @@ def kernel_init(self):
We need a minimal root filesystem with a shell.
"""
self.require_accelerator("kvm")
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/35/Server/s390x/os'
- '/images/kernel.img')
- kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
- kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
- asset_hash=kernel_hash)
-
- initrd_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/35/Server/s390x/os'
- '/images/initrd.img')
- initrd_hash = 'a122057d95725ac030e2ec51df46e172'
- initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
- asset_hash=initrd_hash)
+ kernel_path = self.ASSET_F35_KERNEL.fetch()
+ initrd_path_xz = self.ASSET_F35_INITRD.fetch()
initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
- archive.lzma_uncompress(initrd_path_xz, initrd_path)
+ lzma_uncompress(initrd_path_xz, initrd_path)
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
@@ -115,10 +114,8 @@ def system_init(self):
def test_single(self):
"""
This test checks the simplest topology with a single CPU.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -127,10 +124,8 @@ def test_single(self):
def test_default(self):
"""
This test checks the implicit topology.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'13,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -154,10 +149,8 @@ def test_move(self):
"""
This test checks the topology modification by moving a CPU
to another socket: CPU 0 is moved from socket 0 to socket 2.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -174,10 +167,8 @@ def test_dash_device(self):
"""
This test verifies that a CPU defined with the '-device'
command line option finds its right place inside the topology.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'1,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -221,10 +212,8 @@ def test_polarization(self):
"""
This test verifies that QEMU modifies the entitlement change after
several guest polarization change requests.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -267,10 +256,8 @@ def test_entitlement(self):
"""
This test verifies that QEMU modifies the entitlement
after a guest request and that the guest sees the change.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -313,10 +300,8 @@ def test_dedicated(self):
CPU is made dedicated.
QEMU retains the entitlement value when horizontal polarization is in effect.
For the guest, the field shows the effective value of the entitlement.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -345,10 +330,8 @@ def test_socket_full(self):
This test verifies that QEMU does not accept to overload a socket.
The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
not accept any new CPU while socket-id 0 on book-id 1 is free.
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.add_args('-smp',
'3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
@@ -369,10 +352,8 @@ def test_dedicated_error(self):
"""
This test verifies that QEMU refuses to lower the entitlement
of a dedicated CPU
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -417,10 +398,8 @@ def test_move_error(self):
"""
This test verifies that QEMU refuses to move a CPU to an
nonexistent location
-
- :avocado: tags=arch:s390x
- :avocado: tags=machine:s390-ccw-virtio
"""
+ self.set_machine('s390-ccw-virtio')
self.kernel_init()
self.vm.launch()
self.wait_until_booted()
@@ -437,3 +416,6 @@ def test_move_error(self):
self.assertEqual(res['error']['class'], 'GenericError')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 14/24] tests/functional: Convert the x86_cpu_model_versions test
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (12 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:43 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 15/24] tests/functional: Convert the microblaze avocado tests into standalone tests Daniel P. Berrangé
` (10 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Nothing thrilling in here, it's just a straight forward conversion.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/meson.build | 1 +
.../test_x86_cpu_model_versions.py} | 63 ++++++-------------
2 files changed, 20 insertions(+), 44 deletions(-)
rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
mode change 100644 => 100755
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index ce5e5be2f3..e85d869948 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -61,6 +61,7 @@ tests_x86_64_quick = [
'mem_addr_space',
'pc_cpu_hotplug_props',
'virtio_version',
+ 'x86_cpu_model_versions',
]
tests_x86_64_thorough = [
diff --git a/tests/avocado/x86_cpu_model_versions.py b/tests/functional/test_x86_cpu_model_versions.py
old mode 100644
new mode 100755
similarity index 92%
rename from tests/avocado/x86_cpu_model_versions.py
rename to tests/functional/test_x86_cpu_model_versions.py
index 11101e02b9..a5f27c737d
--- a/tests/avocado/x86_cpu_model_versions.py
+++ b/tests/functional/test_x86_cpu_model_versions.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# Basic validation of x86 versioned CPU models and CPU model aliases
#
@@ -20,11 +21,11 @@
# License along with this library; if not, see <http://www.gnu.org/licenses/>.
#
-
-import avocado_qemu
import re
-class X86CPUModelAliases(avocado_qemu.QemuSystemTest):
+from qemu_test import QemuSystemTest
+
+class X86CPUModelAliases(QemuSystemTest):
"""
Validation of PC CPU model versions and CPU model aliases
@@ -76,9 +77,8 @@ def validate_variant_aliases(self, cpus):
def test_4_0_alias_compatibility(self):
"""
Check if pc-*-4.0 unversioned CPU model won't be reported as aliases
-
- :avocado: tags=machine:pc-i440fx-4.0
"""
+ self.set_machine('pc-i440fx-4.0')
# pc-*-4.0 won't expose non-versioned CPU models as aliases
# We do this to help management software to keep compatibility
# with older QEMU versions that didn't have the versioned CPU model
@@ -110,9 +110,8 @@ def test_4_0_alias_compatibility(self):
def test_4_1_alias(self):
"""
Check if unversioned CPU model is an alias pointing to right version
-
- :avocado: tags=machine:pc-i440fx-4.1
"""
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.vm.launch()
@@ -217,9 +216,8 @@ def test_4_1_alias(self):
def test_none_alias(self):
"""
Check if unversioned CPU model is an alias pointing to some version
-
- :avocado: tags=machine:none
"""
+ self.machine = 'none'
self.vm.add_args('-S')
self.vm.launch()
@@ -243,21 +241,16 @@ def test_none_alias(self):
self.validate_aliases(cpus)
-class CascadelakeArchCapabilities(avocado_qemu.QemuSystemTest):
+class CascadelakeArchCapabilities(QemuSystemTest):
"""
Validation of Cascadelake arch-capabilities
-
- :avocado: tags=arch:x86_64
"""
def get_cpu_prop(self, prop):
cpu_path = self.vm.cmd('query-cpus-fast')[0].get('qom-path')
return self.vm.cmd('qom-get', path=cpu_path, property=prop)
def test_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
# machine-type only:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -268,10 +261,7 @@ def test_4_1(self):
'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
def test_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,'
@@ -281,10 +271,7 @@ def test_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
def test_set_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# command line must override machine-type if CPU model is not versioned:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -295,10 +282,7 @@ def test_set_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
def test_unset_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,'
@@ -308,10 +292,7 @@ def test_unset_4_1(self):
'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
def test_v1_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# versioned CPU model overrides machine-type:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -322,10 +303,7 @@ def test_v1_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
def test_v2_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,'
@@ -335,10 +313,7 @@ def test_v2_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
def test_v1_set_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# command line must override machine-type and versioned CPU model:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -349,10 +324,7 @@ def test_v1_set_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
def test_v2_unset_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,'
@@ -360,3 +332,6 @@ def test_v2_unset_4_1(self):
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server-v2,-arch-capabilities should not have arch-capabilities')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 15/24] tests/functional: Convert the microblaze avocado tests into standalone tests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (13 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 14/24] tests/functional: Convert the x86_cpu_model_versions test Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-02 16:16 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Daniel P. Berrangé
` (9 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
The machine_microblaze.py file contained two tests, one for each
endianess. Since we only support one QEMU target binary per file
in the new functional test environment, we have to split this file
up into two files now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 2 +-
tests/avocado/machine_microblaze.py | 61 -------------------
tests/functional/meson.build | 8 +++
.../functional/test_microblaze_s3adsp1800.py | 39 ++++++++++++
.../test_microblazeel_s3adsp1800.py | 42 +++++++++++++
5 files changed, 90 insertions(+), 62 deletions(-)
delete mode 100644 tests/avocado/machine_microblaze.py
create mode 100755 tests/functional/test_microblaze_s3adsp1800.py
create mode 100755 tests/functional/test_microblazeel_s3adsp1800.py
diff --git a/MAINTAINERS b/MAINTAINERS
index c78edb078a..e1d15c9696 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1327,7 +1327,7 @@ M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
S: Maintained
F: hw/microblaze/petalogix_s3adsp1800_mmu.c
F: include/hw/char/xilinx_uartlite.h
-F: tests/avocado/machine_microblaze.py
+F: tests/functional/test_microblaze*.py
petalogix_ml605
M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
diff --git a/tests/avocado/machine_microblaze.py b/tests/avocado/machine_microblaze.py
deleted file mode 100644
index 807709cd11..0000000000
--- a/tests/avocado/machine_microblaze.py
+++ /dev/null
@@ -1,61 +0,0 @@
-# Functional test that boots a microblaze Linux kernel and checks the console
-#
-# Copyright (c) 2018, 2021 Red Hat, Inc.
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import time
-from avocado_qemu import exec_command, exec_command_and_wait_for_pattern
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
-
-class MicroblazeMachine(QemuSystemTest):
-
- timeout = 90
-
- def test_microblaze_s3adsp1800(self):
- """
- :avocado: tags=arch:microblaze
- :avocado: tags=machine:petalogix-s3adsp1800
- """
-
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day17.tar.xz')
- tar_hash = '08bf3e3bfb6b6c7ce1e54ab65d54e189f2caf13f'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
- self.vm.set_console()
- self.vm.add_args('-kernel', self.workdir + '/day17/ballerina.bin')
- self.vm.launch()
- wait_for_console_pattern(self, 'This architecture does not have '
- 'kernel memory protection')
- # Note:
- # The kernel sometimes gets stuck after the "This architecture ..."
- # message, that's why we don't test for a later string here. This
- # needs some investigation by a microblaze wizard one day...
-
- def test_microblazeel_s3adsp1800(self):
- """
- :avocado: tags=arch:microblazeel
- :avocado: tags=machine:petalogix-s3adsp1800
- """
-
- self.require_netdev('user')
- tar_url = ('http://www.qemu-advent-calendar.org/2023/download/'
- 'day13.tar.gz')
- tar_hash = '6623d5fff5f84cfa8f34e286f32eff6a26546f44'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
- self.vm.set_console()
- self.vm.add_args('-kernel', self.workdir + '/day13/xmaton.bin')
- self.vm.add_args('-nic', 'user,tftp=' + self.workdir + '/day13/')
- self.vm.launch()
- wait_for_console_pattern(self, 'QEMU Advent Calendar 2023')
- time.sleep(0.1)
- exec_command(self, 'root')
- time.sleep(0.1)
- exec_command_and_wait_for_pattern(self,
- 'tftp -g -r xmaton.png 10.0.2.2 ; md5sum xmaton.png',
- '821cd3cab8efd16ad6ee5acc3642a8ea')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index e85d869948..d6a1a3a116 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -34,6 +34,14 @@ tests_loongarch64_thorough = [
'loongarch64_virt',
]
+tests_microblaze_thorough = [
+ 'microblaze_s3adsp1800'
+]
+
+tests_microblazeel_thorough = [
+ 'microblazeel_s3adsp1800'
+]
+
tests_mips64el_thorough = [
'mips64el_loongson3v',
]
diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/functional/test_microblaze_s3adsp1800.py
new file mode 100755
index 0000000000..7618531b8c
--- /dev/null
+++ b/tests/functional/test_microblaze_s3adsp1800.py
@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a microblaze Linux kernel and checks the console
+#
+# Copyright (c) 2018, 2021 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import time
+from qemu_test import exec_command, exec_command_and_wait_for_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+
+class MicroblazeMachine(QemuSystemTest):
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io'
+ '/qac-best-of-multiarch/download/day17.tar.xz'),
+ '08bf3e3bfb6b6c7ce1e54ab65d54e189f2caf13f')
+
+ def test_microblaze_s3adsp1800(self):
+ self.set_machine('petalogix-s3adsp1800')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
+ self.vm.set_console()
+ self.vm.add_args('-kernel', self.workdir + '/day17/ballerina.bin')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'This architecture does not have '
+ 'kernel memory protection')
+ # Note:
+ # The kernel sometimes gets stuck after the "This architecture ..."
+ # message, that's why we don't test for a later string here. This
+ # needs some investigation by a microblaze wizard one day...
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/functional/test_microblazeel_s3adsp1800.py b/tests/functional/test_microblazeel_s3adsp1800.py
new file mode 100755
index 0000000000..e90ab91472
--- /dev/null
+++ b/tests/functional/test_microblazeel_s3adsp1800.py
@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+#
+# Functional test that boots a microblaze Linux kernel and checks the console
+#
+# Copyright (c) 2018, 2021 Red Hat, Inc.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import time
+from qemu_test import exec_command, exec_command_and_wait_for_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+
+class MicroblazeelMachine(QemuSystemTest):
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(('http://www.qemu-advent-calendar.org/2023/download/'
+ 'day13.tar.gz'),
+ '6623d5fff5f84cfa8f34e286f32eff6a26546f44')
+
+ def test_microblazeel_s3adsp1800(self):
+ self.require_netdev('user')
+ self.set_machine('petalogix-s3adsp1800')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
+ self.vm.set_console()
+ self.vm.add_args('-kernel', self.workdir + '/day13/xmaton.bin')
+ self.vm.add_args('-nic', 'user,tftp=' + self.workdir + '/day13/')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'QEMU Advent Calendar 2023')
+ time.sleep(0.1)
+ exec_command(self, 'root')
+ time.sleep(0.1)
+ exec_command_and_wait_for_pattern(self,
+ 'tftp -g -r xmaton.png 10.0.2.2 ; md5sum xmaton.png',
+ '821cd3cab8efd16ad6ee5acc3642a8ea')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (14 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 15/24] tests/functional: Convert the microblaze avocado tests into standalone tests Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-31 12:56 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 17/24] tests/functional: Convert the virtio_gpu " Daniel P. Berrangé
` (8 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Alistair Francis
From: Thomas Huth <thuth@redhat.com>
The avocado test defined test functions for both, riscv32 and riscv64.
Since we can run the whole file with multiple targets in the new
framework, we can now consolidate the functions so we have to only
define one function per machine now.
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/riscv_opensbi.py | 63 --------------------------
tests/functional/meson.build | 8 ++++
tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++
3 files changed, 44 insertions(+), 63 deletions(-)
delete mode 100644 tests/avocado/riscv_opensbi.py
create mode 100755 tests/functional/test_riscv_opensbi.py
diff --git a/tests/avocado/riscv_opensbi.py b/tests/avocado/riscv_opensbi.py
deleted file mode 100644
index bfff9cc3c3..0000000000
--- a/tests/avocado/riscv_opensbi.py
+++ /dev/null
@@ -1,63 +0,0 @@
-# OpenSBI boot test for RISC-V machines
-#
-# Copyright (c) 2022, Ventana Micro
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-
-class RiscvOpenSBI(QemuSystemTest):
- """
- :avocado: tags=accel:tcg
- """
- timeout = 5
-
- def boot_opensbi(self):
- self.vm.set_console()
- self.vm.launch()
- wait_for_console_pattern(self, 'Platform Name')
- wait_for_console_pattern(self, 'Boot HART MEDELEG')
-
- def test_riscv32_spike(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:spike
- """
- self.boot_opensbi()
-
- def test_riscv64_spike(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:spike
- """
- self.boot_opensbi()
-
- def test_riscv32_sifive_u(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:sifive_u
- """
- self.boot_opensbi()
-
- def test_riscv64_sifive_u(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:sifive_u
- """
- self.boot_opensbi()
-
- def test_riscv32_virt(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:virt
- """
- self.boot_opensbi()
-
- def test_riscv64_virt(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:virt
- """
- self.boot_opensbi()
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index d6a1a3a116..55021c4e4d 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -55,6 +55,14 @@ tests_ppc_thorough = [
'ppc_bamboo',
]
+tests_riscv32_quick = [
+ 'riscv_opensbi',
+]
+
+tests_riscv64_quick = [
+ 'riscv_opensbi',
+]
+
tests_s390x_thorough = [
's390x_ccw_virtio',
's390x_topology',
diff --git a/tests/functional/test_riscv_opensbi.py b/tests/functional/test_riscv_opensbi.py
new file mode 100755
index 0000000000..d077e40f42
--- /dev/null
+++ b/tests/functional/test_riscv_opensbi.py
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+#
+# OpenSBI boot test for RISC-V machines
+#
+# Copyright (c) 2022, Ventana Micro
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+from qemu_test import QemuSystemTest
+from qemu_test import wait_for_console_pattern
+
+class RiscvOpenSBI(QemuSystemTest):
+
+ timeout = 5
+
+ def boot_opensbi(self):
+ self.vm.set_console()
+ self.vm.launch()
+ wait_for_console_pattern(self, 'Platform Name')
+ wait_for_console_pattern(self, 'Boot HART MEDELEG')
+
+ def test_riscv_spike(self):
+ self.set_machine('spike')
+ self.boot_opensbi()
+
+ def test_riscv_sifive_u(self):
+ self.set_machine('sifive_u')
+ self.boot_opensbi()
+
+ def test_riscv_virt(self):
+ self.set_machine('virt')
+ self.boot_opensbi()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 17/24] tests/functional: Convert the virtio_gpu avocado test into a standalone test
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (15 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-02 16:35 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 18/24] tests/functional: Convert most ppc avocado tests into standalone tests Daniel P. Berrangé
` (7 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Nothing thrilling in here, it's just a straight forward conversion.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/meson.build | 1 +
.../test_virtio_gpu.py} | 64 +++++++++----------
2 files changed, 30 insertions(+), 35 deletions(-)
rename tests/{avocado/virtio-gpu.py => functional/test_virtio_gpu.py} (73%)
mode change 100644 => 100755
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 55021c4e4d..8fbd7c4129 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -82,6 +82,7 @@ tests_x86_64_quick = [
tests_x86_64_thorough = [
'netdev_ethtool',
+ 'virtio_gpu',
]
precache_all = []
diff --git a/tests/avocado/virtio-gpu.py b/tests/functional/test_virtio_gpu.py
old mode 100644
new mode 100755
similarity index 73%
rename from tests/avocado/virtio-gpu.py
rename to tests/functional/test_virtio_gpu.py
index 6091f614a4..e74da39444
--- a/tests/avocado/virtio-gpu.py
+++ b/tests/functional/test_virtio_gpu.py
@@ -1,14 +1,16 @@
+#!/usr/bin/env python3
+#
# virtio-gpu tests
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import BUILD_DIR
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import is_readable_executable_file
+from qemu_test import BUILD_DIR
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import is_readable_executable_file
from qemu.utils import kvm_available
@@ -28,25 +30,18 @@ def pick_default_vug_bin():
class VirtioGPUx86(QemuSystemTest):
- """
- :avocado: tags=virtio-gpu
- :avocado: tags=arch:x86_64
- :avocado: tags=cpu:host
- """
KERNEL_COMMAND_LINE = "printk.time=0 console=ttyS0 rdinit=/bin/bash"
- KERNEL_URL = (
- "https://archives.fedoraproject.org/pub/archive/fedora"
- "/linux/releases/33/Everything/x86_64/os/images"
- "/pxeboot/vmlinuz"
- )
- KERNEL_HASH = '1433cfe3f2ffaa44de4ecfb57ec25dc2399cdecf'
- INITRD_URL = (
- "https://archives.fedoraproject.org/pub/archive/fedora"
- "/linux/releases/33/Everything/x86_64/os/images"
- "/pxeboot/initrd.img"
- )
- INITRD_HASH = 'c828d68a027b53e5220536585efe03412332c2d9'
+ ASSET_KERNEL = Asset(
+ ("https://archives.fedoraproject.org/pub/archive/fedora"
+ "/linux/releases/33/Everything/x86_64/os/images"
+ "/pxeboot/vmlinuz"),
+ '1433cfe3f2ffaa44de4ecfb57ec25dc2399cdecf')
+ ASSET_INITRD = Asset(
+ ("https://archives.fedoraproject.org/pub/archive/fedora"
+ "/linux/releases/33/Everything/x86_64/os/images"
+ "/pxeboot/initrd.img"),
+ 'c828d68a027b53e5220536585efe03412332c2d9')
def wait_for_console_pattern(self, success_message, vm=None):
wait_for_console_pattern(
@@ -57,16 +52,14 @@ def wait_for_console_pattern(self, success_message, vm=None):
)
def test_virtio_vga_virgl(self):
- """
- :avocado: tags=device:virtio-vga-gl
- """
# FIXME: should check presence of virtio, virgl etc
self.require_accelerator('kvm')
- kernel_path = self.fetch_asset(self.KERNEL_URL, self.KERNEL_HASH)
- initrd_path = self.fetch_asset(self.INITRD_URL, self.INITRD_HASH)
+ kernel_path = self.ASSET_KERNEL.fetch()
+ initrd_path = self.ASSET_INITRD.fetch()
self.vm.set_console()
+ self.vm.add_args("-cpu", "host")
self.vm.add_args("-m", "2G")
self.vm.add_args("-machine", "pc,accel=kvm")
self.vm.add_args("-device", "virtio-vga-gl")
@@ -83,7 +76,7 @@ def test_virtio_vga_virgl(self):
self.vm.launch()
except:
# TODO: probably fails because we are missing the VirGL features
- self.cancel("VirGL not enabled?")
+ self.skipTest("VirGL not enabled?")
self.wait_for_console_pattern("as init process")
exec_command_and_wait_for_pattern(
@@ -92,18 +85,15 @@ def test_virtio_vga_virgl(self):
self.wait_for_console_pattern("features: +virgl +edid")
def test_vhost_user_vga_virgl(self):
- """
- :avocado: tags=device:vhost-user-vga
- """
# FIXME: should check presence of vhost-user-gpu, virgl, memfd etc
self.require_accelerator('kvm')
vug = pick_default_vug_bin()
if not vug:
- self.cancel("Could not find vhost-user-gpu")
+ self.skipTest("Could not find vhost-user-gpu")
- kernel_path = self.fetch_asset(self.KERNEL_URL, self.KERNEL_HASH)
- initrd_path = self.fetch_asset(self.INITRD_URL, self.INITRD_HASH)
+ kernel_path = self.ASSET_KERNEL.fetch()
+ initrd_path = self.ASSET_INITRD.fetch()
# Create socketpair to connect proxy and remote processes
qemu_sock, vug_sock = socket.socketpair(
@@ -129,6 +119,7 @@ def test_vhost_user_vga_virgl(self):
)
self.vm.set_console()
+ self.vm.add_args("-cpu", "host")
self.vm.add_args("-m", "2G")
self.vm.add_args("-object", "memory-backend-memfd,id=mem,size=2G")
self.vm.add_args("-machine", "pc,memory-backend=mem,accel=kvm")
@@ -147,7 +138,7 @@ def test_vhost_user_vga_virgl(self):
self.vm.launch()
except:
# TODO: probably fails because we are missing the VirGL features
- self.cancel("VirGL not enabled?")
+ self.skipTest("VirGL not enabled?")
self.wait_for_console_pattern("as init process")
exec_command_and_wait_for_pattern(self, "/usr/sbin/modprobe virtio_gpu",
"features: +virgl +edid")
@@ -155,3 +146,6 @@ def test_vhost_user_vga_virgl(self):
qemu_sock.close()
vugp.terminate()
vugp.wait()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 18/24] tests/functional: Convert most ppc avocado tests into standalone tests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (16 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 17/24] tests/functional: Convert the virtio_gpu " Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 19/24] tests/functional: Convert the ppc_amiga avocado test into a standalone test Daniel P. Berrangé
` (6 subsequent siblings)
24 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Nothing thrilling in here, just straight forward conversions.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
MAINTAINERS | 8 +--
tests/functional/meson.build | 11 ++++
.../test_ppc64_powernv.py} | 56 ++++++-----------
.../test_ppc64_pseries.py} | 56 ++++++-----------
.../test_ppc_40p.py} | 63 +++++++++----------
.../test_ppc_mpc8544ds.py} | 28 +++++----
.../test_ppc_virtex_ml507.py} | 28 +++++----
7 files changed, 109 insertions(+), 141 deletions(-)
rename tests/{avocado/ppc_powernv.py => functional/test_ppc64_powernv.py} (71%)
mode change 100644 => 100755
rename tests/{avocado/ppc_pseries.py => functional/test_ppc64_pseries.py} (76%)
mode change 100644 => 100755
rename tests/{avocado/ppc_prep_40p.py => functional/test_ppc_40p.py} (54%)
mode change 100644 => 100755
rename tests/{avocado/ppc_mpc8544ds.py => functional/test_ppc_mpc8544ds.py} (55%)
mode change 100644 => 100755
rename tests/{avocado/ppc_virtex_ml507.py => functional/test_ppc_virtex_ml507.py} (60%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index e1d15c9696..758daad29e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1449,7 +1449,7 @@ L: qemu-ppc@nongnu.org
S: Orphan
F: hw/ppc/mpc8544ds.c
F: hw/ppc/mpc8544_guts.c
-F: tests/avocado/ppc_mpc8544ds.py
+F: tests/functional/test_ppc_mpc8544ds.py
New World (mac99)
M: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
@@ -1502,7 +1502,7 @@ F: hw/dma/i82374.c
F: hw/rtc/m48t59-isa.c
F: include/hw/isa/pc87312.h
F: include/hw/rtc/m48t59.h
-F: tests/avocado/ppc_prep_40p.py
+F: tests/functional/test_ppc_40p.py
sPAPR (pseries)
M: Nicholas Piggin <npiggin@gmail.com>
@@ -1526,7 +1526,7 @@ F: tests/qtest/spapr*
F: tests/qtest/libqos/*spapr*
F: tests/qtest/rtas*
F: tests/qtest/libqos/rtas*
-F: tests/avocado/ppc_pseries.py
+F: tests/functional/test_ppc64_pseries.py
F: tests/avocado/ppc_hv_tests.py
PowerNV (Non-Virtualized)
@@ -1558,7 +1558,7 @@ M: Edgar E. Iglesias <edgar.iglesias@gmail.com>
L: qemu-ppc@nongnu.org
S: Odd Fixes
F: hw/ppc/virtex_ml507.c
-F: tests/avocado/ppc_virtex_ml507.py
+F: tests/functional/test_ppc_virtex_ml507.py
sam460ex
M: BALATON Zoltan <balaton@eik.bme.hu>
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 8fbd7c4129..b4f5c9e38e 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -12,6 +12,9 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
'netdev_ethtool' : 180,
+ 'ppc_40p' : 240,
+ 'ppc64_powernv' : 120,
+ 'ppc64_pseries' : 120,
's390x_ccw_virtio' : 180,
}
@@ -52,7 +55,15 @@ tests_ppc_quick = [
tests_ppc_thorough = [
'ppc_405',
+ 'ppc_40p',
'ppc_bamboo',
+ 'ppc_mpc8544ds',
+ 'ppc_virtex_ml507',
+]
+
+tests_ppc64_thorough = [
+ 'ppc64_powernv',
+ 'ppc64_pseries',
]
tests_riscv32_quick = [
diff --git a/tests/avocado/ppc_powernv.py b/tests/functional/test_ppc64_powernv.py
old mode 100644
new mode 100755
similarity index 71%
rename from tests/avocado/ppc_powernv.py
rename to tests/functional/test_ppc64_powernv.py
index 4342941d5d..79cf8cb20c
--- a/tests/avocado/ppc_powernv.py
+++ b/tests/functional/test_ppc64_powernv.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc powernv machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class powernvMachine(QemuSystemTest):
@@ -16,13 +17,14 @@ class powernvMachine(QemuSystemTest):
panic_message = 'Kernel panic - not syncing'
good_message = 'VFS: Cannot open root device'
+ ASSET_KERNEL = Asset(('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/29/Everything/ppc64le/os'
+ '/ppc/ppc64/vmlinuz'),
+ '3fe04abfc852b66653b8c3c897a59a689270bc77')
+
def do_test_linux_boot(self, command_line = KERNEL_COMMON_COMMAND_LINE):
self.require_accelerator("tcg")
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/29/Everything/ppc64le/os'
- '/ppc/ppc64/vmlinuz')
- kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console()
self.vm.add_args('-kernel', kernel_path,
@@ -30,23 +32,13 @@ def do_test_linux_boot(self, command_line = KERNEL_COMMON_COMMAND_LINE):
self.vm.launch()
def test_linux_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.do_test_linux_boot()
console_pattern = 'VFS: Cannot open root device'
wait_for_console_pattern(self, console_pattern, self.panic_message)
def test_linux_smp_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '4')
self.do_test_linux_boot()
console_pattern = 'smp: Brought up 1 node, 4 CPUs'
@@ -54,12 +46,7 @@ def test_linux_smp_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_linux_smp_hpt_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '4')
self.do_test_linux_boot(self.KERNEL_COMMON_COMMAND_LINE +
'disable_radix')
@@ -70,12 +57,7 @@ def test_linux_smp_hpt_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_linux_smt_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '4,threads=4')
self.do_test_linux_boot()
console_pattern = 'CPU maps initialized for 4 threads per core'
@@ -85,12 +67,7 @@ def test_linux_smt_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_linux_big_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
-
+ self.set_machine('powernv')
self.vm.add_args('-smp', '16,threads=4,cores=2,sockets=2')
# powernv does not support NUMA
@@ -100,3 +77,6 @@ def test_linux_big_boot(self):
console_pattern = 'smp: Brought up 2 nodes, 16 CPUs'
wait_for_console_pattern(self, console_pattern, self.panic_message)
wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_pseries.py b/tests/functional/test_ppc64_pseries.py
old mode 100644
new mode 100755
similarity index 76%
rename from tests/avocado/ppc_pseries.py
rename to tests/functional/test_ppc64_pseries.py
index 74aaa4ac4a..a09b661b15
--- a/tests/avocado/ppc_pseries.py
+++ b/tests/functional/test_ppc64_pseries.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class pseriesMachine(QemuSystemTest):
@@ -16,12 +17,13 @@ class pseriesMachine(QemuSystemTest):
panic_message = 'Kernel panic - not syncing'
good_message = 'VFS: Cannot open root device'
+ ASSET_KERNEL = Asset(('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/29/Everything/ppc64le/os'
+ '/ppc/ppc64/vmlinuz'),
+ '3fe04abfc852b66653b8c3c897a59a689270bc77')
+
def do_test_ppc64_linux_boot(self, kernel_command_line = KERNEL_COMMON_COMMAND_LINE):
- kernel_url = ('https://archives.fedoraproject.org/pub/archive'
- '/fedora-secondary/releases/29/Everything/ppc64le/os'
- '/ppc/ppc64/vmlinuz')
- kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console()
self.vm.add_args('-kernel', kernel_path,
@@ -29,32 +31,20 @@ def do_test_ppc64_linux_boot(self, kernel_command_line = KERNEL_COMMON_COMMAND_L
self.vm.launch()
def test_ppc64_vof_linux_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-machine', 'x-vof=on')
self.do_test_ppc64_linux_boot()
console_pattern = 'VFS: Cannot open root device'
wait_for_console_pattern(self, console_pattern, self.panic_message)
def test_ppc64_linux_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.do_test_ppc64_linux_boot()
console_pattern = 'VFS: Cannot open root device'
wait_for_console_pattern(self, console_pattern, self.panic_message)
def test_ppc64_linux_smp_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-smp', '4')
self.do_test_ppc64_linux_boot()
console_pattern = 'smp: Brought up 1 node, 4 CPUs'
@@ -62,11 +52,7 @@ def test_ppc64_linux_smp_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_ppc64_linux_hpt_smp_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-smp', '4')
self.do_test_ppc64_linux_boot(self.KERNEL_COMMON_COMMAND_LINE +
'disable_radix')
@@ -77,11 +63,6 @@ def test_ppc64_linux_hpt_smp_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_ppc64_linux_smt_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
self.vm.add_args('-smp', '4,threads=4')
self.do_test_ppc64_linux_boot()
console_pattern = 'CPU maps initialized for 4 threads per core'
@@ -91,11 +72,7 @@ def test_ppc64_linux_smt_boot(self):
wait_for_console_pattern(self, self.good_message, self.panic_message)
def test_ppc64_linux_big_boot(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- """
-
+ self.set_machine('pseries')
self.vm.add_args('-smp', '16,threads=4,cores=2,sockets=2')
self.vm.add_args('-m', '512M',
'-object', 'memory-backend-ram,size=256M,id=m0',
@@ -108,3 +85,6 @@ def test_ppc64_linux_big_boot(self):
console_pattern = 'smp: Brought up 2 nodes, 16 CPUs'
wait_for_console_pattern(self, console_pattern, self.panic_message)
wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_prep_40p.py b/tests/functional/test_ppc_40p.py
old mode 100644
new mode 100755
similarity index 54%
rename from tests/avocado/ppc_prep_40p.py
rename to tests/functional/test_ppc_40p.py
index d4f1eb7e1d..711b521211
--- a/tests/avocado/ppc_prep_40p.py
+++ b/tests/functional/test_ppc_40p.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a PReP/40p machine and checks its serial console.
#
# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
@@ -7,15 +9,29 @@
import os
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class IbmPrep40pMachine(QemuSystemTest):
timeout = 60
+ ASSET_BIOS = Asset(
+ ('http://ftpmirror.your.org/pub/misc/'
+ 'ftp.software.ibm.com/rs6000/firmware/'
+ '7020-40p/P12H0456.IMG'),
+ '1775face4e6dc27f3a6ed955ef6eb331bf817f03')
+ ASSET_NETBSD40 = Asset(
+ ('https://archive.netbsd.org/pub/NetBSD-archive/'
+ 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs'),
+ 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb')
+ ASSET_NETBSD71 = Asset(
+ ('https://archive.netbsd.org/pub/NetBSD-archive/'
+ 'NetBSD-7.1.2/iso/NetBSD-7.1.2-prep.iso'),
+ 'cc7cb290b06aaa839362deb7bd9f417ac5015557db24088508330f76c3f825ec')
+
# 12H0455 PPS Firmware Licensed Materials
# Property of IBM (C) Copyright IBM Corp. 1994.
# All rights reserved.
@@ -23,23 +39,10 @@ class IbmPrep40pMachine(QemuSystemTest):
# restricted by GSA ADP Schedule Contract with IBM Corp.
@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
def test_factory_firmware_and_netbsd(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:40p
- :avocado: tags=os:netbsd
- :avocado: tags=slowness:high
- :avocado: tags=accel:tcg
- """
+ self.set_machine('40p')
self.require_accelerator("tcg")
- bios_url = ('http://ftpmirror.your.org/pub/misc/'
- 'ftp.software.ibm.com/rs6000/firmware/'
- '7020-40p/P12H0456.IMG')
- bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03'
- bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash)
- drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/'
- 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs')
- drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb'
- drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash)
+ bios_path = self.ASSET_BIOS.fetch()
+ drive_path = self.ASSET_NETBSD40.fetch()
self.vm.set_console()
self.vm.add_args('-bios', bios_path,
@@ -50,11 +53,7 @@ def test_factory_firmware_and_netbsd(self):
wait_for_console_pattern(self, 'Model: IBM PPS Model 6015')
def test_openbios_192m(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:40p
- :avocado: tags=accel:tcg
- """
+ self.set_machine('40p')
self.require_accelerator("tcg")
self.vm.set_console()
self.vm.add_args('-m', '192') # test fw_cfg
@@ -65,21 +64,15 @@ def test_openbios_192m(self):
wait_for_console_pattern(self, '>> CPU type PowerPC,604')
def test_openbios_and_netbsd(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:40p
- :avocado: tags=os:netbsd
- :avocado: tags=accel:tcg
- """
+ self.set_machine('40p')
self.require_accelerator("tcg")
- drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/'
- 'NetBSD-7.1.2/iso/NetBSD-7.1.2-prep.iso')
- drive_hash = 'ac6fa2707d888b36d6fa64de6e7fe48e'
- drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash,
- algorithm='md5')
+ drive_path = self.ASSET_NETBSD71.fetch()
self.vm.set_console()
self.vm.add_args('-cdrom', drive_path,
'-boot', 'd')
self.vm.launch()
wait_for_console_pattern(self, 'NetBSD/prep BOOT, Revision 1.9')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_mpc8544ds.py b/tests/functional/test_ppc_mpc8544ds.py
old mode 100644
new mode 100755
similarity index 55%
rename from tests/avocado/ppc_mpc8544ds.py
rename to tests/functional/test_ppc_mpc8544ds.py
index b599fb1cc9..859cdcac6c
--- a/tests/avocado/ppc_mpc8544ds.py
+++ b/tests/functional/test_ppc_mpc8544ds.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,9 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class Mpc8544dsMachine(QemuSystemTest):
@@ -15,20 +17,20 @@ class Mpc8544dsMachine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
panic_message = 'Kernel panic - not syncing'
+ ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io'
+ '/qac-best-of-multiarch/download/day04.tar.xz'),
+ 'f46724d281a9f30fa892d458be7beb7d34dc25f9')
+
def test_ppc_mpc8544ds(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:mpc8544ds
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day04.tar.xz')
- tar_hash = 'f46724d281a9f30fa892d458be7beb7d34dc25f9'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ self.set_machine('mpc8544ds')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir, member='creek/creek.bin')
self.vm.set_console()
self.vm.add_args('-kernel', self.workdir + '/creek/creek.bin')
self.vm.launch()
wait_for_console_pattern(self, 'QEMU advent calendar 2020',
self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_virtex_ml507.py b/tests/functional/test_ppc_virtex_ml507.py
old mode 100644
new mode 100755
similarity index 60%
rename from tests/avocado/ppc_virtex_ml507.py
rename to tests/functional/test_ppc_virtex_ml507.py
index a73f8ae396..b567f7e4e8
--- a/tests/avocado/ppc_virtex_ml507.py
+++ b/tests/functional/test_ppc_virtex_ml507.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test that Linux kernel boots on ppc machines and check the console
#
# Copyright (c) 2018, 2020 Red Hat, Inc.
@@ -5,9 +7,9 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern
class VirtexMl507Machine(QemuSystemTest):
@@ -15,18 +17,15 @@ class VirtexMl507Machine(QemuSystemTest):
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
panic_message = 'Kernel panic - not syncing'
+ ASSET_IMAGE = Asset(('https://qemu-advcal.gitlab.io'
+ '/qac-best-of-multiarch/download/day08.tar.xz'),
+ '74c68f5af7a7b8f21c03097b298f3bb77ff52c1f')
+
def test_ppc_virtex_ml507(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:virtex-ml507
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
- tar_url = ('https://qemu-advcal.gitlab.io'
- '/qac-best-of-multiarch/download/day08.tar.xz')
- tar_hash = '74c68f5af7a7b8f21c03097b298f3bb77ff52c1f'
- file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
- archive.extract(file_path, self.workdir)
+ self.set_machine('virtex-ml507')
+ file_path = self.ASSET_IMAGE.fetch()
+ archive_extract(file_path, self.workdir)
self.vm.set_console()
self.vm.add_args('-kernel', self.workdir + '/hippo/hippo.linux',
'-dtb', self.workdir + '/hippo/virtex440-ml507.dtb',
@@ -34,3 +33,6 @@ def test_ppc_virtex_ml507(self):
self.vm.launch()
wait_for_console_pattern(self, 'QEMU advent calendar 2020',
self.panic_message)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 19/24] tests/functional: Convert the ppc_amiga avocado test into a standalone test
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (17 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 18/24] tests/functional: Convert most ppc avocado tests into standalone tests Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-02 16:19 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 20/24] tests/functional: Convert the ppc_hv " Daniel P. Berrangé
` (5 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Use the Python standard zipfile module instead of avocado.utils for
extracting the ZIP file that we download here, and use the standard
subprocess module for running the "tail" command.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/ppc_amiga.py | 38 ---------------------------
tests/functional/meson.build | 1 +
tests/functional/test_ppc_amiga.py | 42 ++++++++++++++++++++++++++++++
3 files changed, 43 insertions(+), 38 deletions(-)
delete mode 100644 tests/avocado/ppc_amiga.py
create mode 100755 tests/functional/test_ppc_amiga.py
diff --git a/tests/avocado/ppc_amiga.py b/tests/avocado/ppc_amiga.py
deleted file mode 100644
index b6f866f91d..0000000000
--- a/tests/avocado/ppc_amiga.py
+++ /dev/null
@@ -1,38 +0,0 @@
-# Test AmigaNG boards
-#
-# Copyright (c) 2023 BALATON Zoltan
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-from avocado.utils import archive
-from avocado.utils import process
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-
-class AmigaOneMachine(QemuSystemTest):
-
- timeout = 90
-
- def test_ppc_amigaone(self):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=machine:amigaone
- :avocado: tags=device:articia
- :avocado: tags=accel:tcg
- """
- self.require_accelerator("tcg")
- tar_name = 'A1Firmware_Floppy_05-Mar-2005.zip'
- tar_url = ('https://www.hyperion-entertainment.com/index.php/'
- 'downloads?view=download&format=raw&file=25')
- tar_hash = 'c52e59bc73e31d8bcc3cc2106778f7ac84f6c755'
- zip_file = self.fetch_asset(tar_name, locations=tar_url,
- asset_hash=tar_hash)
- archive.extract(zip_file, self.workdir)
- cmd = f"tail -c 524288 {self.workdir}/floppy_edition/updater.image >{self.workdir}/u-boot-amigaone.bin"
- process.run(cmd, shell=True)
-
- self.vm.set_console()
- self.vm.add_args('-bios', self.workdir + '/u-boot-amigaone.bin')
- self.vm.launch()
- wait_for_console_pattern(self, 'FLASH:')
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index b4f5c9e38e..fba3891e16 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -56,6 +56,7 @@ tests_ppc_quick = [
tests_ppc_thorough = [
'ppc_405',
'ppc_40p',
+ 'ppc_amiga',
'ppc_bamboo',
'ppc_mpc8544ds',
'ppc_virtex_ml507',
diff --git a/tests/functional/test_ppc_amiga.py b/tests/functional/test_ppc_amiga.py
new file mode 100755
index 0000000000..8c356e1ba7
--- /dev/null
+++ b/tests/functional/test_ppc_amiga.py
@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+#
+# Test AmigaNG boards
+#
+# Copyright (c) 2023 BALATON Zoltan
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import subprocess
+
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern, run_cmd
+from zipfile import ZipFile
+
+class AmigaOneMachine(QemuSystemTest):
+
+ timeout = 90
+
+ ASSET_IMAGE = Asset(('https://www.hyperion-entertainment.com/index.php/'
+ 'downloads?view=download&format=raw&file=25'),
+ 'c52e59bc73e31d8bcc3cc2106778f7ac84f6c755')
+
+ def test_ppc_amigaone(self):
+ self.require_accelerator("tcg")
+ self.set_machine('amigaone')
+ tar_name = 'A1Firmware_Floppy_05-Mar-2005.zip'
+ zip_file = self.ASSET_IMAGE.fetch()
+ with ZipFile(zip_file, 'r') as zf:
+ zf.extractall(path=self.workdir)
+ bios_fh = open(self.workdir + "/u-boot-amigaone.bin", "wb")
+ subprocess.run(['tail', '-c', '524288',
+ self.workdir + "/floppy_edition/updater.image"],
+ stdout=bios_fh)
+
+ self.vm.set_console()
+ self.vm.add_args('-bios', self.workdir + '/u-boot-amigaone.bin')
+ self.vm.launch()
+ wait_for_console_pattern(self, 'FLASH:')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 20/24] tests/functional: Convert the ppc_hv avocado test into a standalone test
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (18 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 19/24] tests/functional: Convert the ppc_amiga avocado test into a standalone test Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-05 22:58 ` Nicholas Piggin
2024-07-30 17:03 ` [PATCH v3 21/24] tests/functional: Convert the m68k nextcube test with tesseract Daniel P. Berrangé
` (4 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Note: The original Avocado test seems currently to be broken, it hangs
when the guest is trying to install additional packages. So mark it as
broken for now until it gets fixed.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
.../test_ppc64_hv.py} | 48 ++++++++-----------
1 file changed, 20 insertions(+), 28 deletions(-)
rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (88%)
mode change 100644 => 100755
diff --git a/tests/avocado/ppc_hv_tests.py b/tests/functional/test_ppc64_hv.py
old mode 100644
new mode 100755
similarity index 88%
rename from tests/avocado/ppc_hv_tests.py
rename to tests/functional/test_ppc64_hv.py
index bf8822bb97..a45657c87e
--- a/tests/avocado/ppc_hv_tests.py
+++ b/tests/functional/test_ppc64_hv.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Tests that specifically try to exercise hypervisor features of the
# target machines. powernv supports the Power hypervisor ISA, and
# pseries supports the nested-HV hypervisor spec.
@@ -7,10 +9,10 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado import skipIf, skipUnless
-from avocado.utils import archive
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern, exec_command
+from unittest import skipIf, skipUnless
+from qemu_test.utils import archive_extract
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import wait_for_console_pattern, exec_command
import os
import time
import subprocess
@@ -47,7 +49,7 @@ def missing_deps():
@skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps))
@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test sometimes gets stuck due to console handling problem')
@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
-@skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
+@skip('broken test - guest fails to install packages')
class HypervisorTest(QemuSystemTest):
timeout = 1000
@@ -55,6 +57,12 @@ class HypervisorTest(QemuSystemTest):
panic_message = 'Kernel panic - not syncing'
good_message = 'VFS: Cannot open root device'
+ # Alpine use sha256 so I recalculated this myself
+ ASSET_ISO = Asset(
+ ('https://dl-cdn.alpinelinux.org/alpine/v3.18/'
+ 'releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso'),
+ 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff')
+
def extract_from_iso(self, iso, path):
"""
Extracts a file from an iso file into the test workdir
@@ -84,14 +92,7 @@ def extract_from_iso(self, iso, path):
def setUp(self):
super().setUp()
- iso_url = ('https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso')
-
- # Alpine use sha256 so I recalculated this myself
- iso_sha256 = 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff'
- iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha256,
- algorithm = "sha256")
-
- self.iso_path = iso_path
+ self.iso_path = self.ASSET_ISO.fetch()
self.vmlinuz = self.extract_from_iso(iso_path, '/boot/vmlinuz-lts')
self.initramfs = self.extract_from_iso(iso_path, '/boot/initramfs-lts')
@@ -159,12 +160,8 @@ def do_test_kvm(self, hpt=False):
wait_for_console_pattern(self, 'alpine:~#')
def test_hv_pseries(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
+ self.set_machine('pseries')
self.vm.add_args("-accel", "tcg,thread=multi")
self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on")
@@ -174,12 +171,8 @@ def test_hv_pseries(self):
self.do_stop_alpine()
def test_hv_pseries_kvm(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:pseries
- :avocado: tags=accel:kvm
- """
self.require_accelerator("kvm")
+ self.set_machine('pseries')
self.vm.add_args("-accel", "kvm")
self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on,cap-ccf-assist=off")
@@ -189,12 +182,8 @@ def test_hv_pseries_kvm(self):
self.do_stop_alpine()
def test_hv_powernv(self):
- """
- :avocado: tags=arch:ppc64
- :avocado: tags=machine:powernv
- :avocado: tags=accel:tcg
- """
self.require_accelerator("tcg")
+ self.set_machine('powernv')
self.vm.add_args("-accel", "tcg,thread=multi")
self.vm.add_args('-device', 'nvme,bus=pcie.2,addr=0x0,serial=1234,drive=drive0',
'-device', 'e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0',
@@ -204,3 +193,6 @@ def test_hv_powernv(self):
self.do_test_kvm()
self.do_test_kvm(True)
self.do_stop_alpine()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 21/24] tests/functional: Convert the m68k nextcube test with tesseract
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (19 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 20/24] tests/functional: Convert the ppc_hv " Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:33 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test Daniel P. Berrangé
` (3 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
The code that handles running of tesseract needs to be tweaked a little
bit to be able to run without the functions from avocado.utils, and
while we're at it, drop some legacy stuff that was still there due to
Tesseract 3 support that we already dropped a while ago.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/avocado/tesseract_utils.py | 46 -------------------
tests/functional/meson.build | 4 ++
tests/functional/qemu_test/tesseract.py | 35 ++++++++++++++
.../test_m68k_nextcube.py} | 29 ++++++------
4 files changed, 55 insertions(+), 59 deletions(-)
delete mode 100644 tests/avocado/tesseract_utils.py
create mode 100644 tests/functional/qemu_test/tesseract.py
rename tests/{avocado/machine_m68k_nextcube.py => functional/test_m68k_nextcube.py} (76%)
mode change 100644 => 100755
diff --git a/tests/avocado/tesseract_utils.py b/tests/avocado/tesseract_utils.py
deleted file mode 100644
index 476f528147..0000000000
--- a/tests/avocado/tesseract_utils.py
+++ /dev/null
@@ -1,46 +0,0 @@
-# ...
-#
-# Copyright (c) 2019 Philippe Mathieu-Daudé <f4bug@amsat.org>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-import re
-import logging
-
-from avocado.utils import process
-from avocado.utils.path import find_command, CmdNotFoundError
-
-def tesseract_available(expected_version):
- try:
- find_command('tesseract')
- except CmdNotFoundError:
- return False
- res = process.run('tesseract --version')
- try:
- version = res.stdout_text.split()[1]
- except IndexError:
- version = res.stderr_text.split()[1]
- return int(version.split('.')[0]) >= expected_version
-
- match = re.match(r'tesseract\s(\d)', res)
- if match is None:
- return False
- # now this is guaranteed to be a digit
- return int(match.groups()[0]) >= expected_version
-
-
-def tesseract_ocr(image_path, tesseract_args='', tesseract_version=3):
- console_logger = logging.getLogger('tesseract')
- console_logger.debug(image_path)
- if tesseract_version == 4:
- tesseract_args += ' --oem 1'
- proc = process.run("tesseract {} {} stdout".format(tesseract_args,
- image_path))
- lines = []
- for line in proc.stdout_text.split('\n'):
- sline = line.strip()
- if len(sline):
- console_logger.debug(sline)
- lines += [sline]
- return lines
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index fba3891e16..46b6405a58 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -37,6 +37,10 @@ tests_loongarch64_thorough = [
'loongarch64_virt',
]
+tests_m68k_thorough = [
+ 'm68k_nextcube'
+]
+
tests_microblaze_thorough = [
'microblaze_s3adsp1800'
]
diff --git a/tests/functional/qemu_test/tesseract.py b/tests/functional/qemu_test/tesseract.py
new file mode 100644
index 0000000000..c4087b7c11
--- /dev/null
+++ b/tests/functional/qemu_test/tesseract.py
@@ -0,0 +1,35 @@
+# ...
+#
+# Copyright (c) 2019 Philippe Mathieu-Daudé <f4bug@amsat.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import re
+import logging
+
+from . import has_cmd, run_cmd
+
+def tesseract_available(expected_version):
+ if not has_cmd('tesseract'):
+ return False
+ (stdout, stderr, ret) = run_cmd([ 'tesseract', '--version'])
+ if ret:
+ return False
+ version = stdout.split()[1]
+ return int(version.split('.')[0]) >= expected_version
+
+def tesseract_ocr(image_path, tesseract_args=''):
+ console_logger = logging.getLogger('console')
+ console_logger.debug(image_path)
+ (stdout, stderr, ret) = run_cmd(['tesseract', image_path,
+ 'stdout'])
+ if ret:
+ return None
+ lines = []
+ for line in stdout.split('\n'):
+ sline = line.strip()
+ if len(sline):
+ console_logger.debug(sline)
+ lines += [sline]
+ return lines
diff --git a/tests/avocado/machine_m68k_nextcube.py b/tests/functional/test_m68k_nextcube.py
old mode 100644
new mode 100755
similarity index 76%
rename from tests/avocado/machine_m68k_nextcube.py
rename to tests/functional/test_m68k_nextcube.py
index 1f3c883910..c394499d8c
--- a/tests/avocado/machine_m68k_nextcube.py
+++ b/tests/functional/test_m68k_nextcube.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a VM and run OCR on the framebuffer
#
# Copyright (c) 2019 Philippe Mathieu-Daudé <f4bug@amsat.org>
@@ -8,10 +10,10 @@
import os
import time
-from avocado_qemu import QemuSystemTest
-from avocado import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from unittest import skipUnless
-from tesseract_utils import tesseract_available, tesseract_ocr
+from qemu_test.tesseract import tesseract_available, tesseract_ocr
PIL_AVAILABLE = True
try:
@@ -21,19 +23,15 @@
class NextCubeMachine(QemuSystemTest):
- """
- :avocado: tags=arch:m68k
- :avocado: tags=machine:next-cube
- :avocado: tags=device:framebuffer
- """
timeout = 15
+ ASSET_ROM = Asset(('https://sourceforge.net/p/previous/code/1350/tree/'
+ 'trunk/src/Rev_2.5_v66.BIN?format=raw'),
+ 'b3534796abae238a0111299fc406a9349f7fee24')
+
def check_bootrom_framebuffer(self, screenshot_path):
- rom_url = ('https://sourceforge.net/p/previous/code/1350/tree/'
- 'trunk/src/Rev_2.5_v66.BIN?format=raw')
- rom_hash = 'b3534796abae238a0111299fc406a9349f7fee24'
- rom_path = self.fetch_asset(rom_url, asset_hash=rom_hash)
+ rom_path = self.ASSET_ROM.fetch()
self.vm.add_args('-bios', rom_path)
self.vm.launch()
@@ -48,6 +46,7 @@ def check_bootrom_framebuffer(self, screenshot_path):
@skipUnless(PIL_AVAILABLE, 'Python PIL not installed')
def test_bootrom_framebuffer_size(self):
+ self.set_machine('next-cube')
screenshot_path = os.path.join(self.workdir, "dump.ppm")
self.check_bootrom_framebuffer(screenshot_path)
@@ -60,11 +59,15 @@ def test_bootrom_framebuffer_size(self):
# that it is still alpha-level software.
@skipUnless(tesseract_available(4), 'tesseract OCR tool not available')
def test_bootrom_framebuffer_ocr_with_tesseract(self):
+ self.set_machine('next-cube')
screenshot_path = os.path.join(self.workdir, "dump.ppm")
self.check_bootrom_framebuffer(screenshot_path)
- lines = tesseract_ocr(screenshot_path, tesseract_version=4)
+ lines = tesseract_ocr(screenshot_path)
text = '\n'.join(lines)
self.assertIn('Testing the FPU', text)
self.assertIn('System test failed. Error code', text)
self.assertIn('Boot command', text)
self.assertIn('Next>', text)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (20 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 21/24] tests/functional: Convert the m68k nextcube test with tesseract Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-02 16:10 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 23/24] tests/functional: Convert the rx_gdbsim avocado " Daniel P. Berrangé
` (2 subsequent siblings)
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Mostly a straight-forward conversion. Looks like we can simply drop
the avocado datadrainer stuff when not using the avocado framework
anymore.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
.../acpi-bits/bits-config/bits-cfg.txt | 0
.../acpi-bits/bits-tests/smbios.py2 | 0
.../acpi-bits/bits-tests/smilatency.py2 | 0
.../acpi-bits/bits-tests/testacpi.py2 | 0
.../acpi-bits/bits-tests/testcpuid.py2 | 0
tests/functional/meson.build | 2 +
.../test_acpi_bits.py} | 81 ++++++++++---------
7 files changed, 43 insertions(+), 40 deletions(-)
rename tests/{avocado => functional}/acpi-bits/bits-config/bits-cfg.txt (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%)
rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%)
rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py} (86%)
mode change 100644 => 100755
diff --git a/tests/avocado/acpi-bits/bits-config/bits-cfg.txt b/tests/functional/acpi-bits/bits-config/bits-cfg.txt
similarity index 100%
rename from tests/avocado/acpi-bits/bits-config/bits-cfg.txt
rename to tests/functional/acpi-bits/bits-config/bits-cfg.txt
diff --git a/tests/avocado/acpi-bits/bits-tests/smbios.py2 b/tests/functional/acpi-bits/bits-tests/smbios.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/smbios.py2
rename to tests/functional/acpi-bits/bits-tests/smbios.py2
diff --git a/tests/avocado/acpi-bits/bits-tests/smilatency.py2 b/tests/functional/acpi-bits/bits-tests/smilatency.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/smilatency.py2
rename to tests/functional/acpi-bits/bits-tests/smilatency.py2
diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 b/tests/functional/acpi-bits/bits-tests/testacpi.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/testacpi.py2
rename to tests/functional/acpi-bits/bits-tests/testacpi.py2
diff --git a/tests/avocado/acpi-bits/bits-tests/testcpuid.py2 b/tests/functional/acpi-bits/bits-tests/testcpuid.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/testcpuid.py2
rename to tests/functional/acpi-bits/bits-tests/testcpuid.py2
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 46b6405a58..439997a85d 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -11,6 +11,7 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
+ 'acpi_bits' : 240,
'netdev_ethtool' : 180,
'ppc_40p' : 240,
'ppc64_powernv' : 120,
@@ -97,6 +98,7 @@ tests_x86_64_quick = [
]
tests_x86_64_thorough = [
+ 'acpi_bits',
'netdev_ethtool',
'virtio_gpu',
]
diff --git a/tests/avocado/acpi-bits.py b/tests/functional/test_acpi_bits.py
old mode 100644
new mode 100755
similarity index 86%
rename from tests/avocado/acpi-bits.py
rename to tests/functional/test_acpi_bits.py
index efe4f52ee0..e9f8381f84
--- a/tests/avocado/acpi-bits.py
+++ b/tests/functional/test_acpi_bits.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-# group: rw quick
+#
# Exercise QEMU generated ACPI/SMBIOS tables using biosbits,
# https://biosbits.org/
#
@@ -41,15 +41,16 @@
import tempfile
import time
import zipfile
+
+from pathlib import Path
from typing import (
List,
Optional,
Sequence,
)
from qemu.machine import QEMUMachine
-from avocado import skipIf
-from avocado.utils import datadrainer as drainer
-from avocado_qemu import QemuBaseTest
+from unittest import skipIf
+from qemu_test import QemuBaseTest, Asset
deps = ["xorriso", "mformat"] # dependent tools needed in the test setup/box.
supported_platforms = ['x86_64'] # supported test platforms.
@@ -129,34 +130,32 @@ def base_args(self):
class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
"""
ACPI and SMBIOS tests using biosbits.
-
- :avocado: tags=arch:x86_64
- :avocado: tags=acpi
-
"""
# in slower systems the test can take as long as 3 minutes to complete.
timeout = BITS_TIMEOUT
+ # following are some standard configuration constants
+ # gitlab CI does shallow clones of depth 20
+ BITS_INTERNAL_VER = 2020
+ # commit hash must match the artifact tag below
+ BITS_COMMIT_HASH = 'c7920d2b'
+ # this is the latest bits release as of today.
+ BITS_TAG = "qemu-bits-10262023"
+
+ ASSET_BITS = Asset(("https://gitlab.com/qemu-project/"
+ "biosbits-bits/-/jobs/artifacts/%s/"
+ "download?job=qemu-bits-build" % BITS_TAG),
+ 'b22cdfcfc7453875297d06d626f5474ee36a343f')
+
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._vm = None
self._workDir = None
self._baseDir = None
- # following are some standard configuration constants
- self._bitsInternalVer = 2020 # gitlab CI does shallow clones of depth 20
- self._bitsCommitHash = 'c7920d2b' # commit hash must match
- # the artifact tag below
- self._bitsTag = "qemu-bits-10262023" # this is the latest bits
- # release as of today.
- self._bitsArtSHA1Hash = 'b22cdfcfc7453875297d06d626f5474ee36a343f'
- self._bitsArtURL = ("https://gitlab.com/qemu-project/"
- "biosbits-bits/-/jobs/artifacts/%s/"
- "download?job=qemu-bits-build" %self._bitsTag)
self._debugcon_addr = '0x403'
self._debugcon_log = 'debugcon-log.txt'
- logging.basicConfig(level=logging.INFO)
- self.logger = logging.getLogger('acpi-bits')
+ self.logger = self.log
def _print_log(self, log):
self.logger.info('\nlogs from biosbits follows:')
@@ -171,7 +170,7 @@ def copy_bits_config(self):
bits_config_dir = os.path.join(self._baseDir, 'acpi-bits',
'bits-config')
target_config_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer,
+ 'bits-%d' %self.BITS_INTERNAL_VER,
'boot')
self.assertTrue(os.path.exists(bits_config_dir))
self.assertTrue(os.path.exists(target_config_dir))
@@ -188,7 +187,7 @@ def copy_test_scripts(self):
bits_test_dir = os.path.join(self._baseDir, 'acpi-bits',
'bits-tests')
target_test_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer,
+ 'bits-%d' %self.BITS_INTERNAL_VER,
'boot', 'python')
self.assertTrue(os.path.exists(bits_test_dir))
@@ -248,9 +247,9 @@ def generate_bits_iso(self):
test scripts
"""
bits_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer)
+ 'bits-%d' %self.BITS_INTERNAL_VER)
iso_file = os.path.join(self._workDir,
- 'bits-%d.iso' %self._bitsInternalVer)
+ 'bits-%d.iso' %self.BITS_INTERNAL_VER)
mkrescue_script = os.path.join(self._workDir,
'grub-inst-x86_64-efi', 'bin',
'grub-mkrescue')
@@ -264,8 +263,12 @@ def generate_bits_iso(self):
try:
if os.getenv('V') or os.getenv('BITS_DEBUG'):
- subprocess.check_call([mkrescue_script, '-o', iso_file,
- bits_dir], stderr=subprocess.STDOUT)
+ proc = subprocess.run([mkrescue_script, '-o', iso_file,
+ bits_dir],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ check=True)
+ self.logger.info("grub-mkrescue output %s" % proc.stdout)
else:
subprocess.check_call([mkrescue_script, '-o',
iso_file, bits_dir],
@@ -282,8 +285,9 @@ def generate_bits_iso(self):
def setUp(self): # pylint: disable=arguments-differ
super().setUp('qemu-system-')
+ self.logger = self.log
- self._baseDir = os.getenv('AVOCADO_TEST_BASEDIR')
+ self._baseDir = Path(__file__).parent
# workdir could also be avocado's own workdir in self.workdir.
# At present, I prefer to maintain my own temporary working
@@ -300,15 +304,14 @@ def setUp(self): # pylint: disable=arguments-differ
os.mkdir(prebuiltDir, mode=0o775)
bits_zip_file = os.path.join(prebuiltDir, 'bits-%d-%s.zip'
- %(self._bitsInternalVer,
- self._bitsCommitHash))
+ %(self.BITS_INTERNAL_VER,
+ self.BITS_COMMIT_HASH))
grub_tar_file = os.path.join(prebuiltDir,
'bits-%d-%s-grub.tar.gz'
- %(self._bitsInternalVer,
- self._bitsCommitHash))
+ %(self.BITS_INTERNAL_VER,
+ self.BITS_COMMIT_HASH))
- bitsLocalArtLoc = self.fetch_asset(self._bitsArtURL,
- asset_hash=self._bitsArtSHA1Hash)
+ bitsLocalArtLoc = self.ASSET_BITS.fetch()
self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLoc)
# extract the bits artifact in the temp working directory
@@ -369,7 +372,7 @@ def test_acpi_smbios_bits(self):
"""The main test case implementation."""
iso_file = os.path.join(self._workDir,
- 'bits-%d.iso' %self._bitsInternalVer)
+ 'bits-%d.iso' %self.BITS_INTERNAL_VER)
self.assertTrue(os.access(iso_file, os.R_OK))
@@ -393,12 +396,6 @@ def test_acpi_smbios_bits(self):
self._vm.set_console()
self._vm.launch()
- self.logger.debug("Console output from bits VM follows ...")
- c_drainer = drainer.LineLogger(self._vm.console_socket.fileno(),
- logger=self.logger.getChild("console"),
- stop_check=(lambda :
- not self._vm.is_running()))
- c_drainer.start()
# biosbits has been configured to run all the specified test suites
# in batch mode and then automatically initiate a vm shutdown.
@@ -406,4 +403,8 @@ def test_acpi_smbios_bits(self):
# with the avocado test timeout.
self._vm.event_wait('SHUTDOWN', timeout=BITS_TIMEOUT)
self._vm.wait(timeout=None)
+ self.logger.debug("Checking console output ...")
self.parse_log()
+
+if __name__ == '__main__':
+ QemuBaseTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 23/24] tests/functional: Convert the rx_gdbsim avocado test into a standalone test
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (21 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:40 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 24/24] gitlab-ci: Add "check-functional" to the build tests Daniel P. Berrangé
2024-07-30 18:38 ` [PATCH v3 00/24] Convert avocado tests to normal Python unittests Philippe Mathieu-Daudé
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
From: Thomas Huth <thuth@redhat.com>
Provide a "gzip_uncompress" function based on the standard "gzip" module
to avoid the usage of avocado.utils here.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
tests/functional/meson.build | 4 ++
tests/functional/qemu_test/utils.py | 12 ++++
.../test_rx_gdbsim.py} | 57 ++++++++++---------
3 files changed, 45 insertions(+), 28 deletions(-)
rename tests/{avocado/machine_rx_gdbsim.py => functional/test_rx_gdbsim.py} (54%)
mode change 100644 => 100755
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 439997a85d..b4889d8d7a 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -80,6 +80,10 @@ tests_riscv64_quick = [
'riscv_opensbi',
]
+tests_rx_thorough = [
+ 'rx_gdbsim',
+]
+
tests_s390x_thorough = [
's390x_ccw_virtio',
's390x_topology',
diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py
index a12dac51b6..99eae5fc45 100644
--- a/tests/functional/qemu_test/utils.py
+++ b/tests/functional/qemu_test/utils.py
@@ -8,6 +8,7 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
+import gzip
import lzma
import os
import shutil
@@ -23,6 +24,17 @@ def archive_extract(archive, dest_dir, member=None):
else:
tf.extractall(path=dest_dir)
+def gzip_uncompress(gz_path, output_path):
+ if os.path.exists(output_path):
+ return
+ with gzip.open(gz_path, 'rb') as gz_in:
+ try:
+ with open(output_path, 'wb') as raw_out:
+ shutil.copyfileobj(gz_in, raw_out)
+ except:
+ os.remove(output_path)
+ raise
+
def lzma_uncompress(xz_path, output_path):
if os.path.exists(output_path):
return
diff --git a/tests/avocado/machine_rx_gdbsim.py b/tests/functional/test_rx_gdbsim.py
old mode 100644
new mode 100755
similarity index 54%
rename from tests/avocado/machine_rx_gdbsim.py
rename to tests/functional/test_rx_gdbsim.py
index 412a7a5089..9b2dcc4b90
--- a/tests/avocado/machine_rx_gdbsim.py
+++ b/tests/functional/test_rx_gdbsim.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Functional test that boots a Linux kernel and checks the console
#
# Copyright (c) 2018 Red Hat, Inc.
@@ -10,11 +12,11 @@
import os
-from avocado import skipUnless
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import wait_for_console_pattern
-from avocado.utils import archive
+from unittest import skipUnless
+from qemu_test import QemuSystemTest, Asset
+from qemu_test import exec_command_and_wait_for_pattern
+from qemu_test import wait_for_console_pattern
+from qemu_test.utils import gzip_uncompress
class RxGdbSimMachine(QemuSystemTest):
@@ -22,21 +24,24 @@ class RxGdbSimMachine(QemuSystemTest):
timeout = 30
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
+ ASSET_UBOOT = Asset('https://acc.dl.osdn.jp/users/23/23888/u-boot.bin.gz',
+ '9b78dbd43b40b2526848c0b1ce9de02c24f4dcdb')
+ ASSET_DTB = Asset('https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb',
+ '7b4e4e2c71905da44e86ce47adee2210b026ac18')
+ ASSET_KERNEL = Asset('http://acc.dl.osdn.jp/users/23/23845/zImage',
+ '39a81067f8d72faad90866ddfefa19165d68fc99')
+ @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'),
+ 'Test is unstable on GitLab')
def test_uboot(self):
"""
U-Boot and checks that the console is operational.
-
- :avocado: tags=arch:rx
- :avocado: tags=machine:gdbsim-r5f562n8
- :avocado: tags=endian:little
- :avocado: tags=flaky
"""
- uboot_url = ('https://acc.dl.osdn.jp/users/23/23888/u-boot.bin.gz')
- uboot_hash = '9b78dbd43b40b2526848c0b1ce9de02c24f4dcdb'
- uboot_path = self.fetch_asset(uboot_url, asset_hash=uboot_hash)
- uboot_path = archive.uncompress(uboot_path, self.workdir)
+ self.set_machine('gdbsim-r5f562n8')
+
+ uboot_path_gz = self.ASSET_UBOOT.fetch()
+ uboot_path = os.path.join(self.workdir, 'u-boot.bin')
+ gzip_uncompress(uboot_path_gz, uboot_path)
self.vm.set_console()
self.vm.add_args('-bios', uboot_path,
@@ -48,23 +53,16 @@ def test_uboot(self):
# FIXME limit baudrate on chardev, else we type too fast
#exec_command_and_wait_for_pattern(self, 'version', gcc_version)
- @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
-
+ @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'),
+ 'Test is unstable on GitLab')
def test_linux_sash(self):
"""
Boots a Linux kernel and checks that the console is operational.
-
- :avocado: tags=arch:rx
- :avocado: tags=machine:gdbsim-r5f562n7
- :avocado: tags=endian:little
- :avocado: tags=flaky
"""
- dtb_url = ('https://acc.dl.osdn.jp/users/23/23887/rx-virt.dtb')
- dtb_hash = '7b4e4e2c71905da44e86ce47adee2210b026ac18'
- dtb_path = self.fetch_asset(dtb_url, asset_hash=dtb_hash)
- kernel_url = ('http://acc.dl.osdn.jp/users/23/23845/zImage')
- kernel_hash = '39a81067f8d72faad90866ddfefa19165d68fc99'
- kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.set_machine('gdbsim-r5f562n7')
+
+ dtb_path = self.ASSET_DTB.fetch()
+ kernel_path = self.ASSET_KERNEL.fetch()
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'earlycon'
@@ -75,3 +73,6 @@ def test_linux_sash(self):
wait_for_console_pattern(self, 'Sash command shell (version 1.1.1)',
failure_message='Kernel panic - not syncing')
exec_command_and_wait_for_pattern(self, 'printenv', 'TERM=linux')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v3 24/24] gitlab-ci: Add "check-functional" to the build tests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (22 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 23/24] tests/functional: Convert the rx_gdbsim avocado " Daniel P. Berrangé
@ 2024-07-30 17:03 ` Daniel P. Berrangé
2024-08-01 16:40 ` Philippe Mathieu-Daudé
2024-07-30 18:38 ` [PATCH v3 00/24] Convert avocado tests to normal Python unittests Philippe Mathieu-Daudé
24 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-07-30 17:03 UTC (permalink / raw)
To: qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth, Daniel P . Berrangé
From: Thomas Huth <thuth@redhat.com>
Now that we converted many tests from the "check-avocado" test suite
to the "check-functional" test suite, we should make sure that these
also get tested in the CI.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
.gitlab-ci.d/buildtest-template.yml | 3 +-
.gitlab-ci.d/buildtest.yml | 60 ++++++++++++++---------------
2 files changed, 32 insertions(+), 31 deletions(-)
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 8f7ebfaed8..54cae74a8e 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -71,7 +71,7 @@
reports:
junit: build/meson-logs/testlog.junit.xml
-.avocado_test_job_template:
+.functional_test_job_template:
extends: .common_test_job_template
cache:
key: "${CI_JOB_NAME}-cache"
@@ -98,6 +98,7 @@
du -chs ${CI_PROJECT_DIR}/avocado-cache ;
fi
- export AVOCADO_ALLOW_UNTRUSTED_CODE=1
+ - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
after_script:
- cd build
- du -chs ${CI_PROJECT_DIR}/avocado-cache
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index aa32782405..1d2afae996 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -22,14 +22,14 @@ check-system-alpine:
IMAGE: alpine
MAKE_CHECK_ARGS: check-unit check-qtest
-avocado-system-alpine:
- extends: .avocado_test_job_template
+functional-system-alpine:
+ extends: .functional_test_job_template
needs:
- job: build-system-alpine
artifacts: true
variables:
IMAGE: alpine
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel
build-system-ubuntu:
@@ -53,14 +53,14 @@ check-system-ubuntu:
IMAGE: ubuntu2204
MAKE_CHECK_ARGS: check
-avocado-system-ubuntu:
- extends: .avocado_test_job_template
+functional-system-ubuntu:
+ extends: .functional_test_job_template
needs:
- job: build-system-ubuntu
artifacts: true
variables:
IMAGE: ubuntu2204
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el
build-system-debian:
@@ -85,14 +85,14 @@ check-system-debian:
IMAGE: debian
MAKE_CHECK_ARGS: check
-avocado-system-debian:
- extends: .avocado_test_job_template
+functional-system-debian:
+ extends: .functional_test_job_template
needs:
- job: build-system-debian
artifacts: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa
crash-test-debian:
@@ -129,14 +129,14 @@ check-system-fedora:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-system-fedora:
- extends: .avocado_test_job_template
+functional-system-fedora:
+ extends: .functional_test_job_template
needs:
- job: build-system-fedora
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k
arch:riscv32 arch:ppc arch:sparc64
@@ -243,14 +243,14 @@ check-system-centos:
IMAGE: centos9
MAKE_CHECK_ARGS: check
-avocado-system-centos:
- extends: .avocado_test_job_template
+functional-system-centos:
+ extends: .functional_test_job_template
needs:
- job: build-system-centos
artifacts: true
variables:
IMAGE: centos9
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx
arch:sh4
@@ -274,14 +274,14 @@ check-system-opensuse:
IMAGE: opensuse-leap
MAKE_CHECK_ARGS: check
-avocado-system-opensuse:
- extends: .avocado_test_job_template
+functional-system-opensuse:
+ extends: .functional_test_job_template
needs:
- job: build-system-opensuse
artifacts: true
variables:
IMAGE: opensuse-leap
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64
#
@@ -302,15 +302,15 @@ build-system-flaky:
ppc64-softmmu rx-softmmu s390x-softmmu sh4-softmmu x86_64-softmmu
MAKE_CHECK_ARGS: check-build
-avocado-system-flaky:
- extends: .avocado_test_job_template
+functional-system-flaky:
+ extends: .functional_test_job_template
needs:
- job: build-system-flaky
artifacts: true
allow_failure: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
QEMU_JOB_OPTIONAL: 1
QEMU_TEST_FLAKY_TESTS: 1
AVOCADO_TAGS: flaky
@@ -485,14 +485,14 @@ check-cfi-aarch64:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-aarch64:
- extends: .avocado_test_job_template
+functional-cfi-aarch64:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-aarch64
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
build-cfi-ppc64-s390x:
extends:
@@ -523,14 +523,14 @@ check-cfi-ppc64-s390x:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-ppc64-s390x:
- extends: .avocado_test_job_template
+functional-cfi-ppc64-s390x:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-ppc64-s390x
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
build-cfi-x86_64:
extends:
@@ -557,14 +557,14 @@ check-cfi-x86_64:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-x86_64:
- extends: .avocado_test_job_template
+functional-cfi-x86_64:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-x86_64
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
tsan-build:
extends: .native_build_job_template
--
2.45.2
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v3 00/24] Convert avocado tests to normal Python unittests
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
` (23 preceding siblings ...)
2024-07-30 17:03 ` [PATCH v3 24/24] gitlab-ci: Add "check-functional" to the build tests Daniel P. Berrangé
@ 2024-07-30 18:38 ` Philippe Mathieu-Daudé
24 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-30 18:38 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> This series is an update to Thomas' v2:
>
> https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg05805.html
>
> wherein Thomas suggested someone could do the asset caching updates
> while he is on vacation:
>
> https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg06228.html
>
> hence this posting what I'm calling a v3.
Changes included in my PR (can be considered as v2.5):
https://lore.kernel.org/qemu-devel/20240729152714.10225-1-philmd@linaro.org/
-- >8 --
diff --git a/MAINTAINERS b/MAINTAINERS
index 98eddf7ae1..a906218f9d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -317,6 +317,7 @@ F: configs/devices/ppc*
F: docs/system/ppc/embedded.rst
F: docs/system/target-ppc.rst
F: tests/tcg/ppc*/*
+F: tests/functional/test_ppc_74xx.py
RISC-V TCG CPUs
M: Palmer Dabbelt <palmer@dabbelt.com>
@@ -336,6 +337,7 @@ F: include/hw/riscv/
F: linux-user/host/riscv32/
F: linux-user/host/riscv64/
F: tests/tcg/riscv64/
+F: tests/functional/test_riscv_opensbi.py
RISC-V XThead* extensions
M: Christoph Muellner <christoph.muellner@vrull.eu>
@@ -1830,6 +1832,9 @@ F: hw/isa/apm.c
F: include/hw/isa/apm.h
F: tests/unit/test-x86-topo.c
F: tests/qtest/test-x86-cpuid-compat.c
+F: tests/functional/test_mem_addr_space.py
+F: tests/functional/test_pc_cpu_hotplug_props.py
+F: tests/functional/test_x86_cpu_model_versions.py
PC Chipset
M: Michael S. Tsirkin <mst@redhat.com>
@@ -1896,6 +1901,8 @@ F: include/hw/boards.h
F: include/hw/core/cpu.h
F: include/hw/cpu/cluster.h
F: include/sysemu/numa.h
+F: tests/functional/test_cpu_queries.py
+F: tests/functional/test_empty_cpu_model.py
F: tests/unit/test-smp-parse.c
T: git https://gitlab.com/ehabkost/qemu.git machine-next
@@ -2101,6 +2108,7 @@ S: Odd Fixes
F: hw/net/
F: include/hw/net/
F: tests/qtest/virtio-net-test.c
+F: tests/avocado/info_usernet.py
F: docs/virtio-net-failover.rst
T: git https://github.com/jasowang/qemu.git net
@@ -2236,6 +2244,7 @@ F: net/vhost-user.c
F: include/hw/virtio/
F: docs/devel/virtio*
F: docs/devel/migration/virtio.rst
+F: tests/functional/test_virtio_version.py
virtio-balloon
M: Michael S. Tsirkin <mst@redhat.com>
@@ -2969,6 +2978,7 @@ S: Supported
F: include/qemu/option.h
F: tests/unit/test-keyval.c
F: tests/unit/test-qemu-opts.c
+F: tests/functional/test_version.py
F: util/keyval.c
F: util/qemu-option.c
@@ -4127,6 +4137,7 @@ F: .travis.yml
F: docs/devel/ci*
F: scripts/ci/
F: tests/docker/
+F: tests/functional/
F: tests/vm/
F: tests/lcitool/
F: tests/avocado/tuxrun_baselines.py
diff --git a/tests/functional/test_info_usernet.py
b/tests/functional/test_info_usernet.py
index ded973253a..cd37524d94 100755
--- a/tests/functional/test_info_usernet.py
+++ b/tests/functional/test_info_usernet.py
@@ -19,7 +19,7 @@ class InfoUsernet(QemuSystemTest):
def test_hostfwd(self):
self.require_netdev('user')
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-netdev',
'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
diff --git a/tests/functional/test_version.py
b/tests/functional/test_version.py
index 5e566d76b1..3ab3b67f7e 100755
--- a/tests/functional/test_version.py
+++ b/tests/functional/test_version.py
@@ -17,7 +17,7 @@
class Version(QemuSystemTest):
def test_qmp_human_info_version(self):
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-nodefaults')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
diff --git a/tests/functional/test_x86_cpu_model_versions.py
b/tests/functional/test_x86_cpu_model_versions.py
index a5f27c737d..a7294b4b92 100755
--- a/tests/functional/test_x86_cpu_model_versions.py
+++ b/tests/functional/test_x86_cpu_model_versions.py
@@ -217,7 +217,7 @@ def test_none_alias(self):
"""
Check if unversioned CPU model is an alias pointing to some
version
"""
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()
---
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets
2024-07-30 17:03 ` [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets Daniel P. Berrangé
@ 2024-07-31 5:01 ` Richard Henderson
2024-08-01 16:01 ` Philippe Mathieu-Daudé
1 sibling, 0 replies; 69+ messages in thread
From: Richard Henderson @ 2024-07-31 5:01 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Ani Sinha, Alex Bennée, Thomas Huth
On 7/31/24 03:03, Daniel P. Berrangé wrote:
> Many tests need to access assets stored on remote sites. We don't want
> to download these during test execution when run by meson, since this
> risks hitting test timeouts when data transfers are slow.
>
> Add support for pre-emptive caching of assets by setting the env var
> QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
> instead of running the test, the assets will be downloaded and saved
> to the cache, then the timestamp file created.
>
> A meson custom target is created as a dependency of each test suite
> to trigger the pre-emptive caching logic before the test runs.
>
> When run in caching mode, it will locate assets by looking for class
> level variables with a name prefix "ASSET_", and type "Asset".
>
> At the ninja level
>
> ninja test --suite functional
>
> will speculatively download any assets that are not already cached,
> so it is advisable to set a timeout multiplier.
>
> QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
>
> will fail the test if a required asset is not already cached
>
> nina check-func-precache
ninja or make, precache-functional.
Anyway, thanks for this. I can confirm that it works, and that it takes ~50m to finish here.
Tested-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 01/24] python: Install pycotap in our venv if necessary
2024-07-30 17:03 ` [PATCH v3 01/24] python: Install pycotap in our venv if necessary Daniel P. Berrangé
@ 2024-07-31 12:49 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-31 12:49 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The upcoming functional tests will require pycotap for providing
> TAP output from the python-based tests. Since we want to be able
> to run some of the tests offline by default, too, let's install
> it along with meson in our venv if necessary (it's size is only
> 5 kB, so adding the wheel here should not really be a problem).
>
> The wheel file has been obtained with:
>
> pip download --only-binary :all: --dest . --no-cache pycotap
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
> pythondeps.toml | 1 +
> 2 files changed, 1 insertion(+)
> create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text
2024-07-30 17:03 ` [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text Daniel P. Berrangé
@ 2024-07-31 12:49 ` Philippe Mathieu-Daudé
2024-08-01 10:00 ` Alex Bennée
1 sibling, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-31 12:49 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The next patch is going to add some entries that need more space between
> the command and the help text, so let's increase the indentation here
> first.
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/Makefile.include | 30 +++++++++++++++---------------
> 1 file changed, 15 insertions(+), 15 deletions(-)
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 06/24] tests/functional: Convert simple avocado tests into standalone python tests
2024-07-30 17:03 ` [PATCH v3 06/24] tests/functional: Convert simple avocado tests into standalone python tests Daniel P. Berrangé
@ 2024-07-31 12:52 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-31 12:52 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> These test are rather simple and don't need any modifications apart
> from adjusting the "from avocado_qemu" line. To ease debugging, make
> the files executable and add a shebang line and Python '__main__'
> handling, too, so that these tests can now be run by executing them
> directly.
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/meson.build | 5 ++
> .../test_cpu_queries.py} | 7 ++-
> .../test_empty_cpu_model.py} | 7 ++-
> .../test_mem_addr_space.py} | 52 +++----------------
> .../test_pc_cpu_hotplug_props.py} | 11 ++--
> .../test_virtio_version.py} | 8 +--
> 6 files changed, 34 insertions(+), 56 deletions(-)
> rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%)
> mode change 100644 => 100755
> rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
> mode change 100644 => 100755
> rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
> mode change 100644 => 100755
> rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
> mode change 100644 => 100755
> rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
> mode change 100644 => 100755
Please squash:
-- >8 --
diff --git a/MAINTAINERS b/MAINTAINERS
index 98eddf7ae1..a906218f9d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1830,6 +1832,9 @@ F: hw/isa/apm.c
F: include/hw/isa/apm.h
F: tests/unit/test-x86-topo.c
F: tests/qtest/test-x86-cpuid-compat.c
+F: tests/functional/test_mem_addr_space.py
+F: tests/functional/test_pc_cpu_hotplug_props.py
+F: tests/functional/test_x86_cpu_model_versions.py
PC Chipset
M: Michael S. Tsirkin <mst@redhat.com>
@@ -1896,6 +1901,8 @@ F: include/hw/boards.h
F: include/hw/core/cpu.h
F: include/hw/cpu/cluster.h
F: include/sysemu/numa.h
+F: tests/functional/test_cpu_queries.py
+F: tests/functional/test_empty_cpu_model.py
F: tests/unit/test-smp-parse.c
T: git https://gitlab.com/ehabkost/qemu.git machine-next
@@ -2236,6 +2244,7 @@ F: net/vhost-user.c
F: include/hw/virtio/
F: docs/devel/virtio*
F: docs/devel/migration/virtio.rst
+F: tests/functional/test_virtio_version.py
virtio-balloon
M: Michael S. Tsirkin <mst@redhat.com>
diff --git a/tests/functional/test_x86_cpu_model_versions.py
b/tests/functional/test_x86_cpu_model_versions.py
index a5f27c737d..a7294b4b92 100755
--- a/tests/functional/test_x86_cpu_model_versions.py
+++ b/tests/functional/test_x86_cpu_model_versions.py
@@ -217,7 +217,7 @@ def test_none_alias(self):
"""
Check if unversioned CPU model is an alias pointing to some
version
"""
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()
---
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
2024-07-30 17:03 ` [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests Daniel P. Berrangé
@ 2024-07-31 12:53 ` Philippe Mathieu-Daudé
2024-07-31 14:24 ` Alex Bennée
2024-08-01 16:11 ` Philippe Mathieu-Daudé
2 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-31 12:53 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py
> file with some adjustments to get rid of the Avocado dependencies (i.e.
> we also have to drop the LinuxSSHMixIn and LinuxTest for now).
>
> The emulator binary and build directory are now passed via
> environment variables that will be set via meson.build later.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> [DB: split __init__.py into multiple files]
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
> tests/functional/qemu_test/__init__.py | 13 ++
> tests/functional/qemu_test/cmd.py | 171 +++++++++++++++++++++++++
> tests/functional/qemu_test/config.py | 36 ++++++
> tests/functional/qemu_test/testcase.py | 154 ++++++++++++++++++++++
> 4 files changed, 374 insertions(+)
> create mode 100644 tests/functional/qemu_test/__init__.py
> create mode 100644 tests/functional/qemu_test/cmd.py
> create mode 100644 tests/functional/qemu_test/config.py
> create mode 100644 tests/functional/qemu_test/testcase.py
Please squash:
-- >8 --
diff --git a/MAINTAINERS b/MAINTAINERS
index 98eddf7ae1..a906218f9d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4127,6 +4137,7 @@ F: .travis.yml
F: docs/devel/ci*
F: scripts/ci/
F: tests/docker/
+F: tests/functional/
F: tests/vm/
F: tests/lcitool/
F: tests/avocado/tuxrun_baselines.py
---
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment
2024-07-30 17:03 ` [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment Daniel P. Berrangé
@ 2024-07-31 12:55 ` Philippe Mathieu-Daudé
2024-08-01 16:13 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-31 12:55 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> These simple tests can be converted to stand-alone tests quite easily,
> e.g. by just setting the machine to 'none' now manually or by adding
> "-cpu" command line parameters, since we don't support the corresponding
> avocado tags in the new python test framework.
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/meson.build | 6 ++
> .../test_info_usernet.py} | 11 ++-
> .../test_ppc_74xx.py} | 74 ++++++++-----------
> .../version.py => functional/test_version.py} | 13 ++--
> 4 files changed, 53 insertions(+), 51 deletions(-)
> rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
> mode change 100644 => 100755
> rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
> mode change 100644 => 100755
> rename tests/{avocado/version.py => functional/test_version.py} (78%)
> mode change 100644 => 100755
Please squash:
-- >8 --
diff --git a/MAINTAINERS b/MAINTAINERS
index 98eddf7ae1..a906218f9d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -317,6 +317,7 @@ F: configs/devices/ppc*
F: docs/system/ppc/embedded.rst
F: docs/system/target-ppc.rst
F: tests/tcg/ppc*/*
+F: tests/functional/test_ppc_74xx.py
RISC-V TCG CPUs
M: Palmer Dabbelt <palmer@dabbelt.com>
@@ -2101,6 +2108,7 @@ S: Odd Fixes
F: hw/net/
F: include/hw/net/
F: tests/qtest/virtio-net-test.c
+F: tests/avocado/info_usernet.py
F: docs/virtio-net-failover.rst
T: git https://github.com/jasowang/qemu.git net
@@ -2969,6 +2978,7 @@ S: Supported
F: include/qemu/option.h
F: tests/unit/test-keyval.c
F: tests/unit/test-qemu-opts.c
+F: tests/functional/test_version.py
F: util/keyval.c
F: util/qemu-option.c
diff --git a/tests/functional/test_info_usernet.py
b/tests/functional/test_info_usernet.py
index ded973253a..cd37524d94 100755
--- a/tests/functional/test_info_usernet.py
+++ b/tests/functional/test_info_usernet.py
@@ -19,7 +19,7 @@ class InfoUsernet(QemuSystemTest):
def test_hostfwd(self):
self.require_netdev('user')
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-netdev',
'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
diff --git a/tests/functional/test_version.py
b/tests/functional/test_version.py
index 5e566d76b1..3ab3b67f7e 100755
--- a/tests/functional/test_version.py
+++ b/tests/functional/test_version.py
@@ -17,7 +17,7 @@
class Version(QemuSystemTest):
def test_qmp_human_info_version(self):
- self.machine = 'none'
+ self.set_machine('none')
self.vm.add_args('-nodefaults')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
---
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
2024-07-30 17:03 ` [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Daniel P. Berrangé
@ 2024-07-31 12:56 ` Philippe Mathieu-Daudé
2024-07-31 23:00 ` Richard Henderson
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-31 12:56 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth,
Alistair Francis
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The avocado test defined test functions for both, riscv32 and riscv64.
> Since we can run the whole file with multiple targets in the new
> framework, we can now consolidate the functions so we have to only
> define one function per machine now.
>
> Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/riscv_opensbi.py | 63 --------------------------
> tests/functional/meson.build | 8 ++++
> tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++
> 3 files changed, 44 insertions(+), 63 deletions(-)
> delete mode 100644 tests/avocado/riscv_opensbi.py
> create mode 100755 tests/functional/test_riscv_opensbi.py
Please squash:
-- >8 --
diff --git a/MAINTAINERS b/MAINTAINERS
index 98eddf7ae1..a906218f9d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -336,6 +337,7 @@ F: include/hw/riscv/
F: linux-user/host/riscv32/
F: linux-user/host/riscv64/
F: tests/tcg/riscv64/
+F: tests/functional/test_riscv_opensbi.py
RISC-V XThead* extensions
M: Christoph Muellner <christoph.muellner@vrull.eu>
---
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
2024-07-30 17:03 ` [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests Daniel P. Berrangé
2024-07-31 12:53 ` Philippe Mathieu-Daudé
@ 2024-07-31 14:24 ` Alex Bennée
2024-08-01 10:07 ` Daniel P. Berrangé
2024-08-01 16:11 ` Philippe Mathieu-Daudé
2 siblings, 1 reply; 69+ messages in thread
From: Alex Bennée @ 2024-07-31 14:24 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini,
Philippe Mathieu-Daudé, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Thomas Huth
Daniel P. Berrangé <berrange@redhat.com> writes:
> From: Thomas Huth <thuth@redhat.com>
>
> The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py
> file with some adjustments to get rid of the Avocado dependencies (i.e.
> we also have to drop the LinuxSSHMixIn and LinuxTest for now).
>
> The emulator binary and build directory are now passed via
> environment variables that will be set via meson.build later.
<snip>
> diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
> new file mode 100644
> index 0000000000..82cc1d454f
> --- /dev/null
> +++ b/tests/functional/qemu_test/testcase.py
> @@ -0,0 +1,154 @@
> +# Test class and utilities for functional tests
> +#
> +# Copyright 2018, 2024 Red Hat, Inc.
> +#
> +# Original Author (Avocado-based tests):
> +# Cleber Rosa <crosa@redhat.com>
> +#
> +# Adaption for standalone version:
> +# Thomas Huth <thuth@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +import logging
> +import os
> +import pycotap
> +import sys
> +import unittest
> +import uuid
> +
> +from qemu.machine import QEMUMachine
> +from qemu.utils import kvm_available, tcg_available
> +
> +from .cmd import run_cmd
> +from .config import BUILD_DIR
> +
> +
> +class QemuBaseTest(unittest.TestCase):
> +
> + qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
> + arch = None
> +
> + workdir = None
> + log = logging.getLogger('qemu-test')
> +
> + def setUp(self, bin_prefix):
> + self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
> + self.arch = self.qemu_bin.split('-')[-1]
> +
> + self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch,
> + self.id())
> + if not os.path.exists(self.workdir):
> + os.makedirs(self.workdir)
This is racy under --repeat:
==================================== 1/4 =====================================
test: qemu:func-quick+func-riscv64 / func-riscv64-riscv_opensbi
start time: 14:16:52
duration: 0.06s
result: exit status 1
command: PYTHONPATH=/home/alex/lsrc/qemu.git/python:/home/alex/lsrc/qemu.git/tests/functional QEMU_BUILD_ROOT=/home/alex/lsrc/qemu.git/builds/all QEMU_TEST_QEMU_BINARY=
/home/alex/lsrc/qemu.git/builds/all/qemu-system-riscv64 MALLOC_PERTURB_=71 QEMU_TEST_QEMU_IMG=/home/alex/lsrc/qemu.git/builds/all/qemu-img /home/alex/lsrc/qemu.git/builds/al
l/pyvenv/bin/python3 /home/alex/lsrc/qemu.git/tests/functional/test_riscv_opensbi.py
----------------------------------- stdout -----------------------------------
TAP version 13
not ok 1 test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u
not ok 2 test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike
not ok 3 test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt
1..3
----------------------------------- stderr -----------------------------------
Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u'
Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike'
Traceback (most recent call last):
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
super().setUp('qemu-system-')
File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
os.makedirs(self.workdir)
File "<frozen os>", line 225, in makedirs
FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt'
(test program exited with status code 1)
We could just:
os.makedirs(self.workdir, exist_ok = True)
<snip>
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
2024-07-31 12:56 ` Philippe Mathieu-Daudé
@ 2024-07-31 23:00 ` Richard Henderson
2024-08-01 10:09 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Richard Henderson @ 2024-07-31 23:00 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc, Ani Sinha,
Alex Bennée, Thomas Huth, Alistair Francis
On 7/31/24 22:56, Philippe Mathieu-Daudé wrote:
> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>> From: Thomas Huth <thuth@redhat.com>
>>
>> The avocado test defined test functions for both, riscv32 and riscv64.
>> Since we can run the whole file with multiple targets in the new
>> framework, we can now consolidate the functions so we have to only
>> define one function per machine now.
>>
>> Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/avocado/riscv_opensbi.py | 63 --------------------------
>> tests/functional/meson.build | 8 ++++
>> tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++
>> 3 files changed, 44 insertions(+), 63 deletions(-)
>> delete mode 100644 tests/avocado/riscv_opensbi.py
>> create mode 100755 tests/functional/test_riscv_opensbi.py
>
> Please squash:
>
> -- >8 --
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 98eddf7ae1..a906218f9d 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -336,6 +337,7 @@ F: include/hw/riscv/
> F: linux-user/host/riscv32/
> F: linux-user/host/riscv64/
> F: tests/tcg/riscv64/
> +F: tests/functional/test_riscv_opensbi.py
It seems worth taking this opportunity to add per-target subdirs of tests/functional/, so
that we don't have to list every individual test in MAINTAINERS.
Surely one
F: tests/functional/riscv/
etc, is actually maintainable...
r~
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text
2024-07-30 17:03 ` [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text Daniel P. Berrangé
2024-07-31 12:49 ` Philippe Mathieu-Daudé
@ 2024-08-01 10:00 ` Alex Bennée
1 sibling, 0 replies; 69+ messages in thread
From: Alex Bennée @ 2024-08-01 10:00 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini,
Philippe Mathieu-Daudé, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Thomas Huth
Daniel P. Berrangé <berrange@redhat.com> writes:
> From: Thomas Huth <thuth@redhat.com>
>
> The next patch is going to add some entries that need more space between
> the command and the help text, so let's increase the indentation here
> first.
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
2024-07-31 14:24 ` Alex Bennée
@ 2024-08-01 10:07 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-01 10:07 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini,
Philippe Mathieu-Daudé, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Thomas Huth
On Wed, Jul 31, 2024 at 03:24:35PM +0100, Alex Bennée wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
>
> > From: Thomas Huth <thuth@redhat.com>
> >
> > The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py
> > file with some adjustments to get rid of the Avocado dependencies (i.e.
> > we also have to drop the LinuxSSHMixIn and LinuxTest for now).
> >
> > The emulator binary and build directory are now passed via
> > environment variables that will be set via meson.build later.
> <snip>
> > diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
> > new file mode 100644
> > index 0000000000..82cc1d454f
> > --- /dev/null
> > +++ b/tests/functional/qemu_test/testcase.py
> > @@ -0,0 +1,154 @@
> > +# Test class and utilities for functional tests
> > +#
> > +# Copyright 2018, 2024 Red Hat, Inc.
> > +#
> > +# Original Author (Avocado-based tests):
> > +# Cleber Rosa <crosa@redhat.com>
> > +#
> > +# Adaption for standalone version:
> > +# Thomas Huth <thuth@redhat.com>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > +# later. See the COPYING file in the top-level directory.
> > +
> > +import logging
> > +import os
> > +import pycotap
> > +import sys
> > +import unittest
> > +import uuid
> > +
> > +from qemu.machine import QEMUMachine
> > +from qemu.utils import kvm_available, tcg_available
> > +
> > +from .cmd import run_cmd
> > +from .config import BUILD_DIR
> > +
> > +
> > +class QemuBaseTest(unittest.TestCase):
> > +
> > + qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
> > + arch = None
> > +
> > + workdir = None
> > + log = logging.getLogger('qemu-test')
> > +
> > + def setUp(self, bin_prefix):
> > + self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
> > + self.arch = self.qemu_bin.split('-')[-1]
> > +
> > + self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch,
> > + self.id())
> > + if not os.path.exists(self.workdir):
> > + os.makedirs(self.workdir)
>
> This is racy under --repeat:
>
> ==================================== 1/4 =====================================
> test: qemu:func-quick+func-riscv64 / func-riscv64-riscv_opensbi
> start time: 14:16:52
> duration: 0.06s
> result: exit status 1
> command: PYTHONPATH=/home/alex/lsrc/qemu.git/python:/home/alex/lsrc/qemu.git/tests/functional QEMU_BUILD_ROOT=/home/alex/lsrc/qemu.git/builds/all QEMU_TEST_QEMU_BINARY=
> /home/alex/lsrc/qemu.git/builds/all/qemu-system-riscv64 MALLOC_PERTURB_=71 QEMU_TEST_QEMU_IMG=/home/alex/lsrc/qemu.git/builds/all/qemu-img /home/alex/lsrc/qemu.git/builds/al
> l/pyvenv/bin/python3 /home/alex/lsrc/qemu.git/tests/functional/test_riscv_opensbi.py
> ----------------------------------- stdout -----------------------------------
> TAP version 13
> not ok 1 test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u
> not ok 2 test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike
> not ok 3 test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt
> 1..3
> ----------------------------------- stderr -----------------------------------
> Traceback (most recent call last):
> File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
> super().setUp('qemu-system-')
> File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
> os.makedirs(self.workdir)
> File "<frozen os>", line 225, in makedirs
> FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u'
>
> Traceback (most recent call last):
> File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
> super().setUp('qemu-system-')
> File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
> os.makedirs(self.workdir)
> File "<frozen os>", line 225, in makedirs
> FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike'
>
> Traceback (most recent call last):
> File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 85, in setUp
> super().setUp('qemu-system-')
> File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line 45, in setUp
> os.makedirs(self.workdir)
> File "<frozen os>", line 225, in makedirs
> FileExistsError: [Errno 17] File exists: '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt'
>
>
> (test program exited with status code 1)
>
> We could just:
>
> os.makedirs(self.workdir, exist_ok = True)
Yeah, that's a better approach.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
2024-07-31 23:00 ` Richard Henderson
@ 2024-08-01 10:09 ` Daniel P. Berrangé
2024-08-01 15:11 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-01 10:09 UTC (permalink / raw)
To: Richard Henderson
Cc: Philippe Mathieu-Daudé, qemu-devel, Fabiano Rosas,
Paolo Bonzini, John Snow, qemu-ppc, Ani Sinha, Alex Bennée,
Thomas Huth, Alistair Francis
On Thu, Aug 01, 2024 at 09:00:36AM +1000, Richard Henderson wrote:
> On 7/31/24 22:56, Philippe Mathieu-Daudé wrote:
> > On 30/7/24 19:03, Daniel P. Berrangé wrote:
> > > From: Thomas Huth <thuth@redhat.com>
> > >
> > > The avocado test defined test functions for both, riscv32 and riscv64.
> > > Since we can run the whole file with multiple targets in the new
> > > framework, we can now consolidate the functions so we have to only
> > > define one function per machine now.
> > >
> > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > ---
> > > tests/avocado/riscv_opensbi.py | 63 --------------------------
> > > tests/functional/meson.build | 8 ++++
> > > tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++
> > > 3 files changed, 44 insertions(+), 63 deletions(-)
> > > delete mode 100644 tests/avocado/riscv_opensbi.py
> > > create mode 100755 tests/functional/test_riscv_opensbi.py
> >
> > Please squash:
> >
> > -- >8 --
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 98eddf7ae1..a906218f9d 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -336,6 +337,7 @@ F: include/hw/riscv/
> > F: linux-user/host/riscv32/
> > F: linux-user/host/riscv64/
> > F: tests/tcg/riscv64/
> > +F: tests/functional/test_riscv_opensbi.py
>
> It seems worth taking this opportunity to add per-target subdirs of
> tests/functional/, so that we don't have to list every individual test in
> MAINTAINERS.
>
> Surely one
>
> F: tests/functional/riscv/
>
> etc, is actually maintainable...
Either that or wildcard the filenames
F: tests/functional/test_riscv_*
but subdirs matching the target is probably a little nicer to really
encourage people to follow conventions more strictly. We would also
need a "tests/functional/common/" for tests which aren't target specific
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 03/24] tests/functional: Set up logging
2024-07-30 17:03 ` [PATCH v3 03/24] tests/functional: Set up logging Daniel P. Berrangé
@ 2024-08-01 10:10 ` Alex Bennée
2024-08-13 9:34 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Alex Bennée @ 2024-08-01 10:10 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini,
Philippe Mathieu-Daudé, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Thomas Huth
Daniel P. Berrangé <berrange@redhat.com> writes:
> From: Thomas Huth <thuth@redhat.com>
>
> Create log files for each test separately, one file that contains
> the basic logging and one that contains the console output.
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/qemu_test/testcase.py | 27 +++++++++++++++++++++++++-
> 1 file changed, 26 insertions(+), 1 deletion(-)
>
> diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
> index 82cc1d454f..27bbf4a0af 100644
> --- a/tests/functional/qemu_test/testcase.py
> +++ b/tests/functional/qemu_test/testcase.py
> @@ -31,7 +31,8 @@ class QemuBaseTest(unittest.TestCase):
> arch = None
>
> workdir = None
> - log = logging.getLogger('qemu-test')
> + log = None
> + logdir = None
>
> def setUp(self, bin_prefix):
> self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
> @@ -42,6 +43,20 @@ def setUp(self, bin_prefix):
> if not os.path.exists(self.workdir):
> os.makedirs(self.workdir)
>
> + self.logdir = self.workdir
> + self.log = logging.getLogger('qemu-test')
> + self.log.setLevel(logging.DEBUG)
> + self._log_fh = logging.FileHandler(os.path.join(self.logdir,
> + 'base.log'), mode='w')
> + self._log_fh.setLevel(logging.DEBUG)
> + fileFormatter = logging.Formatter(
> + '%(asctime)s - %(levelname)s: %(message)s')
> + self._log_fh.setFormatter(fileFormatter)
> + self.log.addHandler(self._log_fh)
This is in the wrong place, we have a unique machine directory for each
run, the console.log should go in there.
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
2024-08-01 10:09 ` Daniel P. Berrangé
@ 2024-08-01 15:11 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 15:11 UTC (permalink / raw)
To: Daniel P. Berrangé, Richard Henderson
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Ani Sinha, Alex Bennée, Thomas Huth, Alistair Francis
On 1/8/24 12:09, Daniel P. Berrangé wrote:
> On Thu, Aug 01, 2024 at 09:00:36AM +1000, Richard Henderson wrote:
>> On 7/31/24 22:56, Philippe Mathieu-Daudé wrote:
>>> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>>>> From: Thomas Huth <thuth@redhat.com>
>>>>
>>>> The avocado test defined test functions for both, riscv32 and riscv64.
>>>> Since we can run the whole file with multiple targets in the new
>>>> framework, we can now consolidate the functions so we have to only
>>>> define one function per machine now.
>>>>
>>>> Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>> tests/avocado/riscv_opensbi.py | 63 --------------------------
>>>> tests/functional/meson.build | 8 ++++
>>>> tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++
>>>> 3 files changed, 44 insertions(+), 63 deletions(-)
>>>> delete mode 100644 tests/avocado/riscv_opensbi.py
>>>> create mode 100755 tests/functional/test_riscv_opensbi.py
>>>
>>> Please squash:
>>>
>>> -- >8 --
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 98eddf7ae1..a906218f9d 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -336,6 +337,7 @@ F: include/hw/riscv/
>>> F: linux-user/host/riscv32/
>>> F: linux-user/host/riscv64/
>>> F: tests/tcg/riscv64/
>>> +F: tests/functional/test_riscv_opensbi.py
>>
>> It seems worth taking this opportunity to add per-target subdirs of
>> tests/functional/, so that we don't have to list every individual test in
>> MAINTAINERS.
>>
>> Surely one
>>
>> F: tests/functional/riscv/
>>
>> etc, is actually maintainable...
>
> Either that or wildcard the filenames
>
> F: tests/functional/test_riscv_*
>
>
> but subdirs matching the target is probably a little nicer to really
> encourage people to follow conventions more strictly. We would also
> need a "tests/functional/common/" for tests which aren't target specific
Hmm I'm a bit reluctant to both suggestions. Various files are
added to cover a feature which ends being tested on various archs.
Once I wanted to use Avocado tags like GitLab labels, and keep a
file where developers subscribe to tags; doing the parsing and
label to developer conversion in Python (or Rust), and having a
./get_maintainer.sh calling both the Perl one and anything we can
add in Python/Rust.
That said, wildcard pattern is a simpler start.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets
2024-07-30 17:03 ` [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets Daniel P. Berrangé
2024-07-31 5:01 ` Richard Henderson
@ 2024-08-01 16:01 ` Philippe Mathieu-Daudé
1 sibling, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:01 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> Many tests need to access assets stored on remote sites. We don't want
> to download these during test execution when run by meson, since this
> risks hitting test timeouts when data transfers are slow.
>
> Add support for pre-emptive caching of assets by setting the env var
> QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
> instead of running the test, the assets will be downloaded and saved
> to the cache, then the timestamp file created.
>
> A meson custom target is created as a dependency of each test suite
> to trigger the pre-emptive caching logic before the test runs.
>
> When run in caching mode, it will locate assets by looking for class
> level variables with a name prefix "ASSET_", and type "Asset".
>
> At the ninja level
>
> ninja test --suite functional
>
> will speculatively download any assets that are not already cached,
> so it is advisable to set a timeout multiplier.
>
> QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional
>
> will fail the test if a required asset is not already cached
>
> nina check-func-precache
>
> will download and cache all assets required by the functional
> tests
>
> At the make level, precaching is always done by
>
> make check-functional
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
> tests/Makefile.include | 3 ++-
> tests/functional/meson.build | 33 +++++++++++++++++++++++--
> tests/functional/qemu_test/asset.py | 34 ++++++++++++++++++++++++++
> tests/functional/qemu_test/testcase.py | 8 ++++++
> 4 files changed, 75 insertions(+), 3 deletions(-)
> diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
> index 27bbf4a0af..21dbd90613 100644
> --- a/tests/functional/qemu_test/testcase.py
> +++ b/tests/functional/qemu_test/testcase.py
> @@ -21,6 +21,7 @@
> from qemu.machine import QEMUMachine
> from qemu.utils import kvm_available, tcg_available
>
> +from .asset import Asset
> from .cmd import run_cmd
> from .config import BUILD_DIR
>
> @@ -58,6 +59,13 @@ def tearDown(self):
> self.log.removeHandler(self._log_fh)
>
> def main():
> + path = os.path.basename(sys.argv[0])[:-3]
This line should be added here in patch #2 in order to
avoid ...
> +
> + cache = os.environ.get("QEMU_TEST_PRECACHE", None)
> + if cache is not None:
> + Asset.precache_suites(path, cache)
> + return
> +
> tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
> test_output_log = pycotap.LogMode.LogToError)
> path = os.path.basename(sys.argv[0])[:-3]
.... this duplication.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests
2024-07-30 17:03 ` [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests Daniel P. Berrangé
2024-07-31 12:53 ` Philippe Mathieu-Daudé
2024-07-31 14:24 ` Alex Bennée
@ 2024-08-01 16:11 ` Philippe Mathieu-Daudé
2 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:11 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py
> file with some adjustments to get rid of the Avocado dependencies (i.e.
> we also have to drop the LinuxSSHMixIn and LinuxTest for now).
>
> The emulator binary and build directory are now passed via
> environment variables that will be set via meson.build later.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> [DB: split __init__.py into multiple files]
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
> tests/functional/qemu_test/__init__.py | 13 ++
> tests/functional/qemu_test/cmd.py | 171 +++++++++++++++++++++++++
> tests/functional/qemu_test/config.py | 36 ++++++
> tests/functional/qemu_test/testcase.py | 154 ++++++++++++++++++++++
> 4 files changed, 374 insertions(+)
> create mode 100644 tests/functional/qemu_test/__init__.py
> create mode 100644 tests/functional/qemu_test/cmd.py
> create mode 100644 tests/functional/qemu_test/config.py
> create mode 100644 tests/functional/qemu_test/testcase.py
> diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
> new file mode 100644
> index 0000000000..82cc1d454f
> --- /dev/null
> +++ b/tests/functional/qemu_test/testcase.py
> @@ -0,0 +1,154 @@
> +# Test class and utilities for functional tests
> +#
> +# Copyright 2018, 2024 Red Hat, Inc.
> +#
> +# Original Author (Avocado-based tests):
> +# Cleber Rosa <crosa@redhat.com>
> +#
> +# Adaption for standalone version:
> +# Thomas Huth <thuth@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +import logging
> +import os
> +import pycotap
> +import sys
> +import unittest
> +import uuid
> +
> +from qemu.machine import QEMUMachine
> +from qemu.utils import kvm_available, tcg_available
> +
> +from .cmd import run_cmd
> +from .config import BUILD_DIR
> +
> +
> +class QemuBaseTest(unittest.TestCase):
> +
> + qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
> + arch = None
> +
> + workdir = None
> + log = logging.getLogger('qemu-test')
> +
> + def setUp(self, bin_prefix):
> + self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
> + self.arch = self.qemu_bin.split('-')[-1]
> +
> + self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch,
> + self.id())
> + if not os.path.exists(self.workdir):
> + os.makedirs(self.workdir)
> +
> + def main():
[*]
> + tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
> + test_output_log = pycotap.LogMode.LogToError)
> + path = os.path.basename(sys.argv[0])[:-3]
Moving the 'path' line in [*] simplifies patch #9 (no duplication).
> + unittest.main(module = None, testRunner = tr, argv=["__dummy__", path])
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests
2024-07-30 17:03 ` [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests Daniel P. Berrangé
@ 2024-08-01 16:12 ` Philippe Mathieu-Daudé
2024-08-13 12:00 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:12 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel, Paolo Bonzini
Cc: Fabiano Rosas, John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
Hi,
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Provide a meson.build file for the upcoming python-based functional
> tests, and add some wrapper glue targets to the tests/Makefile.include
> file. We are going to use two "speed" modes for the functional tests:
> The "quick" tests can be run at any time (i.e. also during "make check"),
> while the "thorough" tests should only be run when running a
> "make check-functional" test run (since these tests might download
> additional assets from the internet).
>
> The changes to the meson.build files are partly based on an earlier
> patch by Ani Sinha.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/Makefile.include | 11 ++++++
> tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
> tests/meson.build | 1 +
> 3 files changed, 78 insertions(+)
> create mode 100644 tests/functional/meson.build
> +foreach speed : ['quick', 'thorough']
> + foreach dir : target_dirs
> + if not dir.endswith('-softmmu')
> + continue
> + endif
> +
> + target_base = dir.split('-')[0]
> + test_emulator = emulators['qemu-system-' + target_base]
> +
> + if speed == 'quick'
> + suites = ['func-quick', 'func-' + target_base]
> + target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic
> + else
> + suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
> + target_tests = get_variable('tests_' + target_base + '_' + speed, [])
> + endif
> +
> + test_deps = roms
> + test_env = environment()
> + if have_tools
> + test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img')
> + test_deps += [qemu_img]
> + endif
> + test_env.set('QEMU_TEST_QEMU_BINARY',
> + meson.global_build_root() / 'qemu-system-' + target_base)
> + test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())
> + test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
> + meson.current_source_dir())
> +
> + foreach test : target_tests
> + test('func-@0@/@1@'.format(target_base, test),
> + python,
> + depends: [test_deps, test_emulator, emulator_modules],
> + env: test_env,
> + args: [meson.current_source_dir() / 'test_' + test + '.py'],
> + protocol: 'tap',
> + timeout: test_timeouts.get(test, 60),
> + priority: test_timeouts.get(test, 60),
> + suite: suites)
> + endforeach
> + endforeach
> +endforeach
Can we have meson report why a test is skipped? All Python
unittest @skip* decorators provide a 'reason' argument, so
we already have that information.
Not much help on meson doc:
https://mesonbuild.com/Unit-tests.html#skipped-tests-and-hard-errors
If I run a test manually I see the reason in the tap output:
$ FOO=bar tests/functional/test_mips64el_loongson3v.py
TAP version 13
ok 1 test_mips64el_loongson3v.MipsLoongson3v.test_pmon_serial_console #
SKIP untrusted code
1..1
$
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment
2024-07-31 12:55 ` Philippe Mathieu-Daudé
@ 2024-08-01 16:13 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:13 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 31/7/24 14:55, Philippe Mathieu-Daudé wrote:
> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>> From: Thomas Huth <thuth@redhat.com>
>>
>> These simple tests can be converted to stand-alone tests quite easily,
>> e.g. by just setting the machine to 'none' now manually or by adding
>> "-cpu" command line parameters, since we don't support the corresponding
>> avocado tags in the new python test framework.
>>
>> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/functional/meson.build | 6 ++
>> .../test_info_usernet.py} | 11 ++-
>> .../test_ppc_74xx.py} | 74 ++++++++-----------
>> .../version.py => functional/test_version.py} | 13 ++--
>> 4 files changed, 53 insertions(+), 51 deletions(-)
>> rename tests/{avocado/info_usernet.py =>
>> functional/test_info_usernet.py} (87%)
>> mode change 100644 => 100755
>> rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
>> mode change 100644 => 100755
>> rename tests/{avocado/version.py => functional/test_version.py} (78%)
>> mode change 100644 => 100755
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
2024-07-30 17:03 ` [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching Daniel P. Berrangé
@ 2024-08-01 16:20 ` Philippe Mathieu-Daudé
2024-08-01 17:00 ` Daniel P. Berrangé
2024-08-01 17:02 ` Daniel P. Berrangé
2024-08-01 21:51 ` Richard Henderson
1 sibling, 2 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:20 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> The 'Asset' class is a simple module that declares a downloadable
> asset that can be cached locally. Downloads are stored in the user's
> home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
> tests/functional/qemu_test/__init__.py | 1 +
> tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++
> 2 files changed, 97 insertions(+)
> create mode 100644 tests/functional/qemu_test/asset.py
>
> diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
> index 2f1e0bc70d..db05c8f412 100644
> --- a/tests/functional/qemu_test/__init__.py
> +++ b/tests/functional/qemu_test/__init__.py
> @@ -6,6 +6,7 @@
> # later. See the COPYING file in the top-level directory.
>
>
> +from .asset import Asset
For next patch?
> from .config import BUILD_DIR
> from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
> interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
> diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> new file mode 100644
> index 0000000000..6432da2e0b
> --- /dev/null
> +++ b/tests/functional/qemu_test/asset.py
> @@ -0,0 +1,96 @@
> +# Test utilities for fetching & caching assets
> +#
> +# Copyright 2024 Red Hat, Inc.
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +
> +import hashlib
> +import logging
> +import os
> +import subprocess
> +import urllib.request
> +from pathlib import Path
> +from shutil import copyfileobj
> +
> +
> +# Instances of this class must be declared as class level variables
> +# starting with a name "ASSET_". This enables the pre-caching logic
> +# to easily find all referenced assets and download them prior to
> +# execution of the tests.
> +class Asset:
> +
> + def __init__(self, url, hash):
> + self.url = url
> + self.hash = hash
> + self.cache_dir = Path(Path("~").expanduser(),
> + ".cache", "qemu", "download")
> + self.cache_file = Path(self.cache_dir,
> + hashlib.sha256(url.encode("utf-8")).hexdigest())
> + self.log = logging.getLogger('qemu-test')
> +
> + def __repr__(self):
> + return "Asset: url=%s hash=%s cache=%s" % (
> + self.url, self.hash, self.cache_file)
> +
> + def _check(self, cache_file):
> + if self.hash is None:
> + return True
> + if len(self.hash) == 40:
> + sum_prog = 'sha1sum'
> + elif len(self.hash) == 64:
> + sum_prog = 'sha256sum'
Do we want to support these? Should we declare them deprecated
and emit a warning?
> + elif len(self.hash) == 128:
> + sum_prog = 'sha512sum'
> + else:
> + raise Exception("unknown hash type")
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Very nice :)
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 11/24] tests/functional: Add a function for extracting files from an archive
2024-07-30 17:03 ` [PATCH v3 11/24] tests/functional: Add a function for extracting files from an archive Daniel P. Berrangé
@ 2024-08-01 16:22 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:22 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Some Avocado-based tests use the "archive" module from avocado.utils
> to extract files from an archive. To be able to use these tests
> without Avocado, we have to provide our own function for extracting
> files. Fortunately, there is already the tarfile module that will
> provide us with this functionality, so let's just add a nice wrapper
> function around that.
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/qemu_test/utils.py | 21 +++++++++++++++++++++
> 1 file changed, 21 insertions(+)
> create mode 100644 tests/functional/qemu_test/utils.py
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 12/24] tests/functional: Convert some avocado tests that needed avocado.utils.archive
2024-07-30 17:03 ` [PATCH v3 12/24] tests/functional: Convert some avocado tests that needed avocado.utils.archive Daniel P. Berrangé
@ 2024-08-01 16:27 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:27 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Instead of using the "archive" module from avocado.utils, switch
> these tests to use the new wrapper function that is based on the
> "tarfile" module instead.
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 6 +--
> tests/avocado/machine_sparc64_sun4u.py | 36 -----------------
> tests/functional/meson.build | 6 +++
> .../test_arm_canona1100.py} | 30 ++++++++------
> .../test_ppc_bamboo.py} | 32 +++++++--------
> tests/functional/test_sparc64_sun4u.py | 40 +++++++++++++++++++
> 6 files changed, 82 insertions(+), 68 deletions(-)
> delete mode 100644 tests/avocado/machine_sparc64_sun4u.py
> rename tests/{avocado/machine_arm_canona1100.py => functional/test_arm_canona1100.py} (52%)
> mode change 100644 => 100755
> rename tests/{avocado/ppc_bamboo.py => functional/test_ppc_bamboo.py} (60%)
> mode change 100644 => 100755
> create mode 100755 tests/functional/test_sparc64_sun4u.py
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 21/24] tests/functional: Convert the m68k nextcube test with tesseract
2024-07-30 17:03 ` [PATCH v3 21/24] tests/functional: Convert the m68k nextcube test with tesseract Daniel P. Berrangé
@ 2024-08-01 16:33 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:33 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The code that handles running of tesseract needs to be tweaked a little
> bit to be able to run without the functions from avocado.utils, and
> while we're at it, drop some legacy stuff that was still there due to
> Tesseract 3 support that we already dropped a while ago.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/tesseract_utils.py | 46 -------------------
> tests/functional/meson.build | 4 ++
> tests/functional/qemu_test/tesseract.py | 35 ++++++++++++++
> .../test_m68k_nextcube.py} | 29 ++++++------
> 4 files changed, 55 insertions(+), 59 deletions(-)
> delete mode 100644 tests/avocado/tesseract_utils.py
> create mode 100644 tests/functional/qemu_test/tesseract.py
> rename tests/{avocado/machine_m68k_nextcube.py => functional/test_m68k_nextcube.py} (76%)
> mode change 100644 => 100755
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 23/24] tests/functional: Convert the rx_gdbsim avocado test into a standalone test
2024-07-30 17:03 ` [PATCH v3 23/24] tests/functional: Convert the rx_gdbsim avocado " Daniel P. Berrangé
@ 2024-08-01 16:40 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:40 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Provide a "gzip_uncompress" function based on the standard "gzip" module
> to avoid the usage of avocado.utils here.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/meson.build | 4 ++
> tests/functional/qemu_test/utils.py | 12 ++++
> .../test_rx_gdbsim.py} | 57 ++++++++++---------
> 3 files changed, 45 insertions(+), 28 deletions(-)
> rename tests/{avocado/machine_rx_gdbsim.py => functional/test_rx_gdbsim.py} (54%)
> mode change 100644 => 100755
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 24/24] gitlab-ci: Add "check-functional" to the build tests
2024-07-30 17:03 ` [PATCH v3 24/24] gitlab-ci: Add "check-functional" to the build tests Daniel P. Berrangé
@ 2024-08-01 16:40 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:40 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Now that we converted many tests from the "check-avocado" test suite
> to the "check-functional" test suite, we should make sure that these
> also get tested in the CI.
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> .gitlab-ci.d/buildtest-template.yml | 3 +-
> .gitlab-ci.d/buildtest.yml | 60 ++++++++++++++---------------
> 2 files changed, 32 insertions(+), 31 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 14/24] tests/functional: Convert the x86_cpu_model_versions test
2024-07-30 17:03 ` [PATCH v3 14/24] tests/functional: Convert the x86_cpu_model_versions test Daniel P. Berrangé
@ 2024-08-01 16:43 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 16:43 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Nothing thrilling in here, it's just a straight forward conversion.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/meson.build | 1 +
> .../test_x86_cpu_model_versions.py} | 63 ++++++-------------
> 2 files changed, 20 insertions(+), 44 deletions(-)
> rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
> mode change 100644 => 100755
> @@ -217,9 +216,8 @@ def test_4_1_alias(self):
> def test_none_alias(self):
> """
> Check if unversioned CPU model is an alias pointing to some version
> -
> - :avocado: tags=machine:none
> """
> + self.machine = 'none'
self.set_machine('none')
> self.vm.add_args('-S')
> self.vm.launch()
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
2024-08-01 16:20 ` Philippe Mathieu-Daudé
@ 2024-08-01 17:00 ` Daniel P. Berrangé
2024-08-01 17:02 ` Daniel P. Berrangé
1 sibling, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-01 17:00 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On Thu, Aug 01, 2024 at 06:20:58PM +0200, Philippe Mathieu-Daudé wrote:
> On 30/7/24 19:03, Daniel P. Berrangé wrote:
> > The 'Asset' class is a simple module that declares a downloadable
> > asset that can be cached locally. Downloads are stored in the user's
> > home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> > tests/functional/qemu_test/__init__.py | 1 +
> > tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++
> > 2 files changed, 97 insertions(+)
> > create mode 100644 tests/functional/qemu_test/asset.py
> >
> > diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
> > index 2f1e0bc70d..db05c8f412 100644
> > --- a/tests/functional/qemu_test/__init__.py
> > +++ b/tests/functional/qemu_test/__init__.py
> > @@ -6,6 +6,7 @@
> > # later. See the COPYING file in the top-level directory.
> > +from .asset import Asset
>
> For next patch?
>
> > from .config import BUILD_DIR
> > from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
> > interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
> > diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> > new file mode 100644
> > index 0000000000..6432da2e0b
> > --- /dev/null
> > +++ b/tests/functional/qemu_test/asset.py
> > @@ -0,0 +1,96 @@
> > +# Test utilities for fetching & caching assets
> > +#
> > +# Copyright 2024 Red Hat, Inc.
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > +# later. See the COPYING file in the top-level directory.
> > +
> > +import hashlib
> > +import logging
> > +import os
> > +import subprocess
> > +import urllib.request
> > +from pathlib import Path
> > +from shutil import copyfileobj
> > +
> > +
> > +# Instances of this class must be declared as class level variables
> > +# starting with a name "ASSET_". This enables the pre-caching logic
> > +# to easily find all referenced assets and download them prior to
> > +# execution of the tests.
> > +class Asset:
> > +
> > + def __init__(self, url, hash):
> > + self.url = url
> > + self.hash = hash
> > + self.cache_dir = Path(Path("~").expanduser(),
> > + ".cache", "qemu", "download")
> > + self.cache_file = Path(self.cache_dir,
> > + hashlib.sha256(url.encode("utf-8")).hexdigest())
> > + self.log = logging.getLogger('qemu-test')
> > +
> > + def __repr__(self):
> > + return "Asset: url=%s hash=%s cache=%s" % (
> > + self.url, self.hash, self.cache_file)
> > +
> > + def _check(self, cache_file):
> > + if self.hash is None:
> > + return True
> > + if len(self.hash) == 40:
> > + sum_prog = 'sha1sum'
> > + elif len(self.hash) == 64:
> > + sum_prog = 'sha256sum'
>
> Do we want to support these? Should we declare them deprecated
> and emit a warning?
Oh thanks for the reminder. I wanted to standardize on sha256 ,since
sha1 is broken and sha512 is overkill.
Since I've run the tests once I've got everything download and can
now trivially generate the sha256 for everything we need.
>
> > + elif len(self.hash) == 128:
> > + sum_prog = 'sha512sum'
> > + else:
> > + raise Exception("unknown hash type")
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>
> Very nice :)
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
2024-08-01 16:20 ` Philippe Mathieu-Daudé
2024-08-01 17:00 ` Daniel P. Berrangé
@ 2024-08-01 17:02 ` Daniel P. Berrangé
2024-08-02 13:24 ` Philippe Mathieu-Daudé
1 sibling, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-01 17:02 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On Thu, Aug 01, 2024 at 06:20:58PM +0200, Philippe Mathieu-Daudé wrote:
> On 30/7/24 19:03, Daniel P. Berrangé wrote:
> > The 'Asset' class is a simple module that declares a downloadable
> > asset that can be cached locally. Downloads are stored in the user's
> > home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> > tests/functional/qemu_test/__init__.py | 1 +
> > tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++
> > 2 files changed, 97 insertions(+)
> > create mode 100644 tests/functional/qemu_test/asset.py
> >
> > diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
> > index 2f1e0bc70d..db05c8f412 100644
> > --- a/tests/functional/qemu_test/__init__.py
> > +++ b/tests/functional/qemu_test/__init__.py
> > @@ -6,6 +6,7 @@
> > # later. See the COPYING file in the top-level directory.
> > +from .asset import Asset
>
> For next patch?
No, the idea is that a test class can just do 'from qemu_test import Asset',
so they don't need to be aware of the fact that we've split up the code into
separate files beneath the qemu_test/ directory.
>
> > from .config import BUILD_DIR
> > from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \
> > interrupt_interactive_console_until_pattern, wait_for_console_pattern, \
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
2024-07-30 17:03 ` [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests Daniel P. Berrangé
@ 2024-08-01 17:11 ` Philippe Mathieu-Daudé
2024-08-01 17:38 ` Daniel P. Berrangé
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-01 17:11 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel, qemu-s390x
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> These tests use archive.lzma_uncompress() from the Avocado utils,
> so provide a small helper function for this, based on the
> standard lzma module from Python instead.
>
> And while we're at it, replace the MD5 hashes in the topology test
> with proper SHA256 hashes, since MD5 should not be used anymore
> nowadays.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 4 +-
> tests/functional/meson.build | 6 ++
> tests/functional/qemu_test/utils.py | 14 +++
> .../test_s390x_ccw_virtio.py} | 79 ++++++++---------
> .../test_s390x_topology.py} | 86 ++++++++-----------
> 5 files changed, 94 insertions(+), 95 deletions(-)
> rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
> mode change 100644 => 100755
> rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
> mode change 100644 => 100755
Conversion LGTM, so:
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Now while testing I'm getting errors, but maybe pre-existing...
pyvenv/bin/meson test --no-rebuild -t 1 --setup thorough
--num-processes 1 --print-errorlogs --suite func-s390x --suite
func-s390x-thorough
qemu:func-thorough+func-s390x-thorough+thorough /
func-s390x-s390x_ccw_virtio time out (After 180.0 seconds)
1/5 qemu:func-thorough+func-s390x-thorough+thorough /
func-s390x-s390x_ccw_virtio TIMEOUT 180.05s killed by
signal 15 SIGTERM
▶ 2/5 test_empty_cpu_model.EmptyCPUModel.test
FAIL
2/5 qemu:func-quick+func-s390x / func-s390x-empty_cpu_model
ERROR 0.21s exit status 1
stderr:
Traceback (most recent call last):
File "/home/user/qemu/tests/functional/test_empty_cpu_model.py", line
21, in test
self.assertRegex(self.vm.get_log(), r'-cpu option cannot be empty')
AssertionError: Regex didn't match: '-cpu option cannot be empty' not
found in 'Could not access KVM kernel module: Permission
denied\nqemu-system-s390x: failed to initialize kvm: Permission denied\n'
(test program exited with status code 1)
▶ 4/5 test_version.Version.test_qmp_human_info_version
FAIL
Traceback (most recent call last):
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 425, in
_session_guard
await coro
File "/home/user/qemu/python/qemu/qmp/qmp_client.py", line 250, in
_establish_session
self._greeting = await self._get_greeting()
File "/home/user/qemu/python/qemu/qmp/qmp_client.py", line 270, in
_get_greeting
msg = await self._recv()
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 1009, in _recv
message = await self._do_recv()
File "/home/user/qemu/python/qemu/qmp/qmp_client.py", line 402, in
_do_recv
msg_bytes = await self._readline()
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 977, in
_readline
raise EOFError
EOFError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/qemu/python/qemu/machine/machine.py", line 448, in
launch
self._launch()
File "/home/user/qemu/python/qemu/machine/machine.py", line 487, in
_launch
self._post_launch()
File "/home/user/qemu/python/qemu/machine/machine.py", line 381, in
_post_launch
self._qmp.connect()
File "/home/user/qemu/python/qemu/qmp/legacy.py", line 153, in connect
self._sync(
File "/home/user/qemu/python/qemu/qmp/legacy.py", line 102, in _sync
return self._aloop.run_until_complete(
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in
run_until_complete
return future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 382, in connect
await self._session_guard(
File "/home/user/qemu/python/qemu/qmp/protocol.py", line 456, in
_session_guard
raise ConnectError(emsg, err) from err
qemu.qmp.protocol.ConnectError: Failed to establish session: EOFError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/qemu/tests/functional/test_version.py", line 22, in
test_qmp_human_info_version
self.vm.launch()
File "/home/user/qemu/python/qemu/machine/machine.py", line 461, in
launch
raise VMLaunchFailure(
qemu.machine.machine.VMLaunchFailure: ConnectError: Failed to establish
session: EOFError
Exit code: 1
Command: qemu-system-s390x -display none -vga none -chardev
socket,id=mon,fd=5 -mon chardev=mon,mode=control -machine none -nodefaults
Output: Could not access KVM kernel module: Permission denied
qemu-system-s390x: failed to initialize kvm: Permission denied
5/5 qemu:func-thorough+func-s390x-thorough+thorough /
func-s390x-s390x_topology SKIP 1.14s 0 subtests
passed
Summary of Failures:
1/5 qemu:func-thorough+func-s390x-thorough+thorough /
func-s390x-s390x_ccw_virtio TIMEOUT 180.05s killed by signal 15
SIGTERM
2/5 qemu:func-quick+func-s390x / func-s390x-empty_cpu_model
ERROR 0.21s exit status 1
4/5 qemu:func-quick+func-s390x / func-s390x-version
ERROR 0.25s exit status 1
I'm surprised by "Could not access KVM kernel module" because
kvm_available() checks for RW access:
def kvm_available(target_arch: Optional[str] = None,
qemu_bin: Optional[str] = None) -> bool:
if not os.access("/dev/kvm", os.R_OK | os.W_OK):
return False
Maybe missing require_accelerator() in tests?
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
2024-08-01 17:11 ` Philippe Mathieu-Daudé
@ 2024-08-01 17:38 ` Daniel P. Berrangé
2024-08-02 13:25 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-01 17:38 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, qemu-s390x, Fabiano Rosas, Paolo Bonzini, John Snow,
qemu-ppc, Richard Henderson, Ani Sinha, Alex Bennée,
Thomas Huth
On Thu, Aug 01, 2024 at 07:11:01PM +0200, Philippe Mathieu-Daudé wrote:
> On 30/7/24 19:03, Daniel P. Berrangé wrote:
> > From: Thomas Huth <thuth@redhat.com>
> >
> > These tests use archive.lzma_uncompress() from the Avocado utils,
> > so provide a small helper function for this, based on the
> > standard lzma module from Python instead.
> >
> > And while we're at it, replace the MD5 hashes in the topology test
> > with proper SHA256 hashes, since MD5 should not be used anymore
> > nowadays.
> >
> > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > ---
> > MAINTAINERS | 4 +-
> > tests/functional/meson.build | 6 ++
> > tests/functional/qemu_test/utils.py | 14 +++
> > .../test_s390x_ccw_virtio.py} | 79 ++++++++---------
> > .../test_s390x_topology.py} | 86 ++++++++-----------
> > 5 files changed, 94 insertions(+), 95 deletions(-)
> > rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
> > mode change 100644 => 100755
> > rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
> > mode change 100644 => 100755
>
> Conversion LGTM, so:
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>
> Now while testing I'm getting errors, but maybe pre-existing...
What host arch / OS are you running tests from ?
I got through the full set of testing on Fedora x86_64, so
wondering if something differs.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
2024-07-30 17:03 ` [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching Daniel P. Berrangé
2024-08-01 16:20 ` Philippe Mathieu-Daudé
@ 2024-08-01 21:51 ` Richard Henderson
2024-08-02 8:32 ` Daniel P. Berrangé
1 sibling, 1 reply; 69+ messages in thread
From: Richard Henderson @ 2024-08-01 21:51 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Ani Sinha, Alex Bennée, Thomas Huth
On 7/31/24 03:03, Daniel P. Berrangé wrote:
> + def __init__(self, url, hash):
> + self.url = url
> + self.hash = hash
> + self.cache_dir = Path(Path("~").expanduser(),
> + ".cache", "qemu", "download")
> + self.cache_file = Path(self.cache_dir,
> + hashlib.sha256(url.encode("utf-8")).hexdigest())
Out of curiosity, why are we hashing the url from which we downloaded?
Why not use the known hash of the contents?
If a url goes stale, and we adjust the url to a new host, we shouldn't need to re-download
just because the url changed.
r~
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
2024-08-01 21:51 ` Richard Henderson
@ 2024-08-02 8:32 ` Daniel P. Berrangé
0 siblings, 0 replies; 69+ messages in thread
From: Daniel P. Berrangé @ 2024-08-02 8:32 UTC (permalink / raw)
To: Richard Henderson
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini,
Philippe Mathieu-Daudé, John Snow, qemu-ppc, Ani Sinha,
Alex Bennée, Thomas Huth
On Fri, Aug 02, 2024 at 07:51:14AM +1000, Richard Henderson wrote:
> On 7/31/24 03:03, Daniel P. Berrangé wrote:
> > + def __init__(self, url, hash):
> > + self.url = url
> > + self.hash = hash
> > + self.cache_dir = Path(Path("~").expanduser(),
> > + ".cache", "qemu", "download")
> > + self.cache_file = Path(self.cache_dir,
> > + hashlib.sha256(url.encode("utf-8")).hexdigest())
>
> Out of curiosity, why are we hashing the url from which we downloaded?
> Why not use the known hash of the contents?
>
> If a url goes stale, and we adjust the url to a new host, we shouldn't need
> to re-download just because the url changed.
Hmmm, good point, will change it.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching
2024-08-01 17:02 ` Daniel P. Berrangé
@ 2024-08-02 13:24 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-02 13:24 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 1/8/24 19:02, Daniel P. Berrangé wrote:
> On Thu, Aug 01, 2024 at 06:20:58PM +0200, Philippe Mathieu-Daudé wrote:
>> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>>> The 'Asset' class is a simple module that declares a downloadable
>>> asset that can be cached locally. Downloads are stored in the user's
>>> home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.
>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>> tests/functional/qemu_test/__init__.py | 1 +
>>> tests/functional/qemu_test/asset.py | 96 ++++++++++++++++++++++++++
>>> 2 files changed, 97 insertions(+)
>>> create mode 100644 tests/functional/qemu_test/asset.py
>>>
>>> diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
>>> index 2f1e0bc70d..db05c8f412 100644
>>> --- a/tests/functional/qemu_test/__init__.py
>>> +++ b/tests/functional/qemu_test/__init__.py
>>> @@ -6,6 +6,7 @@
>>> # later. See the COPYING file in the top-level directory.
>>> +from .asset import Asset
>>
>> For next patch?
>
> No, the idea is that a test class can just do 'from qemu_test import Asset',
> so they don't need to be aware of the fact that we've split up the code into
> separate files beneath the qemu_test/ directory.
Oh indeed I missed that, thanks for clarifying.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
2024-08-01 17:38 ` Daniel P. Berrangé
@ 2024-08-02 13:25 ` Philippe Mathieu-Daudé
2024-08-13 12:04 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-02 13:25 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, qemu-s390x, Fabiano Rosas, Paolo Bonzini, John Snow,
qemu-ppc, Richard Henderson, Ani Sinha, Alex Bennée,
Thomas Huth
On 1/8/24 19:38, Daniel P. Berrangé wrote:
> On Thu, Aug 01, 2024 at 07:11:01PM +0200, Philippe Mathieu-Daudé wrote:
>> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>>> From: Thomas Huth <thuth@redhat.com>
>>>
>>> These tests use archive.lzma_uncompress() from the Avocado utils,
>>> so provide a small helper function for this, based on the
>>> standard lzma module from Python instead.
>>>
>>> And while we're at it, replace the MD5 hashes in the topology test
>>> with proper SHA256 hashes, since MD5 should not be used anymore
>>> nowadays.
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>> MAINTAINERS | 4 +-
>>> tests/functional/meson.build | 6 ++
>>> tests/functional/qemu_test/utils.py | 14 +++
>>> .../test_s390x_ccw_virtio.py} | 79 ++++++++---------
>>> .../test_s390x_topology.py} | 86 ++++++++-----------
>>> 5 files changed, 94 insertions(+), 95 deletions(-)
>>> rename tests/{avocado/machine_s390_ccw_virtio.py => functional/test_s390x_ccw_virtio.py} (85%)
>>> mode change 100644 => 100755
>>> rename tests/{avocado/s390_topology.py => functional/test_s390x_topology.py} (88%)
>>> mode change 100644 => 100755
>>
>> Conversion LGTM, so:
>>
>> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>
>> Now while testing I'm getting errors, but maybe pre-existing...
>
> What host arch / OS are you running tests from ?
>
> I got through the full set of testing on Fedora x86_64, so
> wondering if something differs.
Linux s390x but I'm not in the KVM group, I need to find
another place to run these (or ask Alex :P).
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test
2024-07-30 17:03 ` [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test Daniel P. Berrangé
@ 2024-08-02 16:10 ` Philippe Mathieu-Daudé
2024-08-02 16:12 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-02 16:10 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Mostly a straight-forward conversion. Looks like we can simply drop
> the avocado datadrainer stuff when not using the avocado framework
> anymore.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> .../acpi-bits/bits-config/bits-cfg.txt | 0
> .../acpi-bits/bits-tests/smbios.py2 | 0
> .../acpi-bits/bits-tests/smilatency.py2 | 0
> .../acpi-bits/bits-tests/testacpi.py2 | 0
> .../acpi-bits/bits-tests/testcpuid.py2 | 0
> tests/functional/meson.build | 2 +
> .../test_acpi_bits.py} | 81 ++++++++++---------
> 7 files changed, 43 insertions(+), 40 deletions(-)
> rename tests/{avocado => functional}/acpi-bits/bits-config/bits-cfg.txt (100%)
> rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2 (100%)
> rename tests/{avocado => functional}/acpi-bits/bits-tests/smilatency.py2 (100%)
> rename tests/{avocado => functional}/acpi-bits/bits-tests/testacpi.py2 (100%)
> rename tests/{avocado => functional}/acpi-bits/bits-tests/testcpuid.py2 (100%)
> rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py} (86%)
> mode change 100644 => 100755
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test
2024-08-02 16:10 ` Philippe Mathieu-Daudé
@ 2024-08-02 16:12 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-02 16:12 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 2/8/24 18:10, Philippe Mathieu-Daudé wrote:
> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>> From: Thomas Huth <thuth@redhat.com>
>>
>> Mostly a straight-forward conversion. Looks like we can simply drop
>> the avocado datadrainer stuff when not using the avocado framework
>> anymore.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> .../acpi-bits/bits-config/bits-cfg.txt | 0
>> .../acpi-bits/bits-tests/smbios.py2 | 0
>> .../acpi-bits/bits-tests/smilatency.py2 | 0
>> .../acpi-bits/bits-tests/testacpi.py2 | 0
>> .../acpi-bits/bits-tests/testcpuid.py2 | 0
>> tests/functional/meson.build | 2 +
>> .../test_acpi_bits.py} | 81 ++++++++++---------
>> 7 files changed, 43 insertions(+), 40 deletions(-)
>> rename tests/{avocado =>
>> functional}/acpi-bits/bits-config/bits-cfg.txt (100%)
>> rename tests/{avocado => functional}/acpi-bits/bits-tests/smbios.py2
>> (100%)
>> rename tests/{avocado =>
>> functional}/acpi-bits/bits-tests/smilatency.py2 (100%)
>> rename tests/{avocado =>
>> functional}/acpi-bits/bits-tests/testacpi.py2 (100%)
>> rename tests/{avocado =>
>> functional}/acpi-bits/bits-tests/testcpuid.py2 (100%)
>> rename tests/{avocado/acpi-bits.py => functional/test_acpi_bits.py}
>> (86%)
>> mode change 100644 => 100755
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
(missing MAINTAINERS update)
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 15/24] tests/functional: Convert the microblaze avocado tests into standalone tests
2024-07-30 17:03 ` [PATCH v3 15/24] tests/functional: Convert the microblaze avocado tests into standalone tests Daniel P. Berrangé
@ 2024-08-02 16:16 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-02 16:16 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> The machine_microblaze.py file contained two tests, one for each
> endianess. Since we only support one QEMU target binary per file
> in the new functional test environment, we have to split this file
> up into two files now.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> MAINTAINERS | 2 +-
> tests/avocado/machine_microblaze.py | 61 -------------------
> tests/functional/meson.build | 8 +++
> .../functional/test_microblaze_s3adsp1800.py | 39 ++++++++++++
> .../test_microblazeel_s3adsp1800.py | 42 +++++++++++++
> 5 files changed, 90 insertions(+), 62 deletions(-)
> delete mode 100644 tests/avocado/machine_microblaze.py
> create mode 100755 tests/functional/test_microblaze_s3adsp1800.py
> create mode 100755 tests/functional/test_microblazeel_s3adsp1800.py
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 19/24] tests/functional: Convert the ppc_amiga avocado test into a standalone test
2024-07-30 17:03 ` [PATCH v3 19/24] tests/functional: Convert the ppc_amiga avocado test into a standalone test Daniel P. Berrangé
@ 2024-08-02 16:19 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-02 16:19 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Use the Python standard zipfile module instead of avocado.utils for
> extracting the ZIP file that we download here, and use the standard
> subprocess module for running the "tail" command.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/avocado/ppc_amiga.py | 38 ---------------------------
> tests/functional/meson.build | 1 +
> tests/functional/test_ppc_amiga.py | 42 ++++++++++++++++++++++++++++++
> 3 files changed, 43 insertions(+), 38 deletions(-)
> delete mode 100644 tests/avocado/ppc_amiga.py
> create mode 100755 tests/functional/test_ppc_amiga.py
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 17/24] tests/functional: Convert the virtio_gpu avocado test into a standalone test
2024-07-30 17:03 ` [PATCH v3 17/24] tests/functional: Convert the virtio_gpu " Daniel P. Berrangé
@ 2024-08-02 16:35 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 69+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-08-02 16:35 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha, Alex Bennée, Thomas Huth
On 30/7/24 19:03, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Nothing thrilling in here, it's just a straight forward conversion.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> tests/functional/meson.build | 1 +
> .../test_virtio_gpu.py} | 64 +++++++++----------
> 2 files changed, 30 insertions(+), 35 deletions(-)
> rename tests/{avocado/virtio-gpu.py => functional/test_virtio_gpu.py} (73%)
> mode change 100644 => 100755
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 20/24] tests/functional: Convert the ppc_hv avocado test into a standalone test
2024-07-30 17:03 ` [PATCH v3 20/24] tests/functional: Convert the ppc_hv " Daniel P. Berrangé
@ 2024-08-05 22:58 ` Nicholas Piggin
0 siblings, 0 replies; 69+ messages in thread
From: Nicholas Piggin @ 2024-08-05 22:58 UTC (permalink / raw)
To: Daniel P. Berrangé, qemu-devel
Cc: Fabiano Rosas, Paolo Bonzini, Philippe Mathieu-Daudé,
John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée, Thomas Huth
Thanks for doing this.
I have a fix for this problem, it's a bug with the test harness
code but was not merged yet. I'll re-send it.
Thanks,
Nick
On Wed Jul 31, 2024 at 3:03 AM AEST, Daniel P. Berrangé wrote:
> From: Thomas Huth <thuth@redhat.com>
>
> Note: The original Avocado test seems currently to be broken, it hangs
> when the guest is trying to install additional packages. So mark it as
> broken for now until it gets fixed.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> .../test_ppc64_hv.py} | 48 ++++++++-----------
> 1 file changed, 20 insertions(+), 28 deletions(-)
> rename tests/{avocado/ppc_hv_tests.py => functional/test_ppc64_hv.py} (88%)
> mode change 100644 => 100755
>
> diff --git a/tests/avocado/ppc_hv_tests.py b/tests/functional/test_ppc64_hv.py
> old mode 100644
> new mode 100755
> similarity index 88%
> rename from tests/avocado/ppc_hv_tests.py
> rename to tests/functional/test_ppc64_hv.py
> index bf8822bb97..a45657c87e
> --- a/tests/avocado/ppc_hv_tests.py
> +++ b/tests/functional/test_ppc64_hv.py
> @@ -1,3 +1,5 @@
> +#!/usr/bin/env python3
> +#
> # Tests that specifically try to exercise hypervisor features of the
> # target machines. powernv supports the Power hypervisor ISA, and
> # pseries supports the nested-HV hypervisor spec.
> @@ -7,10 +9,10 @@
> # This work is licensed under the terms of the GNU GPL, version 2 or
> # later. See the COPYING file in the top-level directory.
>
> -from avocado import skipIf, skipUnless
> -from avocado.utils import archive
> -from avocado_qemu import QemuSystemTest
> -from avocado_qemu import wait_for_console_pattern, exec_command
> +from unittest import skipIf, skipUnless
> +from qemu_test.utils import archive_extract
> +from qemu_test import QemuSystemTest, Asset
> +from qemu_test import wait_for_console_pattern, exec_command
> import os
> import time
> import subprocess
> @@ -47,7 +49,7 @@ def missing_deps():
> @skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps))
> @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test sometimes gets stuck due to console handling problem')
> @skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
> -@skipUnless(os.getenv('SPEED') == 'slow', 'runtime limited')
> +@skip('broken test - guest fails to install packages')
> class HypervisorTest(QemuSystemTest):
>
> timeout = 1000
> @@ -55,6 +57,12 @@ class HypervisorTest(QemuSystemTest):
> panic_message = 'Kernel panic - not syncing'
> good_message = 'VFS: Cannot open root device'
>
> + # Alpine use sha256 so I recalculated this myself
> + ASSET_ISO = Asset(
> + ('https://dl-cdn.alpinelinux.org/alpine/v3.18/'
> + 'releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso'),
> + 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff')
> +
> def extract_from_iso(self, iso, path):
> """
> Extracts a file from an iso file into the test workdir
> @@ -84,14 +92,7 @@ def extract_from_iso(self, iso, path):
> def setUp(self):
> super().setUp()
>
> - iso_url = ('https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/ppc64le/alpine-standard-3.18.4-ppc64le.iso')
> -
> - # Alpine use sha256 so I recalculated this myself
> - iso_sha256 = 'c26b8d3e17c2f3f0fed02b4b1296589c2390e6d5548610099af75300edd7b3ff'
> - iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha256,
> - algorithm = "sha256")
> -
> - self.iso_path = iso_path
> + self.iso_path = self.ASSET_ISO.fetch()
> self.vmlinuz = self.extract_from_iso(iso_path, '/boot/vmlinuz-lts')
> self.initramfs = self.extract_from_iso(iso_path, '/boot/initramfs-lts')
>
> @@ -159,12 +160,8 @@ def do_test_kvm(self, hpt=False):
> wait_for_console_pattern(self, 'alpine:~#')
>
> def test_hv_pseries(self):
> - """
> - :avocado: tags=arch:ppc64
> - :avocado: tags=machine:pseries
> - :avocado: tags=accel:tcg
> - """
> self.require_accelerator("tcg")
> + self.set_machine('pseries')
> self.vm.add_args("-accel", "tcg,thread=multi")
> self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
> self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on")
> @@ -174,12 +171,8 @@ def test_hv_pseries(self):
> self.do_stop_alpine()
>
> def test_hv_pseries_kvm(self):
> - """
> - :avocado: tags=arch:ppc64
> - :avocado: tags=machine:pseries
> - :avocado: tags=accel:kvm
> - """
> self.require_accelerator("kvm")
> + self.set_machine('pseries')
> self.vm.add_args("-accel", "kvm")
> self.vm.add_args('-device', 'nvme,serial=1234,drive=drive0')
> self.vm.add_args("-machine", "x-vof=on,cap-nested-hv=on,cap-ccf-assist=off")
> @@ -189,12 +182,8 @@ def test_hv_pseries_kvm(self):
> self.do_stop_alpine()
>
> def test_hv_powernv(self):
> - """
> - :avocado: tags=arch:ppc64
> - :avocado: tags=machine:powernv
> - :avocado: tags=accel:tcg
> - """
> self.require_accelerator("tcg")
> + self.set_machine('powernv')
> self.vm.add_args("-accel", "tcg,thread=multi")
> self.vm.add_args('-device', 'nvme,bus=pcie.2,addr=0x0,serial=1234,drive=drive0',
> '-device', 'e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0',
> @@ -204,3 +193,6 @@ def test_hv_powernv(self):
> self.do_test_kvm()
> self.do_test_kvm(True)
> self.do_stop_alpine()
> +
> +if __name__ == '__main__':
> + QemuSystemTest.main()
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 03/24] tests/functional: Set up logging
2024-08-01 10:10 ` Alex Bennée
@ 2024-08-13 9:34 ` Thomas Huth
0 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-13 9:34 UTC (permalink / raw)
To: Alex Bennée, Daniel P. Berrangé
Cc: qemu-devel, Fabiano Rosas, Paolo Bonzini,
Philippe Mathieu-Daudé, John Snow, qemu-ppc,
Richard Henderson, Ani Sinha
On 01/08/2024 12.10, Alex Bennée wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
>
>> From: Thomas Huth <thuth@redhat.com>
>>
>> Create log files for each test separately, one file that contains
>> the basic logging and one that contains the console output.
>>
>> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/functional/qemu_test/testcase.py | 27 +++++++++++++++++++++++++-
>> 1 file changed, 26 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py
>> index 82cc1d454f..27bbf4a0af 100644
>> --- a/tests/functional/qemu_test/testcase.py
>> +++ b/tests/functional/qemu_test/testcase.py
>> @@ -31,7 +31,8 @@ class QemuBaseTest(unittest.TestCase):
>> arch = None
>>
>> workdir = None
>> - log = logging.getLogger('qemu-test')
>> + log = None
>> + logdir = None
>>
>> def setUp(self, bin_prefix):
>> self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
>> @@ -42,6 +43,20 @@ def setUp(self, bin_prefix):
>> if not os.path.exists(self.workdir):
>> os.makedirs(self.workdir)
>>
>> + self.logdir = self.workdir
>> + self.log = logging.getLogger('qemu-test')
>> + self.log.setLevel(logging.DEBUG)
>> + self._log_fh = logging.FileHandler(os.path.join(self.logdir,
>> + 'base.log'), mode='w')
>> + self._log_fh.setLevel(logging.DEBUG)
>> + fileFormatter = logging.Formatter(
>> + '%(asctime)s - %(levelname)s: %(message)s')
>> + self._log_fh.setFormatter(fileFormatter)
>> + self.log.addHandler(self._log_fh)
>
> This is in the wrong place, we have a unique machine directory for each
> run, the console.log should go in there.
self.workdir already points to the unique directory of the test (see second
patch), so setting logdir = workdir should be fine, I think?
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests
2024-08-01 16:12 ` Philippe Mathieu-Daudé
@ 2024-08-13 12:00 ` Thomas Huth
0 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-13 12:00 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Daniel P. Berrangé, qemu-devel,
Paolo Bonzini
Cc: Fabiano Rosas, John Snow, qemu-ppc, Richard Henderson, Ani Sinha,
Alex Bennée
On 01/08/2024 18.12, Philippe Mathieu-Daudé wrote:
> Hi,
>
> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>> From: Thomas Huth <thuth@redhat.com>
>>
>> Provide a meson.build file for the upcoming python-based functional
>> tests, and add some wrapper glue targets to the tests/Makefile.include
>> file. We are going to use two "speed" modes for the functional tests:
>> The "quick" tests can be run at any time (i.e. also during "make check"),
>> while the "thorough" tests should only be run when running a
>> "make check-functional" test run (since these tests might download
>> additional assets from the internet).
>>
>> The changes to the meson.build files are partly based on an earlier
>> patch by Ani Sinha.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>> tests/Makefile.include | 11 ++++++
>> tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
>> tests/meson.build | 1 +
>> 3 files changed, 78 insertions(+)
>> create mode 100644 tests/functional/meson.build
>
>
>> +foreach speed : ['quick', 'thorough']
>> + foreach dir : target_dirs
>> + if not dir.endswith('-softmmu')
>> + continue
>> + endif
>> +
>> + target_base = dir.split('-')[0]
>> + test_emulator = emulators['qemu-system-' + target_base]
>> +
>> + if speed == 'quick'
>> + suites = ['func-quick', 'func-' + target_base]
>> + target_tests = get_variable('tests_' + target_base + '_quick', [])
>> + tests_generic
>> + else
>> + suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
>> + target_tests = get_variable('tests_' + target_base + '_' + speed, [])
>> + endif
>> +
>> + test_deps = roms
>> + test_env = environment()
>> + if have_tools
>> + test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() /
>> 'qemu-img')
>> + test_deps += [qemu_img]
>> + endif
>> + test_env.set('QEMU_TEST_QEMU_BINARY',
>> + meson.global_build_root() / 'qemu-system-' + target_base)
>> + test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())
>> + test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
>> + meson.current_source_dir())
>> +
>> + foreach test : target_tests
>> + test('func-@0@/@1@'.format(target_base, test),
>> + python,
>> + depends: [test_deps, test_emulator, emulator_modules],
>> + env: test_env,
>> + args: [meson.current_source_dir() / 'test_' + test + '.py'],
>> + protocol: 'tap',
>> + timeout: test_timeouts.get(test, 60),
>> + priority: test_timeouts.get(test, 60),
>> + suite: suites)
>> + endforeach
>> + endforeach
>> +endforeach
>
> Can we have meson report why a test is skipped? All Python
> unittest @skip* decorators provide a 'reason' argument, so
> we already have that information.
I guess you'd need to extend the meson test runner for this...
> Not much help on meson doc:
> https://mesonbuild.com/Unit-tests.html#skipped-tests-and-hard-errors
>
> If I run a test manually I see the reason in the tap output:
>
> $ FOO=bar tests/functional/test_mips64el_loongson3v.py
> TAP version 13
> ok 1 test_mips64el_loongson3v.MipsLoongson3v.test_pmon_serial_console # SKIP
> untrusted code
> 1..1
You can also see this information in the txt file in the meson-logs folder,
so it is certainly not completely lost.
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
2024-08-02 13:25 ` Philippe Mathieu-Daudé
@ 2024-08-13 12:04 ` Thomas Huth
2024-08-13 12:08 ` Thomas Huth
0 siblings, 1 reply; 69+ messages in thread
From: Thomas Huth @ 2024-08-13 12:04 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Daniel P. Berrangé,
Nina Schoetterl-Glausch
Cc: qemu-devel, qemu-s390x, Fabiano Rosas, Paolo Bonzini, John Snow,
qemu-ppc, Richard Henderson, Ani Sinha, Alex Bennée
On 02/08/2024 15.25, Philippe Mathieu-Daudé wrote:
> On 1/8/24 19:38, Daniel P. Berrangé wrote:
>> On Thu, Aug 01, 2024 at 07:11:01PM +0200, Philippe Mathieu-Daudé wrote:
>>> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>>>> From: Thomas Huth <thuth@redhat.com>
>>>>
>>>> These tests use archive.lzma_uncompress() from the Avocado utils,
>>>> so provide a small helper function for this, based on the
>>>> standard lzma module from Python instead.
>>>>
>>>> And while we're at it, replace the MD5 hashes in the topology test
>>>> with proper SHA256 hashes, since MD5 should not be used anymore
>>>> nowadays.
>>>>
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>> MAINTAINERS | 4 +-
>>>> tests/functional/meson.build | 6 ++
>>>> tests/functional/qemu_test/utils.py | 14 +++
>>>> .../test_s390x_ccw_virtio.py} | 79 ++++++++---------
>>>> .../test_s390x_topology.py} | 86 ++++++++-----------
>>>> 5 files changed, 94 insertions(+), 95 deletions(-)
>>>> rename tests/{avocado/machine_s390_ccw_virtio.py =>
>>>> functional/test_s390x_ccw_virtio.py} (85%)
>>>> mode change 100644 => 100755
>>>> rename tests/{avocado/s390_topology.py =>
>>>> functional/test_s390x_topology.py} (88%)
>>>> mode change 100644 => 100755
>>>
>>> Conversion LGTM, so:
>>>
>>> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>>
>>> Now while testing I'm getting errors, but maybe pre-existing...
>>
>> What host arch / OS are you running tests from ?
>>
>> I got through the full set of testing on Fedora x86_64, so
>> wondering if something differs.
>
> Linux s390x but I'm not in the KVM group, I need to find
> another place to run these (or ask Alex :P).
Sounds like a pre-existing problem ... I guess the test should check whether
it can access /dev/kvm before trying to use it...?
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests
2024-08-13 12:04 ` Thomas Huth
@ 2024-08-13 12:08 ` Thomas Huth
0 siblings, 0 replies; 69+ messages in thread
From: Thomas Huth @ 2024-08-13 12:08 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Daniel P. Berrangé,
Nina Schoetterl-Glausch
Cc: qemu-devel, qemu-s390x, Fabiano Rosas, Paolo Bonzini, John Snow,
qemu-ppc, Richard Henderson, Ani Sinha, Alex Bennée
On 13/08/2024 14.04, Thomas Huth wrote:
> On 02/08/2024 15.25, Philippe Mathieu-Daudé wrote:
>> On 1/8/24 19:38, Daniel P. Berrangé wrote:
>>> On Thu, Aug 01, 2024 at 07:11:01PM +0200, Philippe Mathieu-Daudé wrote:
>>>> On 30/7/24 19:03, Daniel P. Berrangé wrote:
>>>>> From: Thomas Huth <thuth@redhat.com>
>>>>>
>>>>> These tests use archive.lzma_uncompress() from the Avocado utils,
>>>>> so provide a small helper function for this, based on the
>>>>> standard lzma module from Python instead.
>>>>>
>>>>> And while we're at it, replace the MD5 hashes in the topology test
>>>>> with proper SHA256 hashes, since MD5 should not be used anymore
>>>>> nowadays.
>>>>>
>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>> ---
>>>>> MAINTAINERS | 4 +-
>>>>> tests/functional/meson.build | 6 ++
>>>>> tests/functional/qemu_test/utils.py | 14 +++
>>>>> .../test_s390x_ccw_virtio.py} | 79 ++++++++---------
>>>>> .../test_s390x_topology.py} | 86 ++++++++-----------
>>>>> 5 files changed, 94 insertions(+), 95 deletions(-)
>>>>> rename tests/{avocado/machine_s390_ccw_virtio.py =>
>>>>> functional/test_s390x_ccw_virtio.py} (85%)
>>>>> mode change 100644 => 100755
>>>>> rename tests/{avocado/s390_topology.py =>
>>>>> functional/test_s390x_topology.py} (88%)
>>>>> mode change 100644 => 100755
>>>>
>>>> Conversion LGTM, so:
>>>>
>>>> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>>>
>>>> Now while testing I'm getting errors, but maybe pre-existing...
>>>
>>> What host arch / OS are you running tests from ?
>>>
>>> I got through the full set of testing on Fedora x86_64, so
>>> wondering if something differs.
>>
>> Linux s390x but I'm not in the KVM group, I need to find
>> another place to run these (or ask Alex :P).
>
> Sounds like a pre-existing problem ... I guess the test should check whether
> it can access /dev/kvm before trying to use it...?
Ah, no, I missed your comment in your earlier patch that kvm_available()
already checks this ... so the problem must be something else?
Thomas
^ permalink raw reply [flat|nested] 69+ messages in thread
end of thread, other threads:[~2024-08-13 12:09 UTC | newest]
Thread overview: 69+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-30 17:03 [PATCH v3 00/24] Convert avocado tests to normal Python unittests Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 01/24] python: Install pycotap in our venv if necessary Daniel P. Berrangé
2024-07-31 12:49 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests Daniel P. Berrangé
2024-07-31 12:53 ` Philippe Mathieu-Daudé
2024-07-31 14:24 ` Alex Bennée
2024-08-01 10:07 ` Daniel P. Berrangé
2024-08-01 16:11 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 03/24] tests/functional: Set up logging Daniel P. Berrangé
2024-08-01 10:10 ` Alex Bennée
2024-08-13 9:34 ` Thomas Huth
2024-07-30 17:03 ` [PATCH v3 04/24] tests/Makefile.include: Increase the level of indentation in the help text Daniel P. Berrangé
2024-07-31 12:49 ` Philippe Mathieu-Daudé
2024-08-01 10:00 ` Alex Bennée
2024-07-30 17:03 ` [PATCH v3 05/24] tests/functional: Prepare the meson build system for the functional tests Daniel P. Berrangé
2024-08-01 16:12 ` Philippe Mathieu-Daudé
2024-08-13 12:00 ` Thomas Huth
2024-07-30 17:03 ` [PATCH v3 06/24] tests/functional: Convert simple avocado tests into standalone python tests Daniel P. Berrangé
2024-07-31 12:52 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 07/24] tests/functional: Convert avocado tests that just need a small adjustment Daniel P. Berrangé
2024-07-31 12:55 ` Philippe Mathieu-Daudé
2024-08-01 16:13 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 08/24] tests/functional: add a module for handling asset download & caching Daniel P. Berrangé
2024-08-01 16:20 ` Philippe Mathieu-Daudé
2024-08-01 17:00 ` Daniel P. Berrangé
2024-08-01 17:02 ` Daniel P. Berrangé
2024-08-02 13:24 ` Philippe Mathieu-Daudé
2024-08-01 21:51 ` Richard Henderson
2024-08-02 8:32 ` Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets Daniel P. Berrangé
2024-07-31 5:01 ` Richard Henderson
2024-08-01 16:01 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 10/24] tests/functional: Convert some tests that download files via fetch_asset() Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 11/24] tests/functional: Add a function for extracting files from an archive Daniel P. Berrangé
2024-08-01 16:22 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 12/24] tests/functional: Convert some avocado tests that needed avocado.utils.archive Daniel P. Berrangé
2024-08-01 16:27 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 13/24] tests/functional: Convert the s390x avocado tests into standalone tests Daniel P. Berrangé
2024-08-01 17:11 ` Philippe Mathieu-Daudé
2024-08-01 17:38 ` Daniel P. Berrangé
2024-08-02 13:25 ` Philippe Mathieu-Daudé
2024-08-13 12:04 ` Thomas Huth
2024-08-13 12:08 ` Thomas Huth
2024-07-30 17:03 ` [PATCH v3 14/24] tests/functional: Convert the x86_cpu_model_versions test Daniel P. Berrangé
2024-08-01 16:43 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 15/24] tests/functional: Convert the microblaze avocado tests into standalone tests Daniel P. Berrangé
2024-08-02 16:16 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 16/24] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Daniel P. Berrangé
2024-07-31 12:56 ` Philippe Mathieu-Daudé
2024-07-31 23:00 ` Richard Henderson
2024-08-01 10:09 ` Daniel P. Berrangé
2024-08-01 15:11 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 17/24] tests/functional: Convert the virtio_gpu " Daniel P. Berrangé
2024-08-02 16:35 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 18/24] tests/functional: Convert most ppc avocado tests into standalone tests Daniel P. Berrangé
2024-07-30 17:03 ` [PATCH v3 19/24] tests/functional: Convert the ppc_amiga avocado test into a standalone test Daniel P. Berrangé
2024-08-02 16:19 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 20/24] tests/functional: Convert the ppc_hv " Daniel P. Berrangé
2024-08-05 22:58 ` Nicholas Piggin
2024-07-30 17:03 ` [PATCH v3 21/24] tests/functional: Convert the m68k nextcube test with tesseract Daniel P. Berrangé
2024-08-01 16:33 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 22/24] tests/functional: Convert the acpi-bits test into a standalone test Daniel P. Berrangé
2024-08-02 16:10 ` Philippe Mathieu-Daudé
2024-08-02 16:12 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 23/24] tests/functional: Convert the rx_gdbsim avocado " Daniel P. Berrangé
2024-08-01 16:40 ` Philippe Mathieu-Daudé
2024-07-30 17:03 ` [PATCH v3 24/24] gitlab-ci: Add "check-functional" to the build tests Daniel P. Berrangé
2024-08-01 16:40 ` Philippe Mathieu-Daudé
2024-07-30 18:38 ` [PATCH v3 00/24] Convert avocado tests to normal Python unittests Philippe Mathieu-Daudé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).