* [PULL 00/14] Docs / testing patches for 2024-07-29
@ 2024-07-29 15:27 Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 01/14] docs: add test for firmware.json QAPI Philippe Mathieu-Daudé
` (14 more replies)
0 siblings, 15 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daudé
The following changes since commit 93b799fafd9170da3a79a533ea6f73a18de82e22:
Merge tag 'pull-ppc-for-9.1-2-20240726-1' of https://gitlab.com/npiggin/qemu into staging (2024-07-26 15:10:45 +1000)
are available in the Git repository at:
https://github.com/philmd/qemu.git tags/docs-testing-20240729
for you to fetch changes up to 8e2275592299b637e87cc905d20bfdeffa586a83:
tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset (2024-07-29 16:33:24 +0200)
Ignored warnings:
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
WARNING: line over 80 characters
----------------------------------------------------------------
Docs & testing patch queue
- Test QAPI firmware.json schema (Thomas)
- Handle new env.doc2path() return value (Peter)
- Convert some Avocado tests to simpler python-based functional testing (Thomas)
- Improve how assets are used by some Avocado tests (Cleber)
----------------------------------------------------------------
Cleber Rosa (3):
tests/avocado: mips: fallback to HTTP given certificate expiration
tests/avocado: mips: add hint for fetchasset plugin
tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset
Peter Maydell (1):
docs/sphinx/depfile.py: Handle env.doc2path() returning a Path not a
str
Thomas Huth (9):
python: Install pycotap in our venv if necessary
tests/functional: Add base classes for the upcoming pytest-based tests
tests/Makefile.include: Increase the level of indentation in the help
text
tests/functional: Prepare the meson build system for the functional
tests
tests/functional: Convert simple avocado tests into standalone python
tests
tests/functional: Convert avocado tests that just need a small
adjustment
tests/functional: Convert the x86_cpu_model_versions test
tests/functional: Convert the riscv_opensbi avocado test into a
standalone test
gitlab-ci: Add "check-functional" to the build tests
Thomas Weißschuh (1):
docs: add test for firmware.json QAPI
MAINTAINERS | 11 +
docs/meson.build | 5 +
docs/sphinx/depfile.py | 2 +-
.gitlab-ci.d/buildtest-template.yml | 3 +-
.gitlab-ci.d/buildtest.yml | 60 ++--
python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
pythondeps.toml | 1 +
tests/Makefile.include | 41 ++-
tests/avocado/boot_linux_console.py | 24 +-
tests/avocado/riscv_opensbi.py | 63 ----
tests/functional/meson.build | 86 +++++
tests/functional/qemu_test/__init__.py | 328 ++++++++++++++++++
.../test_cpu_queries.py} | 7 +-
.../test_empty_cpu_model.py} | 7 +-
.../test_info_usernet.py} | 11 +-
.../test_mem_addr_space.py} | 52 +--
.../test_pc_cpu_hotplug_props.py} | 11 +-
.../test_ppc_74xx.py} | 74 ++--
tests/functional/test_riscv_opensbi.py | 36 ++
.../version.py => functional/test_version.py} | 13 +-
.../test_virtio_version.py} | 8 +-
.../test_x86_cpu_model_versions.py} | 63 +---
tests/meson.build | 1 +
23 files changed, 636 insertions(+), 271 deletions(-)
create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
delete mode 100644 tests/avocado/riscv_opensbi.py
create mode 100644 tests/functional/meson.build
create mode 100644 tests/functional/qemu_test/__init__.py
rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%)
mode change 100644 => 100755
rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
mode change 100644 => 100755
rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
mode change 100644 => 100755
rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
mode change 100644 => 100755
rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
mode change 100644 => 100755
rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
mode change 100644 => 100755
create mode 100755 tests/functional/test_riscv_opensbi.py
rename tests/{avocado/version.py => functional/test_version.py} (78%)
mode change 100644 => 100755
rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
mode change 100644 => 100755
rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
mode change 100644 => 100755
--
2.45.2
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PULL 01/14] docs: add test for firmware.json QAPI
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 02/14] docs/sphinx/depfile.py: Handle env.doc2path() returning a Path not a str Philippe Mathieu-Daudé
` (13 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Weißschuh, Philippe Mathieu-Daudé,
Daniel P . Berrangé, Markus Armbruster
From: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
To make sure that the QAPI description stays valid, add a testcase.
Suggested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Link: https://lore.kernel.org/qemu-devel/d9ce0234-4beb-4b90-b14c-76810d3b81d7@linaro.org/
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724-qapi-firmware-json-v7-1-12341f7e362d@linutronix.de>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
docs/meson.build | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/docs/meson.build b/docs/meson.build
index 9040f860ae..322452c877 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -99,3 +99,8 @@ if build_docs
alias_target('html', sphinxdocs)
alias_target('man', sphinxmans)
endif
+
+test('QAPI firmware.json regression tests', qapi_gen,
+ args: ['-o', meson.current_build_dir() / 'qapi',
+ meson.current_source_dir() / 'interop/firmware.json'],
+ suite: ['qapi-schema', 'qapi-interop'])
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 02/14] docs/sphinx/depfile.py: Handle env.doc2path() returning a Path not a str
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 01/14] docs: add test for firmware.json QAPI Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 03/14] python: Install pycotap in our venv if necessary Philippe Mathieu-Daudé
` (12 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, qemu-stable, Philippe Mathieu-Daudé
From: Peter Maydell <peter.maydell@linaro.org>
In newer versions of Sphinx the env.doc2path() API is going to change
to return a Path object rather than a str. This was originally visible
in Sphinx 8.0.0rc1, but has been rolled back for the final 8.0.0
release. However it will probably emit a deprecation warning and is
likely to change for good in 9.0:
https://github.com/sphinx-doc/sphinx/issues/12686
Our use in depfile.py assumes a str, and if it is passed a Path
it will fall over:
Handler <function write_depfile at 0x77a1775ff560> for event 'build-finished' threw an exception (exception: unsupported operand type(s) for +: 'PosixPath' and 'str')
Wrapping the env.doc2path() call in str() will coerce a Path object
to the str we expect, and have no effect in older Sphinx versions
that do return a str.
Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2458
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240729120533.2486427-1-peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
docs/sphinx/depfile.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/sphinx/depfile.py b/docs/sphinx/depfile.py
index afdcbcec6e..e74be6af98 100644
--- a/docs/sphinx/depfile.py
+++ b/docs/sphinx/depfile.py
@@ -19,7 +19,7 @@
def get_infiles(env):
for x in env.found_docs:
- yield env.doc2path(x)
+ yield str(env.doc2path(x))
yield from ((os.path.join(env.srcdir, dep)
for dep in env.dependencies[x]))
for mod in sys.modules.values():
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 03/14] python: Install pycotap in our venv if necessary
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 01/14] docs: add test for firmware.json QAPI Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 02/14] docs/sphinx/depfile.py: Handle env.doc2path() returning a Path not a str Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 04/14] tests/functional: Add base classes for the upcoming pytest-based tests Philippe Mathieu-Daudé
` (11 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Thomas Huth, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
The upcoming functional tests will require pycotap for providing
TAP output from the python-based tests. Since we want to be able
to run some of the tests offline by default, too, let's install
it along with meson in our venv if necessary (it's size is only
5 kB, so adding the wheel here should not really be a problem).
The wheel file has been obtained with:
pip download --only-binary :all: --dest . --no-cache pycotap
Signed-off-by: Thomas Huth <thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-2-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes
pythondeps.toml | 1 +
2 files changed, 1 insertion(+)
create mode 100644 python/wheels/pycotap-1.3.1-py3-none-any.whl
diff --git a/python/wheels/pycotap-1.3.1-py3-none-any.whl b/python/wheels/pycotap-1.3.1-py3-none-any.whl
new file mode 100644
index 0000000000000000000000000000000000000000..9c2c7d259365c8a0487228c6878e2c0b10ba6bdd
GIT binary patch
literal 5119
zcmeI0XH-+^*2hB?0-*~??*uUfq=gPrO$Z=_u0ZG@O^6~QNbe=o02+`gMUW;%Ec7O!
zi(;gsC?Ft3iYPD6y)(}E&UnAx5BIF|JZGH`=ePD+XaE0u@5cy1M$Q5N0H^?|l6bQv
zES3d80RXr$0|3W2z$d`L8-w<N+S<B#xnXQ=C42%NS`fV!<QRf>%?J9a_|Q(1A4=i{
z)PdwgFGfrnh-nv{0;09#Ai6t_@|AgzZr?iK|5`(*Jq7N=Ia;g5F5Z#uXinMQAi&b>
zl?`99PYQbEzKE-9v!mkOduJ*1JcCqzb2hr~Q?gI^$_YA?wn?>Hew1tQvP5O>WGQYj
z#;O2G-DIqMmOWT1DxnLkUdbQ<Ey0Q~o{bgqiKpUKwY}BcbO2(#sGe|I-&lcNXa&Ru
zd>0XX%Q$JV?u?prCOFbKW%1n53GtgZwD_58-z`aJDI=$mn=>B(cKmxBOmq~V&0-@U
zP*r-#42w!!ou^m`|Goe+OED6bGN;@nEqL}Ee-pbaThKhhVvGGPA6!#xXD>BMBJU_R
zWwV|sCuh@Ewq1Vanb2J7m9kVlR%zQosM^WAC3xhBd!l~ZsTeF|MCFrSMt)wdt)bhN
zJh%?kTnr7nA@P!Zfz-J!!pue5-2Tx<sEd1ASvHEa<01V%W%&q4F)a>;*dINCgo$^$
z35b$oCn+N@m+fkp1hz>ab4kX|*%8L3?_YsT)F%|>bWpN62_w5ZHO`vfLHj3S@el0_
z@%!Dmjp%x)Yo*k3B%6I@HpGVNX{=WM3f4M*U~a+RHhav6sBbSOaS2Y&JOwH)?e~Mf
z^MV!@G48+CkC?f*1fxo_pARp5Z4{7JBl4`(29!D703;qg0ut=3S%B0Wi%gGvm8w+4
zFN>DabeFD@KS%>NTUd^(gwz&Y8XBF6CE9SVYAFoUagJ9z;iL~}-?+dv&;>>mU7Ko{
z?1~s-_HaUT*9&+^iI%2@Aj$nA0`nLK7-`twU!MEA*Qp`)%&2~qm3F>=eqiAR{y2l;
z#uXDbw(<0|o?4PslQpjMu(&zBWl~@5!WGjD$I@`2Ybp=mRq*7K)ixkZH*BLp$7AX;
zzqfoeer=XMBX8Gpdh<>P?;W~Qd#5{_C4G<a(OUWQ{nSwkk*Iv511`5ZT7O;{*(w1U
z%+0Z$hUf~wsd95xfgrf=5}rtTz?wmy)TTHcbWb~-tBaxa<Jf!uRld>g-cN_DZ4#H`
z9tdOr5=;+xbVj9v@;>qReBh^fNk#n}1W6RJ%}CvFHh)Q8YQ;hlCS2YlNf_QKzlw|;
zkhQl-tjnmku6rsa>(t`H+v+M)-tW}HvPfa!xebBc+jyw~3zvyH5f&#o*<{1#lNOaC
zX6n0RZWePR=KwIxM`dN4wYI4hgs%U}c%@_jGWw=CeE$ox<Aq=aSrIltQIq%)!=xn&
zDJ$k9Tp&GWtw*S9<YcIU5=-a}V<<XX)jLKzPuKxy&~6j<)$Y(W0gOL5Fq(**N#@|m
zM0Grk7(FS%$onQ%<(^h|0X8Z5{0o^5*i%A5%t34XYiBYuc>_Ndl~<Rtg|>9;@EVv?
zFuiDviL61-$1u7Tnykm1+%q%WGuEnoxOlP(&o{a^yqrmGoor%NhFc>UZ`H&$(zARi
z3BvRv2I=BcyRAwMO2Nrqx1Kc(ZdInck*~XHimC<U@?LWVeOsQ~<)x*W6*s9z48zTo
z^b8<|oeHg!Y(*1+$_@#uZpsIhqSv+U2VK8tPby3#80{k6sO63{d*2d=Rt^TeCal-p
zSF{^;cn<MH+eeEZF2d5REaX80$TaoOM*`N~Q^7B<KXY*3EjP_xHO&{ZA+za7xVsOA
z^7xaQidgVV?y(vDVVH9gc(YG4LOpiJMj3d#{%V+>Xg9Cl4AaC6H4KGFj2iY2j2RNd
zO~9pR`g(_?-_7?AKzgC4FMtWp&xoCeNQBa`phT8%LaaVl+8noinWeX6e78*Ce#R#~
zP9AP<8DC{lXVNGR2q@E5=PzY_!q5}K(9|Tt2KW(>T&|1`&tfVuh7<sRkplp{KO7t)
zB_S;#CE<w1prOBQnubP}=M8jCFA%(llC)31h{fbuwIUixTDD7G!376WK$dQqg%091
zay>vHCyhK;`dT*54<(mr*d?`s4}*IL@VQS^wi&&MN_z+U&*hEY?L80u>~;91C((!4
za~(*(yX(Bi66A-e!#5*R`}Yb#Z369-?-S))Twr_72_*>hdLl_<bc6R{TM+}d%!2ft
z1bW!)wVIkjZu(*McuI!sP9C=~Vh>^g6sF{Q&Zp87ORxAcH`zG>DfHwCw|)rm6=ze~
z17>;uG(DFblGQAu^@kK}oPqvHv&fj~hRzg$Qtm<Sfa{+0pIsjSWmG+d`P_WzvO;c_
z_+ao;V>9|S21`?RntA73VsH2-S<s?92zn-23)2Bq&g?O51?yrfw_B(+Ck*R{uU}&P
z&|d7XyN4w=uUGrRC`^zUQ@CiEb9dfwOsAaT=6Y1yC9u^>_n>jX23Cl}nlSiP&Lk68
zm@HB7f{g|fGDb9gP&q9BKFFntM)Q&qX=dt6ST1RFOmt(@m$a=k9`2(jJExYR3gd6j
zd1uY@xtF5$g?w{6Koy^Y(%N<2J2g*)P<0hepKAt)G;Un#Ri^k>X#BV;9~En18aZph
zd{%s-8JjHGqhRR{?u5Gvy%5IEX23Ucdd)66A#T<-gCnL(su`>VTx!85B}=nL1+3Zh
zH6EIoTKc6ArI-PmjY`|iFD{=dw+HPptKPM#EyyZ*#W7XUXKQ9=EzE=C$aG5aJ2R-D
z=&%*mArzTk8y0H{RIskm^}Tmji&eI*YE2GdP)xqV9-PfuT*PiE_ORV0PAub_+9O6;
z!%(pcW9L}QwQg968t`f9)e8&XLg$-yM91?*Ot0iT3+Pi2$5xbduD8`K^8QF-CPTd0
ze@J3CKMcgp%h?<Hza;)&693mG@d*CS;|MYU;3Fjf!1w>1_>Uw;YNIr?G*B9mro<#p
zdD@`72Ykagq+RlDR*E<cqw%S`9~&keFSJl>j;9E}E?i9jzpq`}jy~YuufgWjpvgG5
zyBf4ol$Di>)}_P>Q?0&5@>}_WZkjwRpDrWe+$bz5x_MRZN%=FU1rhpipaa=uy@^Du
zqzKid0xecCBgA)8LHkt|FQ46j%8>!Nl$Z-jvhGP$a4`wah8t#zOHC%YU#%{C-eNUY
z4TLF=E=Qp8Cc62dYKyE5p>OUT^%7npxY9pz?i~aYMl!8-I+D$H(-XDIr>zntd^5ka
zc<v}A=>kRMTG9MUw}HX?)oVsmpHJ(wH1;-6&oX~rJb|H>J73I|?>4Js(XPptcGce4
z%;VM0^|$^@Op4)@6O$GgjxJBSZj{ZFsnkZR)mtJ&qT{F6%9^+n>+=59pH+vMb-mzH
zqFY;x^7V$gCpXHIbnMr{fOpQ*(F-!apUqwQ@QG&lDT}L;pIo8EH;+J({m)-JqFel*
zp-f*45-zvt7%_aD4ODh|j3VTc)_*Kx^9+O(mS<>HUt8_{5OJuLbK^Rs96b_~mQyJk
z*+Ktt#%1ahr_sIOn`R*|AkCc@S;UKz@>vM1sG8QCVF~u>vMko+oJ+V(3RMev@vl+g
z9SGmAd&Yy%+&-#_u86qgz?!qd?@f)U(L>{q(|{Mdm~;W%{>@J7WV<obQ>g<d^xf`^
z(xCbXqr>cipHc*;I`1nPDF<+qDy&`eR!~^F=HdA^vqA&Ae>*FEUx_FA;Z{eFh0d_}
z*WPy-%7^aoX4>~+dIrw>+90(=uEYyK=k9=a7t-PbW?kZq<IL#BnJrCbVUAqqLZnD#
zZ;anp2r7%Mm3k$ukISK*sIEDy&opQMkm~Y%LM%|)T)DQVb>Pa<;{GWliLc1mtM`Gj
zxm~`iO`x#S>W%;hYC_B)2=@)%RMdof(?;E^)ki82x=>QafOvZt$a_6wvpwKW66KTo
zl=lJ-S#js)spm=q1A8?-pZ9Y{`fEJ~C@N-F6wXg7wJG(%AE{k22hNcun4Zv8&V9tt
zr@$Xr;!uE6AF7gfH(rnIAXUA;KGR+$3MmbsikMGNUvu0P^bdLtpj|UXBf$cBt9D0D
z-<(`o>YwEYvP2n3A6!v{K2$-zH_(>h3K)uW;h4q_GTHQ*K~B!8!CAY<>$r!goJ2SA
zol40IMKjAgM_)sIx*r|T{<RF{7fY>OJXY@a#r+Efo9k(7>(dPBAqgWO1myJKz)+pB
z<hb;#uOR^efuHGv)*9=<`_4$jLQj-k7?vECZWxvxSp@bEK}UAZOn|@;IH6C!))*{y
z8u=D@Ucb>8Tq-j`cs_)9fgGq<oG1NLi}$@cUQ!p(1s;b(b-aXs5f;YV$JWE?nv;hF
z<~l~aRDTe9{><#pFx7k3iY_F_lm2nf$ny)Bv9_k6vDU-kX0JkZK**fxP#Co|SCl|k
zmY^KnJ4rg-aA&C=7#Fj2H}+mKORV0cX}{;UP4QsGC6$hCspopS7MY(g@1i&~Edy>;
zAc=-lksjf8DN%{Vvd*)%d|vJ%a<}_(mjmJs;~jldOI+!<iSAM#f-RJq6VKGVi4y47
zReEOqRcF`i1^QMlLQK^r-uS4sNDoiMlNntsJvpn*v*9wZv0#pQiMSTM6iMNM8s8l(
z)XXWX#CvYj^*q3lSE7R$@$e{lncU<jUDvo6P;=fmS`Y0<wMX7n4pF`8{8qC~EpB})
zHk2vsFvrJ_8m6@8<&?WvCCLs8R=OnxI=Cr9$yRpM1fR~iUe1BW1J^08s_%-N(k-iY
z^-WY~e;OEYbQ9co^`o<vSMeL+k9oOcvkyd8+$LBYMm9X`<2^y`%c{I%%b2Ocl=P>s
z7e0{fiDrp06*TUlBK@qyY|?#}A7Jd3z3hD6bch-uNJv@8{`pwu_&@!26@`9({LLZF
zp8<b2ru^%30H7#T>iD$h2jGvP<<Ee>8!LVQ+J6B4X~_69?C;vUAF$}-oc?z-zr+5h
zlmBk^eun*AW`Dr+Y5p1Zd(r)w_Os-F(CFy@nf99j{EYfpraw@{zd-#(v_B(%7VGcG
edXmjwApa#@BLu~9Bl6ufr~s&Av1c%TfBQc=vADJX
literal 0
HcmV?d00001
diff --git a/pythondeps.toml b/pythondeps.toml
index f6e590fdd8..0bc4144d4b 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -20,6 +20,7 @@
[meson]
# The install key should match the version in python/wheels/
meson = { accepted = ">=1.1.0", installed = "1.2.3", canary = "meson" }
+pycotap = { accepted = ">=1.1.0", installed = "1.3.1" }
[docs]
# Please keep the installed versions in sync with docs/requirements.txt
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 04/14] tests/functional: Add base classes for the upcoming pytest-based tests
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2024-07-29 15:27 ` [PULL 03/14] python: Install pycotap in our venv if necessary Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 05/14] tests/Makefile.include: Increase the level of indentation in the help text Philippe Mathieu-Daudé
` (10 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Thomas Huth, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py
file with some adjustments to get rid of the Avocado dependencies (i.e.
we also have to drop the LinuxSSHMixIn and LinuxTest for now).
The emulator binary and build directory are now passed via
environment variables that will be set via meson.build later.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-3-thuth@redhat.com>
[PMD: Updated MAINTAINERS]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
MAINTAINERS | 1 +
tests/functional/qemu_test/__init__.py | 328 +++++++++++++++++++++++++
2 files changed, 329 insertions(+)
create mode 100644 tests/functional/qemu_test/__init__.py
diff --git a/MAINTAINERS b/MAINTAINERS
index 98eddf7ae1..aa03a40d01 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4127,6 +4127,7 @@ F: .travis.yml
F: docs/devel/ci*
F: scripts/ci/
F: tests/docker/
+F: tests/functional/
F: tests/vm/
F: tests/lcitool/
F: tests/avocado/tuxrun_baselines.py
diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
new file mode 100644
index 0000000000..fc98222c52
--- /dev/null
+++ b/tests/functional/qemu_test/__init__.py
@@ -0,0 +1,328 @@
+# Test class and utilities for functional tests
+#
+# Copyright 2018, 2024 Red Hat, Inc.
+#
+# Original Author (Avocado-based tests):
+# Cleber Rosa <crosa@redhat.com>
+#
+# Adaption for standalone version:
+# Thomas Huth <thuth@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import logging
+import os
+import pycotap
+import shutil
+import subprocess
+import sys
+import tempfile
+import time
+import uuid
+import unittest
+
+from pathlib import Path
+from qemu.machine import QEMUMachine
+from qemu.utils import kvm_available, tcg_available
+
+def _source_dir():
+ # Determine top-level directory of the QEMU sources
+ return Path(__file__).parent.parent.parent.parent
+
+def _build_dir():
+ root = os.getenv('QEMU_BUILD_ROOT')
+ if root is not None:
+ return Path(root)
+ # Makefile.mtest only exists in build dir, so if it is available, use CWD
+ if os.path.exists('Makefile.mtest'):
+ return Path(os.getcwd())
+
+ root = os.path.join(_source_dir(), 'build')
+ if os.path.exists(root):
+ return Path(root)
+
+ raise Exception("Cannot identify build dir, set QEMU_BUILD_ROOT")
+
+BUILD_DIR = _build_dir()
+
+def has_cmd(name, args=None):
+ """
+ This function is for use in a @skipUnless decorator, e.g.:
+
+ @skipUnless(*has_cmd('sudo -n', ('sudo', '-n', 'true')))
+ def test_something_that_needs_sudo(self):
+ ...
+ """
+
+ if args is None:
+ args = ('which', name)
+
+ try:
+ _, stderr, exitcode = run_cmd(args)
+ except Exception as e:
+ exitcode = -1
+ stderr = str(e)
+
+ if exitcode != 0:
+ cmd_line = ' '.join(args)
+ err = f'{name} required, but "{cmd_line}" failed: {stderr.strip()}'
+ return (False, err)
+ else:
+ return (True, '')
+
+def has_cmds(*cmds):
+ """
+ This function is for use in a @skipUnless decorator and
+ allows checking for the availability of multiple commands, e.g.:
+
+ @skipUnless(*has_cmds(('cmd1', ('cmd1', '--some-parameter')),
+ 'cmd2', 'cmd3'))
+ def test_something_that_needs_cmd1_and_cmd2(self):
+ ...
+ """
+
+ for cmd in cmds:
+ if isinstance(cmd, str):
+ cmd = (cmd,)
+
+ ok, errstr = has_cmd(*cmd)
+ if not ok:
+ return (False, errstr)
+
+ return (True, '')
+
+def run_cmd(args):
+ subp = subprocess.Popen(args,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ universal_newlines=True)
+ stdout, stderr = subp.communicate()
+ ret = subp.returncode
+
+ return (stdout, stderr, ret)
+
+def is_readable_executable_file(path):
+ return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK)
+
+def _console_interaction(test, success_message, failure_message,
+ send_string, keep_sending=False, vm=None):
+ assert not keep_sending or send_string
+ if vm is None:
+ vm = test.vm
+ console = vm.console_file
+ console_logger = logging.getLogger('console')
+ while True:
+ if send_string:
+ vm.console_socket.sendall(send_string.encode())
+ if not keep_sending:
+ send_string = None # send only once
+ try:
+ msg = console.readline().decode().strip()
+ except UnicodeDecodeError:
+ msg = None
+ if not msg:
+ continue
+ console_logger.debug(msg)
+ if success_message is None or success_message in msg:
+ break
+ if failure_message and failure_message in msg:
+ console.close()
+ fail = 'Failure message found in console: "%s". Expected: "%s"' % \
+ (failure_message, success_message)
+ test.fail(fail)
+
+def interrupt_interactive_console_until_pattern(test, success_message,
+ failure_message=None,
+ interrupt_string='\r'):
+ """
+ Keep sending a string to interrupt a console prompt, while logging the
+ console output. Typical use case is to break a boot loader prompt, such:
+
+ Press a key within 5 seconds to interrupt boot process.
+ 5
+ 4
+ 3
+ 2
+ 1
+ Booting default image...
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ :param interrupt_string: a string to send to the console before trying
+ to read a new line
+ """
+ _console_interaction(test, success_message, failure_message,
+ interrupt_string, True)
+
+def wait_for_console_pattern(test, success_message, failure_message=None,
+ vm=None):
+ """
+ Waits for messages to appear on the console, while logging the content
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ """
+ _console_interaction(test, success_message, failure_message, None, vm=vm)
+
+def exec_command(test, command):
+ """
+ Send a command to a console (appending CRLF characters), while logging
+ the content.
+
+ :param test: a test containing a VM.
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param command: the command to send
+ :type command: str
+ """
+ _console_interaction(test, None, None, command + '\r')
+
+def exec_command_and_wait_for_pattern(test, command,
+ success_message, failure_message=None):
+ """
+ Send a command to a console (appending CRLF characters), then wait
+ for success_message to appear on the console, while logging the.
+ content. Mark the test as failed if failure_message is found instead.
+
+ :param test: a test containing a VM that will have its console
+ read and probed for a success or failure message
+ :type test: :class:`qemu_test.QemuSystemTest`
+ :param command: the command to send
+ :param success_message: if this message appears, test succeeds
+ :param failure_message: if this message appears, test fails
+ """
+ _console_interaction(test, success_message, failure_message, command + '\r')
+
+class QemuBaseTest(unittest.TestCase):
+
+ qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
+ arch = None
+
+ workdir = None
+ log = logging.getLogger('qemu-test')
+
+ def setUp(self, bin_prefix):
+ self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set')
+ self.arch = self.qemu_bin.split('-')[-1]
+
+ self.workdir = os.path.join(BUILD_DIR, 'tests/functional', self.arch,
+ self.id())
+ if not os.path.exists(self.workdir):
+ os.makedirs(self.workdir)
+
+ def main():
+ tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
+ test_output_log = pycotap.LogMode.LogToError)
+ path = os.path.basename(sys.argv[0])[:-3]
+ unittest.main(module = None, testRunner = tr, argv=["__dummy__", path])
+
+
+class QemuSystemTest(QemuBaseTest):
+ """Facilitates system emulation tests."""
+
+ cpu = None
+ machine = None
+ _machinehelp = None
+
+ def setUp(self):
+ self._vms = {}
+
+ super().setUp('qemu-system-')
+
+ def set_machine(self, machinename):
+ # TODO: We should use QMP to get the list of available machines
+ if not self._machinehelp:
+ self._machinehelp = run_cmd([self.qemu_bin, '-M', 'help'])[0];
+ if self._machinehelp.find(machinename) < 0:
+ self.skipTest('no support for machine ' + machinename)
+ self.machine = machinename
+
+ def require_accelerator(self, accelerator):
+ """
+ Requires an accelerator to be available for the test to continue
+
+ It takes into account the currently set qemu binary.
+
+ If the check fails, the test is canceled. If the check itself
+ for the given accelerator is not available, the test is also
+ canceled.
+
+ :param accelerator: name of the accelerator, such as "kvm" or "tcg"
+ :type accelerator: str
+ """
+ checker = {'tcg': tcg_available,
+ 'kvm': kvm_available}.get(accelerator)
+ if checker is None:
+ self.skipTest("Don't know how to check for the presence "
+ "of accelerator %s" % accelerator)
+ if not checker(qemu_bin=self.qemu_bin):
+ self.skipTest("%s accelerator does not seem to be "
+ "available" % accelerator)
+
+ def require_netdev(self, netdevname):
+ netdevhelp = run_cmd([self.qemu_bin,
+ '-M', 'none', '-netdev', 'help'])[0];
+ if netdevhelp.find('\n' + netdevname + '\n') < 0:
+ self.skipTest('no support for " + netdevname + " networking')
+
+ def require_device(self, devicename):
+ devhelp = run_cmd([self.qemu_bin,
+ '-M', 'none', '-device', 'help'])[0];
+ if devhelp.find(devicename) < 0:
+ self.skipTest('no support for device ' + devicename)
+
+ def _new_vm(self, name, *args):
+ vm = QEMUMachine(self.qemu_bin, base_temp_dir=self.workdir)
+ self.log.debug('QEMUMachine "%s" created', name)
+ self.log.debug('QEMUMachine "%s" temp_dir: %s', name, vm.temp_dir)
+ self.log.debug('QEMUMachine "%s" log_dir: %s', name, vm.log_dir)
+ if args:
+ vm.add_args(*args)
+ return vm
+
+ @property
+ def vm(self):
+ return self.get_vm(name='default')
+
+ def get_vm(self, *args, name=None):
+ if not name:
+ name = str(uuid.uuid4())
+ if self._vms.get(name) is None:
+ self._vms[name] = self._new_vm(name, *args)
+ if self.cpu is not None:
+ self._vms[name].add_args('-cpu', self.cpu)
+ if self.machine is not None:
+ self._vms[name].set_machine(self.machine)
+ return self._vms[name]
+
+ def set_vm_arg(self, arg, value):
+ """
+ Set an argument to list of extra arguments to be given to the QEMU
+ binary. If the argument already exists then its value is replaced.
+
+ :param arg: the QEMU argument, such as "-cpu" in "-cpu host"
+ :type arg: str
+ :param value: the argument value, such as "host" in "-cpu host"
+ :type value: str
+ """
+ if not arg or not value:
+ return
+ if arg not in self.vm.args:
+ self.vm.args.extend([arg, value])
+ else:
+ idx = self.vm.args.index(arg) + 1
+ if idx < len(self.vm.args):
+ self.vm.args[idx] = value
+ else:
+ self.vm.args.append(value)
+
+ def tearDown(self):
+ for vm in self._vms.values():
+ vm.shutdown()
+ super().tearDown()
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 05/14] tests/Makefile.include: Increase the level of indentation in the help text
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (3 preceding siblings ...)
2024-07-29 15:27 ` [PULL 04/14] tests/functional: Add base classes for the upcoming pytest-based tests Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 06/14] tests/functional: Prepare the meson build system for the functional tests Philippe Mathieu-Daudé
` (9 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Thomas Huth, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
The next patch is going to add some entries that need more space between
the command and the help text, so let's increase the indentation here
first.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-4-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
tests/Makefile.include | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6618bfed70..167d8f29a4 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -3,28 +3,28 @@
.PHONY: check-help
check-help:
@echo "Regression testing targets:"
- @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
- @echo " $(MAKE) bench Run speed tests"
+ @echo " $(MAKE) check Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
+ @echo " $(MAKE) bench Run speed tests"
@echo
@echo "Individual test suites:"
- @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
- @echo " $(MAKE) check-qtest Run qtest tests"
- @echo " $(MAKE) check-unit Run qobject tests"
- @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
- @echo " $(MAKE) check-block Run block tests"
+ @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
+ @echo " $(MAKE) check-qtest Run qtest tests"
+ @echo " $(MAKE) check-unit Run qobject tests"
+ @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
+ @echo " $(MAKE) check-block Run block tests"
ifneq ($(filter $(all-check-targets), check-softfloat),)
- @echo " $(MAKE) check-tcg Run TCG tests"
- @echo " $(MAKE) check-softfloat Run FPU emulation tests"
+ @echo " $(MAKE) check-tcg Run TCG tests"
+ @echo " $(MAKE) check-softfloat Run FPU emulation tests"
endif
- @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets"
+ @echo " $(MAKE) check-avocado Run avocado (integration) tests for currently configured targets"
@echo
- @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
- @echo " $(MAKE) check-venv Creates a Python venv for tests"
- @echo " $(MAKE) check-clean Clean the tests and related data"
+ @echo " $(MAKE) check-report.junit.xml Generates an aggregated XML test report"
+ @echo " $(MAKE) check-venv Creates a Python venv for tests"
+ @echo " $(MAKE) check-clean Clean the tests and related data"
@echo
@echo "The following are useful for CI builds"
- @echo " $(MAKE) check-build Build most test binaries"
- @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
+ @echo " $(MAKE) check-build Build most test binaries"
+ @echo " $(MAKE) get-vm-images Downloads all images used by avocado tests, according to configured targets (~350 MB each, 1.5 GB max)"
@echo
@echo
@echo "The variable SPEED can be set to control the gtester speed setting."
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 06/14] tests/functional: Prepare the meson build system for the functional tests
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (4 preceding siblings ...)
2024-07-29 15:27 ` [PULL 05/14] tests/Makefile.include: Increase the level of indentation in the help text Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 07/14] tests/functional: Convert simple avocado tests into standalone python tests Philippe Mathieu-Daudé
` (8 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Fabiano Rosas, Daniel P . Berrangé,
Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Provide a meson.build file for the upcoming python-based functional
tests, and add some wrapper glue targets to the tests/Makefile.include
file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"),
while the "thorough" tests should only be run when running a
"make check-functional" test run (since these tests might download
additional assets from the internet).
The changes to the meson.build files are partly based on an earlier
patch by Ani Sinha.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-5-thuth@redhat.com>
[PMD: Removed call to check-func target in check-functional rule]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
tests/Makefile.include | 11 ++++++
tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
tests/meson.build | 1 +
3 files changed, 78 insertions(+)
create mode 100644 tests/functional/meson.build
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 167d8f29a4..9077d5c6f0 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -9,6 +9,8 @@ check-help:
@echo "Individual test suites:"
@echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
@echo " $(MAKE) check-qtest Run qtest tests"
+ @echo " $(MAKE) check-functional Run python-based functional tests"
+ @echo " $(MAKE) check-functional-TARGET Run functional tests for a given target"
@echo " $(MAKE) check-unit Run qobject tests"
@echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
@echo " $(MAKE) check-block Run block tests"
@@ -152,6 +154,15 @@ check-acceptance-deprecated-warning:
check-acceptance: check-acceptance-deprecated-warning | check-avocado
+FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
+.PHONY: $(FUNCTIONAL_TARGETS)
+$(FUNCTIONAL_TARGETS):
+ @$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
+
+.PHONY: check-functional
+check-functional:
+ @$(MAKE) SPEED=thorough check-func-quick
+
# Consolidated targets
.PHONY: check check-clean get-vm-images
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
new file mode 100644
index 0000000000..7fc9c4e513
--- /dev/null
+++ b/tests/functional/meson.build
@@ -0,0 +1,66 @@
+# QEMU functional tests:
+# Tests that are put in the 'quick' category are run by default during
+# 'make check'. Everything that should not be run during 'make check'
+# (e.g. tests that fetch assets from the internet) should be put into
+# the 'thorough' category instead.
+
+# Most tests run too slow with TCI enabled, so skip the functional tests there
+if get_option('tcg_interpreter')
+ subdir_done()
+endif
+
+# Timeouts for individual tests that can be slow e.g. with debugging enabled
+test_timeouts = {
+}
+
+tests_generic = [
+]
+
+tests_x86_64_quick = [
+]
+
+tests_x86_64_thorough = [
+]
+
+foreach speed : ['quick', 'thorough']
+ foreach dir : target_dirs
+ if not dir.endswith('-softmmu')
+ continue
+ endif
+
+ target_base = dir.split('-')[0]
+ test_emulator = emulators['qemu-system-' + target_base]
+
+ if speed == 'quick'
+ suites = ['func-quick', 'func-' + target_base]
+ target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic
+ else
+ suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
+ target_tests = get_variable('tests_' + target_base + '_' + speed, [])
+ endif
+
+ test_deps = roms
+ test_env = environment()
+ if have_tools
+ test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img')
+ test_deps += [qemu_img]
+ endif
+ test_env.set('QEMU_TEST_QEMU_BINARY',
+ meson.global_build_root() / 'qemu-system-' + target_base)
+ test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())
+ test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
+ meson.current_source_dir())
+
+ foreach test : target_tests
+ test('func-@0@/@1@'.format(target_base, test),
+ python,
+ depends: [test_deps, test_emulator, emulator_modules],
+ env: test_env,
+ args: [meson.current_source_dir() / 'test_' + test + '.py'],
+ protocol: 'tap',
+ timeout: test_timeouts.get(test, 60),
+ priority: test_timeouts.get(test, 60),
+ suite: suites)
+ endforeach
+ endforeach
+endforeach
diff --git a/tests/meson.build b/tests/meson.build
index acb6807094..3345ad2098 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -85,3 +85,4 @@ subdir('unit')
subdir('qapi-schema')
subdir('qtest')
subdir('migration')
+subdir('functional')
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 07/14] tests/functional: Convert simple avocado tests into standalone python tests
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (5 preceding siblings ...)
2024-07-29 15:27 ` [PULL 06/14] tests/functional: Prepare the meson build system for the functional tests Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 08/14] tests/functional: Convert avocado tests that just need a small adjustment Philippe Mathieu-Daudé
` (7 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrangé,
Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
These test are rather simple and don't need any modifications apart
from adjusting the "from avocado_qemu" line. To ease debugging, make
the files executable and add a shebang line and Python '__main__'
handling, too, so that these tests can now be run by executing them
directly.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-6-thuth@redhat.com>
[PMD: Updated MAINTAINERS]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
MAINTAINERS | 5 ++
tests/functional/meson.build | 5 ++
.../test_cpu_queries.py} | 7 ++-
.../test_empty_cpu_model.py} | 7 ++-
.../test_mem_addr_space.py} | 52 +++----------------
.../test_pc_cpu_hotplug_props.py} | 11 ++--
.../test_virtio_version.py} | 8 +--
7 files changed, 39 insertions(+), 56 deletions(-)
rename tests/{avocado/cpu_queries.py => functional/test_cpu_queries.py} (89%)
mode change 100644 => 100755
rename tests/{avocado/empty_cpu_model.py => functional/test_empty_cpu_model.py} (84%)
mode change 100644 => 100755
rename tests/{avocado/mem-addr-space-check.py => functional/test_mem_addr_space.py} (93%)
mode change 100644 => 100755
rename tests/{avocado/pc_cpu_hotplug_props.py => functional/test_pc_cpu_hotplug_props.py} (90%)
mode change 100644 => 100755
rename tests/{avocado/virtio_version.py => functional/test_virtio_version.py} (98%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index aa03a40d01..1d84803b69 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1830,6 +1830,8 @@ F: hw/isa/apm.c
F: include/hw/isa/apm.h
F: tests/unit/test-x86-topo.c
F: tests/qtest/test-x86-cpuid-compat.c
+F: tests/functional/test_mem_addr_space.py
+F: tests/functional/test_pc_cpu_hotplug_props.py
PC Chipset
M: Michael S. Tsirkin <mst@redhat.com>
@@ -1896,6 +1898,8 @@ F: include/hw/boards.h
F: include/hw/core/cpu.h
F: include/hw/cpu/cluster.h
F: include/sysemu/numa.h
+F: tests/functional/test_cpu_queries.py
+F: tests/functional/test_empty_cpu_model.py
F: tests/unit/test-smp-parse.c
T: git https://gitlab.com/ehabkost/qemu.git machine-next
@@ -2236,6 +2240,7 @@ F: net/vhost-user.c
F: include/hw/virtio/
F: docs/devel/virtio*
F: docs/devel/migration/virtio.rst
+F: tests/functional/test_virtio_version.py
virtio-balloon
M: Michael S. Tsirkin <mst@redhat.com>
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 7fc9c4e513..a8cc0e6330 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -14,9 +14,14 @@ test_timeouts = {
}
tests_generic = [
+ 'empty_cpu_model',
]
tests_x86_64_quick = [
+ 'cpu_queries',
+ 'mem_addr_space',
+ 'pc_cpu_hotplug_props',
+ 'virtio_version',
]
tests_x86_64_thorough = [
diff --git a/tests/avocado/cpu_queries.py b/tests/functional/test_cpu_queries.py
old mode 100644
new mode 100755
similarity index 89%
rename from tests/avocado/cpu_queries.py
rename to tests/functional/test_cpu_queries.py
index d3faa14720..97dd600c01
--- a/tests/avocado/cpu_queries.py
+++ b/tests/functional/test_cpu_queries.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Sanity check of query-cpu-* results
#
# Copyright (c) 2019 Red Hat, Inc.
@@ -8,7 +10,7 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class QueryCPUModelExpansion(QemuSystemTest):
"""
@@ -33,3 +35,6 @@ def test(self):
e = self.vm.cmd('query-cpu-model-expansion', model=model,
type='full')
self.assertEqual(e['model']['name'], c['name'])
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/empty_cpu_model.py b/tests/functional/test_empty_cpu_model.py
old mode 100644
new mode 100755
similarity index 84%
rename from tests/avocado/empty_cpu_model.py
rename to tests/functional/test_empty_cpu_model.py
index d906ef3d3c..0081b06d85
--- a/tests/avocado/empty_cpu_model.py
+++ b/tests/functional/test_empty_cpu_model.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Check for crash when using empty -cpu option
#
# Copyright (c) 2019 Red Hat, Inc.
@@ -7,7 +9,7 @@
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class EmptyCPUModel(QemuSystemTest):
def test(self):
@@ -17,3 +19,6 @@ def test(self):
self.vm.wait()
self.assertEqual(self.vm.exitcode(), 1, "QEMU exit code should be 1")
self.assertRegex(self.vm.get_log(), r'-cpu option cannot be empty')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/mem-addr-space-check.py b/tests/functional/test_mem_addr_space.py
old mode 100644
new mode 100755
similarity index 93%
rename from tests/avocado/mem-addr-space-check.py
rename to tests/functional/test_mem_addr_space.py
index d3974599f4..bb0cf062ca
--- a/tests/avocado/mem-addr-space-check.py
+++ b/tests/functional/test_mem_addr_space.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Check for crash when using memory beyond the available guest processor
# address space.
#
@@ -8,7 +10,7 @@
#
# SPDX-License-Identifier: GPL-2.0-or-later
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
import time
class MemAddrCheck(QemuSystemTest):
@@ -22,9 +24,6 @@ class MemAddrCheck(QemuSystemTest):
# for all 32-bit cases, pci64_hole_size is 0.
def test_phybits_low_pse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
With pse36 feature ON, a processor has 36 bits of addressing. So it can
access up to a maximum of 64GiB of memory. Memory hotplug region begins
at 4 GiB boundary when "above_4g_mem_size" is 0 (this would be true when
@@ -52,9 +51,6 @@ def test_phybits_low_pse36(self):
def test_phybits_low_pae(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
With pae feature ON, a processor has 36 bits of addressing. So it can
access up to a maximum of 64GiB of memory. Rest is the same as the case
with pse36 above.
@@ -72,9 +68,6 @@ def test_phybits_low_pae(self):
def test_phybits_ok_pentium_pse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Setting maxmem to 59.5G and making sure that QEMU can start with the
same options as the failing case above with pse36 cpu feature.
"""
@@ -91,9 +84,6 @@ def test_phybits_ok_pentium_pse36(self):
def test_phybits_ok_pentium_pae(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Test is same as above but now with pae cpu feature turned on.
Setting maxmem to 59.5G and making sure that QEMU can start fine
with the same options as the case above.
@@ -111,9 +101,6 @@ def test_phybits_ok_pentium_pae(self):
def test_phybits_ok_pentium2(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Pentium2 has 36 bits of addressing, so its same as pentium
with pse36 ON.
"""
@@ -130,9 +117,6 @@ def test_phybits_ok_pentium2(self):
def test_phybits_low_nonpse36(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Pentium processor has 32 bits of addressing without pse36 or pae
so it can access physical address up to 4 GiB. Setting maxmem to
4 GiB should make QEMU fail to start with "phys-bits too low"
@@ -153,9 +137,6 @@ def test_phybits_low_nonpse36(self):
# now lets test some 64-bit CPU cases.
def test_phybits_low_tcg_q35_70_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
For q35 7.1 machines and above, there is a HT window that starts at
1024 GiB and ends at 1 TiB - 1. If the max GPA falls in this range,
"above_4G" memory is adjusted to start at 1 TiB boundary for AMD cpus
@@ -182,9 +163,6 @@ def test_phybits_low_tcg_q35_70_amd(self):
def test_phybits_low_tcg_q35_71_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD_HT_START is defined to be at 1012 GiB. So for q35 machines
version > 7.0 and AMD cpus, instead of 1024 GiB limit for 40 bit
processor address space, it has to be 1012 GiB , that is 12 GiB
@@ -205,9 +183,6 @@ def test_phybits_low_tcg_q35_71_amd(self):
def test_phybits_ok_tcg_q35_70_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as q35-7.0 AMD case except that here we check that QEMU can
successfully start when maxmem is < 988G.
"""
@@ -224,9 +199,6 @@ def test_phybits_ok_tcg_q35_70_amd(self):
def test_phybits_ok_tcg_q35_71_amd(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as q35-7.1 AMD case except that here we check that QEMU can
successfully start when maxmem is < 976G.
"""
@@ -243,9 +215,6 @@ def test_phybits_ok_tcg_q35_71_amd(self):
def test_phybits_ok_tcg_q35_71_intel(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same parameters as test_phybits_low_tcg_q35_71_amd() but use
Intel cpu instead. QEMU should start fine in this case as
"above_4G" memory starts at 4G.
@@ -264,9 +233,6 @@ def test_phybits_ok_tcg_q35_71_intel(self):
def test_phybits_low_tcg_q35_71_amd_41bits(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD processor with 41 bits. Max cpu hw address = 2 TiB.
By setting maxram above 1012 GiB - 32 GiB - 4 GiB = 976 GiB, we can
force "above_4G" memory to start at 1 TiB for q35-7.1 machines
@@ -291,9 +257,6 @@ def test_phybits_low_tcg_q35_71_amd_41bits(self):
def test_phybits_ok_tcg_q35_71_amd_41bits(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
AMD processor with 41 bits. Max cpu hw address = 2 TiB.
Same as above but by setting maxram between 976 GiB and 992 Gib,
QEMU should start fine.
@@ -312,9 +275,6 @@ def test_phybits_ok_tcg_q35_71_amd_41bits(self):
def test_phybits_low_tcg_q35_intel_cxl(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
cxl memory window starts after memory device range. Here, we use 1 GiB
of cxl window memory. 4G_mem end aligns at 4G. pci64_hole is 32 GiB and
starts after the cxl memory window.
@@ -335,9 +295,6 @@ def test_phybits_low_tcg_q35_intel_cxl(self):
def test_phybits_ok_tcg_q35_intel_cxl(self):
"""
- :avocado: tags=machine:q35
- :avocado: tags=arch:x86_64
-
Same as above but here we do not reserve any cxl memory window. Hence,
with the exact same parameters as above, QEMU should start fine even
with cxl enabled.
@@ -352,3 +309,6 @@ def test_phybits_ok_tcg_q35_intel_cxl(self):
time.sleep(self.DELAY_Q35_BOOT_SEQUENCE)
self.vm.shutdown()
self.assertNotRegex(self.vm.get_log(), r'phys-bits too low')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/pc_cpu_hotplug_props.py b/tests/functional/test_pc_cpu_hotplug_props.py
old mode 100644
new mode 100755
similarity index 90%
rename from tests/avocado/pc_cpu_hotplug_props.py
rename to tests/functional/test_pc_cpu_hotplug_props.py
index 4bd3e02665..9d5a37cb17
--- a/tests/avocado/pc_cpu_hotplug_props.py
+++ b/tests/functional/test_pc_cpu_hotplug_props.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# Ensure CPU die-id can be omitted on -device
#
@@ -20,16 +21,16 @@
# License along with this library; if not, see <http://www.gnu.org/licenses/>.
#
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class OmittedCPUProps(QemuSystemTest):
- """
- :avocado: tags=arch:x86_64
- :avocado: tags=cpu:qemu64
- """
+
def test_no_die_id(self):
self.vm.add_args('-nodefaults', '-S')
self.vm.add_args('-smp', '1,sockets=2,cores=2,threads=2,maxcpus=8')
self.vm.add_args('-device', 'qemu64-x86_64-cpu,socket-id=1,core-id=0,thread-id=0')
self.vm.launch()
self.assertEqual(len(self.vm.cmd('query-cpus-fast')), 2)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/virtio_version.py b/tests/functional/test_virtio_version.py
old mode 100644
new mode 100755
similarity index 98%
rename from tests/avocado/virtio_version.py
rename to tests/functional/test_virtio_version.py
index afe5e828b5..eb23060564
--- a/tests/avocado/virtio_version.py
+++ b/tests/functional/test_virtio_version.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
"""
Check compatibility of virtio device types
"""
@@ -12,7 +13,7 @@
import os
from qemu.machine import QEMUMachine
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
# Virtio Device IDs:
VIRTIO_NET = 1
@@ -60,8 +61,6 @@ class VirtioVersionCheck(QemuSystemTest):
Check if virtio-version-specific device types result in the
same device tree created by `disable-modern` and
`disable-legacy`.
-
- :avocado: tags=arch:x86_64
"""
# just in case there are failures, show larger diff:
@@ -173,3 +172,6 @@ def test_modern_only_devs(self):
self.check_modern_only('virtio-mouse-pci', VIRTIO_INPUT)
self.check_modern_only('virtio-tablet-pci', VIRTIO_INPUT)
self.check_modern_only('virtio-keyboard-pci', VIRTIO_INPUT)
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 08/14] tests/functional: Convert avocado tests that just need a small adjustment
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (6 preceding siblings ...)
2024-07-29 15:27 ` [PULL 07/14] tests/functional: Convert simple avocado tests into standalone python tests Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 09/14] tests/functional: Convert the x86_cpu_model_versions test Philippe Mathieu-Daudé
` (6 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrangé,
Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
These simple tests can be converted to stand-alone tests quite easily,
e.g. by just setting the machine to 'none' now manually or by adding
"-cpu" command line parameters, since we don't support the corresponding
avocado tags in the new python test framework.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-7-thuth@redhat.com>
[PMD: More uses of set_machine(), updated MAINTAINERS]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
MAINTAINERS | 3 +
tests/functional/meson.build | 6 ++
.../test_info_usernet.py} | 11 ++-
.../test_ppc_74xx.py} | 74 ++++++++-----------
.../version.py => functional/test_version.py} | 13 ++--
5 files changed, 56 insertions(+), 51 deletions(-)
rename tests/{avocado/info_usernet.py => functional/test_info_usernet.py} (87%)
mode change 100644 => 100755
rename tests/{avocado/ppc_74xx.py => functional/test_ppc_74xx.py} (74%)
mode change 100644 => 100755
rename tests/{avocado/version.py => functional/test_version.py} (78%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 1d84803b69..4c9a36dfd3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -317,6 +317,7 @@ F: configs/devices/ppc*
F: docs/system/ppc/embedded.rst
F: docs/system/target-ppc.rst
F: tests/tcg/ppc*/*
+F: tests/functional/test_ppc_74xx.py
RISC-V TCG CPUs
M: Palmer Dabbelt <palmer@dabbelt.com>
@@ -2105,6 +2106,7 @@ S: Odd Fixes
F: hw/net/
F: include/hw/net/
F: tests/qtest/virtio-net-test.c
+F: tests/avocado/info_usernet.py
F: docs/virtio-net-failover.rst
T: git https://github.com/jasowang/qemu.git net
@@ -2974,6 +2976,7 @@ S: Supported
F: include/qemu/option.h
F: tests/unit/test-keyval.c
F: tests/unit/test-qemu-opts.c
+F: tests/functional/test_version.py
F: util/keyval.c
F: util/qemu-option.c
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index a8cc0e6330..8a8fa0ab99 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -15,6 +15,12 @@ test_timeouts = {
tests_generic = [
'empty_cpu_model',
+ 'info_usernet',
+ 'version',
+]
+
+tests_ppc_quick = [
+ 'ppc_74xx',
]
tests_x86_64_quick = [
diff --git a/tests/avocado/info_usernet.py b/tests/functional/test_info_usernet.py
old mode 100644
new mode 100755
similarity index 87%
rename from tests/avocado/info_usernet.py
rename to tests/functional/test_info_usernet.py
index e1aa7a6e0a..cd37524d94
--- a/tests/avocado/info_usernet.py
+++ b/tests/functional/test_info_usernet.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Test for the hmp command "info usernet"
#
# Copyright (c) 2021 Red Hat, Inc.
@@ -8,18 +10,16 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
from qemu.utils import get_info_usernet_hostfwd_port
class InfoUsernet(QemuSystemTest):
- """
- :avocado: tags=machine:none
- """
def test_hostfwd(self):
self.require_netdev('user')
+ self.set_machine('none')
self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
@@ -31,3 +31,6 @@ def test_hostfwd(self):
self.assertGreater(port, 0,
('Found a redirected port that is not greater than'
' zero'))
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/ppc_74xx.py b/tests/functional/test_ppc_74xx.py
old mode 100644
new mode 100755
similarity index 74%
rename from tests/avocado/ppc_74xx.py
rename to tests/functional/test_ppc_74xx.py
index f54757c243..5386016f26
--- a/tests/avocado/ppc_74xx.py
+++ b/tests/functional/test_ppc_74xx.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Smoke tests for 74xx cpus (aka G4).
#
# Copyright (c) 2021, IBM Corp.
@@ -5,132 +7,120 @@
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
+from qemu_test import QemuSystemTest
+from qemu_test import wait_for_console_pattern
class ppc74xxCpu(QemuSystemTest):
- """
- :avocado: tags=arch:ppc
- :avocado: tags=accel:tcg
- """
+
timeout = 5
def test_ppc_7400(self):
- """
- :avocado: tags=cpu:7400
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7400')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7410(self):
- """
- :avocado: tags=cpu:7410
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7410')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,74xx')
def test_ppc_7441(self):
- """
- :avocado: tags=cpu:7441
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7441')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7445(self):
- """
- :avocado: tags=cpu:7445
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7445')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7447(self):
- """
- :avocado: tags=cpu:7447
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7447')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7447a(self):
- """
- :avocado: tags=cpu:7447a
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7447a')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7448(self):
- """
- :avocado: tags=cpu:7448
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7448')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,MPC86xx')
def test_ppc_7450(self):
- """
- :avocado: tags=cpu:7450
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7450')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7451(self):
- """
- :avocado: tags=cpu:7451
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7451')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7455(self):
- """
- :avocado: tags=cpu:7455
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7455')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7457(self):
- """
- :avocado: tags=cpu:7457
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7457')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
def test_ppc_7457a(self):
- """
- :avocado: tags=cpu:7457a
- """
self.require_accelerator("tcg")
+ self.set_machine('g3beige')
self.vm.set_console()
+ self.vm.add_args('-cpu', '7457a')
self.vm.launch()
wait_for_console_pattern(self, '>> OpenBIOS')
wait_for_console_pattern(self, '>> CPU type PowerPC,G4')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
diff --git a/tests/avocado/version.py b/tests/functional/test_version.py
old mode 100644
new mode 100755
similarity index 78%
rename from tests/avocado/version.py
rename to tests/functional/test_version.py
index c6139568a1..3ab3b67f7e
--- a/tests/avocado/version.py
+++ b/tests/functional/test_version.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
# Version check example test
#
# Copyright (c) 2018 Red Hat, Inc.
@@ -9,17 +11,18 @@
# later. See the COPYING file in the top-level directory.
-from avocado_qemu import QemuSystemTest
+from qemu_test import QemuSystemTest
class Version(QemuSystemTest):
- """
- :avocado: tags=quick
- :avocado: tags=machine:none
- """
+
def test_qmp_human_info_version(self):
+ self.set_machine('none')
self.vm.add_args('-nodefaults')
self.vm.launch()
res = self.vm.cmd('human-monitor-command',
command_line='info version')
self.assertRegex(res, r'^(\d+\.\d+\.\d)')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 09/14] tests/functional: Convert the x86_cpu_model_versions test
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (7 preceding siblings ...)
2024-07-29 15:27 ` [PULL 08/14] tests/functional: Convert avocado tests that just need a small adjustment Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 10/14] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Philippe Mathieu-Daudé
` (5 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Thomas Huth, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Nothing thrilling in here, it's just a straight forward conversion.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-14-thuth@redhat.com>
[PMD: More uses of set_machine(), updated MAINTAINERS]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
MAINTAINERS | 1 +
tests/functional/meson.build | 1 +
.../test_x86_cpu_model_versions.py} | 63 ++++++-------------
3 files changed, 21 insertions(+), 44 deletions(-)
rename tests/{avocado/x86_cpu_model_versions.py => functional/test_x86_cpu_model_versions.py} (92%)
mode change 100644 => 100755
diff --git a/MAINTAINERS b/MAINTAINERS
index 4c9a36dfd3..a768808a4a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1833,6 +1833,7 @@ F: tests/unit/test-x86-topo.c
F: tests/qtest/test-x86-cpuid-compat.c
F: tests/functional/test_mem_addr_space.py
F: tests/functional/test_pc_cpu_hotplug_props.py
+F: tests/functional/test_x86_cpu_model_versions.py
PC Chipset
M: Michael S. Tsirkin <mst@redhat.com>
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 8a8fa0ab99..48a617033e 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -28,6 +28,7 @@ tests_x86_64_quick = [
'mem_addr_space',
'pc_cpu_hotplug_props',
'virtio_version',
+ 'x86_cpu_model_versions',
]
tests_x86_64_thorough = [
diff --git a/tests/avocado/x86_cpu_model_versions.py b/tests/functional/test_x86_cpu_model_versions.py
old mode 100644
new mode 100755
similarity index 92%
rename from tests/avocado/x86_cpu_model_versions.py
rename to tests/functional/test_x86_cpu_model_versions.py
index 11101e02b9..a7294b4b92
--- a/tests/avocado/x86_cpu_model_versions.py
+++ b/tests/functional/test_x86_cpu_model_versions.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
#
# Basic validation of x86 versioned CPU models and CPU model aliases
#
@@ -20,11 +21,11 @@
# License along with this library; if not, see <http://www.gnu.org/licenses/>.
#
-
-import avocado_qemu
import re
-class X86CPUModelAliases(avocado_qemu.QemuSystemTest):
+from qemu_test import QemuSystemTest
+
+class X86CPUModelAliases(QemuSystemTest):
"""
Validation of PC CPU model versions and CPU model aliases
@@ -76,9 +77,8 @@ def validate_variant_aliases(self, cpus):
def test_4_0_alias_compatibility(self):
"""
Check if pc-*-4.0 unversioned CPU model won't be reported as aliases
-
- :avocado: tags=machine:pc-i440fx-4.0
"""
+ self.set_machine('pc-i440fx-4.0')
# pc-*-4.0 won't expose non-versioned CPU models as aliases
# We do this to help management software to keep compatibility
# with older QEMU versions that didn't have the versioned CPU model
@@ -110,9 +110,8 @@ def test_4_0_alias_compatibility(self):
def test_4_1_alias(self):
"""
Check if unversioned CPU model is an alias pointing to right version
-
- :avocado: tags=machine:pc-i440fx-4.1
"""
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.vm.launch()
@@ -217,9 +216,8 @@ def test_4_1_alias(self):
def test_none_alias(self):
"""
Check if unversioned CPU model is an alias pointing to some version
-
- :avocado: tags=machine:none
"""
+ self.set_machine('none')
self.vm.add_args('-S')
self.vm.launch()
@@ -243,21 +241,16 @@ def test_none_alias(self):
self.validate_aliases(cpus)
-class CascadelakeArchCapabilities(avocado_qemu.QemuSystemTest):
+class CascadelakeArchCapabilities(QemuSystemTest):
"""
Validation of Cascadelake arch-capabilities
-
- :avocado: tags=arch:x86_64
"""
def get_cpu_prop(self, prop):
cpu_path = self.vm.cmd('query-cpus-fast')[0].get('qom-path')
return self.vm.cmd('qom-get', path=cpu_path, property=prop)
def test_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
# machine-type only:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -268,10 +261,7 @@ def test_4_1(self):
'pc-i440fx-4.1 + Cascadelake-Server should not have arch-capabilities')
def test_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,'
@@ -281,10 +271,7 @@ def test_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server should not have arch-capabilities')
def test_set_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# command line must override machine-type if CPU model is not versioned:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -295,10 +282,7 @@ def test_set_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities should have arch-capabilities')
def test_unset_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,'
@@ -308,10 +292,7 @@ def test_unset_4_1(self):
'pc-i440fx-4.1 + Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
def test_v1_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# versioned CPU model overrides machine-type:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -322,10 +303,7 @@ def test_v1_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v1 should not have arch-capabilities')
def test_v2_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,'
@@ -335,10 +313,7 @@ def test_v2_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v2 should have arch-capabilities')
def test_v1_set_4_0(self):
- """
- :avocado: tags=machine:pc-i440fx-4.0
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.0')
# command line must override machine-type and versioned CPU model:
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
@@ -349,10 +324,7 @@ def test_v1_set_4_0(self):
'pc-i440fx-4.0 + Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
def test_v2_unset_4_1(self):
- """
- :avocado: tags=machine:pc-i440fx-4.1
- :avocado: tags=cpu:Cascadelake-Server
- """
+ self.set_machine('pc-i440fx-4.1')
self.vm.add_args('-S')
self.set_vm_arg('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,'
@@ -360,3 +332,6 @@ def test_v2_unset_4_1(self):
self.vm.launch()
self.assertFalse(self.get_cpu_prop('arch-capabilities'),
'pc-i440fx-4.1 + Cascadelake-Server-v2,-arch-capabilities should not have arch-capabilities')
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 10/14] tests/functional: Convert the riscv_opensbi avocado test into a standalone test
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (8 preceding siblings ...)
2024-07-29 15:27 ` [PULL 09/14] tests/functional: Convert the x86_cpu_model_versions test Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 11/14] gitlab-ci: Add "check-functional" to the build tests Philippe Mathieu-Daudé
` (4 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Thomas Huth, Alistair Francis, Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
The avocado test defined test functions for both, riscv32 and riscv64.
Since we can run the whole file with multiple targets in the new
framework, we can now consolidate the functions so we have to only
define one function per machine now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240724175248.1389201-16-thuth@redhat.com>
[PMD: Updated MAINTAINERS]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
MAINTAINERS | 1 +
tests/avocado/riscv_opensbi.py | 63 --------------------------
tests/functional/meson.build | 8 ++++
tests/functional/test_riscv_opensbi.py | 36 +++++++++++++++
4 files changed, 45 insertions(+), 63 deletions(-)
delete mode 100644 tests/avocado/riscv_opensbi.py
create mode 100755 tests/functional/test_riscv_opensbi.py
diff --git a/MAINTAINERS b/MAINTAINERS
index a768808a4a..a906218f9d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -337,6 +337,7 @@ F: include/hw/riscv/
F: linux-user/host/riscv32/
F: linux-user/host/riscv64/
F: tests/tcg/riscv64/
+F: tests/functional/test_riscv_opensbi.py
RISC-V XThead* extensions
M: Christoph Muellner <christoph.muellner@vrull.eu>
diff --git a/tests/avocado/riscv_opensbi.py b/tests/avocado/riscv_opensbi.py
deleted file mode 100644
index bfff9cc3c3..0000000000
--- a/tests/avocado/riscv_opensbi.py
+++ /dev/null
@@ -1,63 +0,0 @@
-# OpenSBI boot test for RISC-V machines
-#
-# Copyright (c) 2022, Ventana Micro
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or
-# later. See the COPYING file in the top-level directory.
-
-from avocado_qemu import QemuSystemTest
-from avocado_qemu import wait_for_console_pattern
-
-class RiscvOpenSBI(QemuSystemTest):
- """
- :avocado: tags=accel:tcg
- """
- timeout = 5
-
- def boot_opensbi(self):
- self.vm.set_console()
- self.vm.launch()
- wait_for_console_pattern(self, 'Platform Name')
- wait_for_console_pattern(self, 'Boot HART MEDELEG')
-
- def test_riscv32_spike(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:spike
- """
- self.boot_opensbi()
-
- def test_riscv64_spike(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:spike
- """
- self.boot_opensbi()
-
- def test_riscv32_sifive_u(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:sifive_u
- """
- self.boot_opensbi()
-
- def test_riscv64_sifive_u(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:sifive_u
- """
- self.boot_opensbi()
-
- def test_riscv32_virt(self):
- """
- :avocado: tags=arch:riscv32
- :avocado: tags=machine:virt
- """
- self.boot_opensbi()
-
- def test_riscv64_virt(self):
- """
- :avocado: tags=arch:riscv64
- :avocado: tags=machine:virt
- """
- self.boot_opensbi()
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 48a617033e..94ac97ac53 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -23,6 +23,14 @@ tests_ppc_quick = [
'ppc_74xx',
]
+tests_riscv32_quick = [
+ 'riscv_opensbi',
+]
+
+tests_riscv64_quick = [
+ 'riscv_opensbi',
+]
+
tests_x86_64_quick = [
'cpu_queries',
'mem_addr_space',
diff --git a/tests/functional/test_riscv_opensbi.py b/tests/functional/test_riscv_opensbi.py
new file mode 100755
index 0000000000..d077e40f42
--- /dev/null
+++ b/tests/functional/test_riscv_opensbi.py
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+#
+# OpenSBI boot test for RISC-V machines
+#
+# Copyright (c) 2022, Ventana Micro
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+from qemu_test import QemuSystemTest
+from qemu_test import wait_for_console_pattern
+
+class RiscvOpenSBI(QemuSystemTest):
+
+ timeout = 5
+
+ def boot_opensbi(self):
+ self.vm.set_console()
+ self.vm.launch()
+ wait_for_console_pattern(self, 'Platform Name')
+ wait_for_console_pattern(self, 'Boot HART MEDELEG')
+
+ def test_riscv_spike(self):
+ self.set_machine('spike')
+ self.boot_opensbi()
+
+ def test_riscv_sifive_u(self):
+ self.set_machine('sifive_u')
+ self.boot_opensbi()
+
+ def test_riscv_virt(self):
+ self.set_machine('virt')
+ self.boot_opensbi()
+
+if __name__ == '__main__':
+ QemuSystemTest.main()
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 11/14] gitlab-ci: Add "check-functional" to the build tests
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (9 preceding siblings ...)
2024-07-29 15:27 ` [PULL 10/14] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 12/14] tests/avocado: mips: fallback to HTTP given certificate expiration Philippe Mathieu-Daudé
` (3 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrangé,
Philippe Mathieu-Daudé
From: Thomas Huth <thuth@redhat.com>
Now that we converted many tests from the "check-avocado" test suite
to the "check-functional" test suite, we should make sure that these
also get tested in the CI.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240724175248.1389201-24-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
.gitlab-ci.d/buildtest-template.yml | 3 +-
.gitlab-ci.d/buildtest.yml | 60 ++++++++++++++---------------
2 files changed, 32 insertions(+), 31 deletions(-)
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 8f7ebfaed8..54cae74a8e 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -71,7 +71,7 @@
reports:
junit: build/meson-logs/testlog.junit.xml
-.avocado_test_job_template:
+.functional_test_job_template:
extends: .common_test_job_template
cache:
key: "${CI_JOB_NAME}-cache"
@@ -98,6 +98,7 @@
du -chs ${CI_PROJECT_DIR}/avocado-cache ;
fi
- export AVOCADO_ALLOW_UNTRUSTED_CODE=1
+ - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
after_script:
- cd build
- du -chs ${CI_PROJECT_DIR}/avocado-cache
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index aa32782405..1d2afae996 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -22,14 +22,14 @@ check-system-alpine:
IMAGE: alpine
MAKE_CHECK_ARGS: check-unit check-qtest
-avocado-system-alpine:
- extends: .avocado_test_job_template
+functional-system-alpine:
+ extends: .functional_test_job_template
needs:
- job: build-system-alpine
artifacts: true
variables:
IMAGE: alpine
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:avr arch:loongarch64 arch:mips64 arch:mipsel
build-system-ubuntu:
@@ -53,14 +53,14 @@ check-system-ubuntu:
IMAGE: ubuntu2204
MAKE_CHECK_ARGS: check
-avocado-system-ubuntu:
- extends: .avocado_test_job_template
+functional-system-ubuntu:
+ extends: .functional_test_job_template
needs:
- job: build-system-ubuntu
artifacts: true
variables:
IMAGE: ubuntu2204
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:alpha arch:microblazeel arch:mips64el
build-system-debian:
@@ -85,14 +85,14 @@ check-system-debian:
IMAGE: debian
MAKE_CHECK_ARGS: check
-avocado-system-debian:
- extends: .avocado_test_job_template
+functional-system-debian:
+ extends: .functional_test_job_template
needs:
- job: build-system-debian
artifacts: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:arm arch:i386 arch:riscv64 arch:sh4 arch:sparc arch:xtensa
crash-test-debian:
@@ -129,14 +129,14 @@ check-system-fedora:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-system-fedora:
- extends: .avocado_test_job_template
+functional-system-fedora:
+ extends: .functional_test_job_template
needs:
- job: build-system-fedora
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:microblaze arch:mips arch:xtensa arch:m68k
arch:riscv32 arch:ppc arch:sparc64
@@ -243,14 +243,14 @@ check-system-centos:
IMAGE: centos9
MAKE_CHECK_ARGS: check
-avocado-system-centos:
- extends: .avocado_test_job_template
+functional-system-centos:
+ extends: .functional_test_job_template
needs:
- job: build-system-centos
artifacts: true
variables:
IMAGE: centos9
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:ppc64 arch:or1k arch:s390x arch:x86_64 arch:rx
arch:sh4
@@ -274,14 +274,14 @@ check-system-opensuse:
IMAGE: opensuse-leap
MAKE_CHECK_ARGS: check
-avocado-system-opensuse:
- extends: .avocado_test_job_template
+functional-system-opensuse:
+ extends: .functional_test_job_template
needs:
- job: build-system-opensuse
artifacts: true
variables:
IMAGE: opensuse-leap
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
AVOCADO_TAGS: arch:s390x arch:x86_64 arch:aarch64
#
@@ -302,15 +302,15 @@ build-system-flaky:
ppc64-softmmu rx-softmmu s390x-softmmu sh4-softmmu x86_64-softmmu
MAKE_CHECK_ARGS: check-build
-avocado-system-flaky:
- extends: .avocado_test_job_template
+functional-system-flaky:
+ extends: .functional_test_job_template
needs:
- job: build-system-flaky
artifacts: true
allow_failure: true
variables:
IMAGE: debian
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
QEMU_JOB_OPTIONAL: 1
QEMU_TEST_FLAKY_TESTS: 1
AVOCADO_TAGS: flaky
@@ -485,14 +485,14 @@ check-cfi-aarch64:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-aarch64:
- extends: .avocado_test_job_template
+functional-cfi-aarch64:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-aarch64
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
build-cfi-ppc64-s390x:
extends:
@@ -523,14 +523,14 @@ check-cfi-ppc64-s390x:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-ppc64-s390x:
- extends: .avocado_test_job_template
+functional-cfi-ppc64-s390x:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-ppc64-s390x
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
build-cfi-x86_64:
extends:
@@ -557,14 +557,14 @@ check-cfi-x86_64:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-avocado-cfi-x86_64:
- extends: .avocado_test_job_template
+functional-cfi-x86_64:
+ extends: .functional_test_job_template
needs:
- job: build-cfi-x86_64
artifacts: true
variables:
IMAGE: fedora
- MAKE_CHECK_ARGS: check-avocado
+ MAKE_CHECK_ARGS: check-avocado check-functional
tsan-build:
extends: .native_build_job_template
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 12/14] tests/avocado: mips: fallback to HTTP given certificate expiration
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (10 preceding siblings ...)
2024-07-29 15:27 ` [PULL 11/14] gitlab-ci: Add "check-functional" to the build tests Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 13/14] tests/avocado: mips: add hint for fetchasset plugin Philippe Mathieu-Daudé
` (2 subsequent siblings)
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel
Cc: Cleber Rosa, Alex Bennée, Akihiko Odaki,
Philippe Mathieu-Daudé
From: Cleber Rosa <crosa@redhat.com>
The SSL certificate installed at mipsdistros.mips.com has expired:
0 s:CN = mipsdistros.mips.com
i:C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Dec 23 00:00:00 2019 GMT; NotAfter: Jan 23 12:00:00 2021 GMT
Because this project has no control over that certificate and host,
this falls back to plain HTTP instead. The integrity of the
downloaded files can be guaranteed by the existing hashes for those
files (which are not modified here).
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240726134438.14720-2-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
tests/avocado/boot_linux_console.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index c35fc5e9ba..450d67be6a 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -299,7 +299,7 @@ def test_mips_malta32el_nanomips_4k(self):
:avocado: tags=endian:little
:avocado: tags=cpu:I7200
"""
- kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
+ kernel_url = ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
'generic_nano32r6el_page4k.xz')
kernel_hash = '477456aafd2a0f1ddc9482727f20fe9575565dd6'
@@ -312,7 +312,7 @@ def test_mips_malta32el_nanomips_16k_up(self):
:avocado: tags=endian:little
:avocado: tags=cpu:I7200
"""
- kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
+ kernel_url = ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
'generic_nano32r6el_page16k_up.xz')
kernel_hash = 'e882868f944c71c816e832e2303b7874d044a7bc'
@@ -325,7 +325,7 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
:avocado: tags=endian:little
:avocado: tags=cpu:I7200
"""
- kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
+ kernel_url = ('http://mipsdistros.mips.com/LinuxDistro/nanomips/'
'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
'generic_nano32r6el_page64k_dbg.xz')
kernel_hash = '18d1c68f2e23429e266ca39ba5349ccd0aeb7180'
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 13/14] tests/avocado: mips: add hint for fetchasset plugin
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (11 preceding siblings ...)
2024-07-29 15:27 ` [PULL 12/14] tests/avocado: mips: fallback to HTTP given certificate expiration Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 14/14] tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset Philippe Mathieu-Daudé
2024-07-29 15:36 ` [PULL 00/14] Docs / testing patches for 2024-07-29 Daniel P. Berrangé
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel; +Cc: Cleber Rosa, Akihiko Odaki, Philippe Mathieu-Daudé
From: Cleber Rosa <crosa@redhat.com>
Avocado's fetchasset plugin runs before the actual Avocado job (and
any test). It analyses the test's code looking for occurrences of
"self.fetch_asset()" in the either the actual test or setUp() method.
It's not able to fully analyze all code, though.
The way these tests are written, make the fetchasset plugin blind to
the assets. This adds some more code duplication, true, but it will
aid the fetchasset plugin to download or verify the existence of these
assets in advance.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240726134438.14720-3-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
tests/avocado/boot_linux_console.py | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index 450d67be6a..b8b0a4df10 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -274,8 +274,7 @@ def test_mips64el_malta_5KEc_cpio(self):
# Wait for VM to shut down gracefully
self.vm.wait()
- def do_test_mips_malta32el_nanomips(self, kernel_url, kernel_hash):
- kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ def do_test_mips_malta32el_nanomips(self, kernel_path_xz):
kernel_path = self.workdir + "kernel"
with lzma.open(kernel_path_xz, 'rb') as f_in:
with open(kernel_path, 'wb') as f_out:
@@ -303,7 +302,8 @@ def test_mips_malta32el_nanomips_4k(self):
'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
'generic_nano32r6el_page4k.xz')
kernel_hash = '477456aafd2a0f1ddc9482727f20fe9575565dd6'
- self.do_test_mips_malta32el_nanomips(kernel_url, kernel_hash)
+ kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.do_test_mips_malta32el_nanomips(kernel_path_xz)
def test_mips_malta32el_nanomips_16k_up(self):
"""
@@ -316,7 +316,8 @@ def test_mips_malta32el_nanomips_16k_up(self):
'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
'generic_nano32r6el_page16k_up.xz')
kernel_hash = 'e882868f944c71c816e832e2303b7874d044a7bc'
- self.do_test_mips_malta32el_nanomips(kernel_url, kernel_hash)
+ kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.do_test_mips_malta32el_nanomips(kernel_path_xz)
def test_mips_malta32el_nanomips_64k_dbg(self):
"""
@@ -329,7 +330,8 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
'generic_nano32r6el_page64k_dbg.xz')
kernel_hash = '18d1c68f2e23429e266ca39ba5349ccd0aeb7180'
- self.do_test_mips_malta32el_nanomips(kernel_url, kernel_hash)
+ kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+ self.do_test_mips_malta32el_nanomips(kernel_path_xz)
def test_aarch64_xlnx_versal_virt(self):
"""
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PULL 14/14] tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (12 preceding siblings ...)
2024-07-29 15:27 ` [PULL 13/14] tests/avocado: mips: add hint for fetchasset plugin Philippe Mathieu-Daudé
@ 2024-07-29 15:27 ` Philippe Mathieu-Daudé
2024-07-29 15:36 ` [PULL 00/14] Docs / testing patches for 2024-07-29 Daniel P. Berrangé
14 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-29 15:27 UTC (permalink / raw)
To: qemu-devel
Cc: Cleber Rosa, Akihiko Odaki, Daniel P . Berrangé,
Philippe Mathieu-Daudé
From: Cleber Rosa <crosa@redhat.com>
The asset used in the mentioned test gets truncated before it's used
in the test. This means that the file gets modified, and thus the
asset's expected hash doesn't match anymore. This causes cache misses
and re-downloads every time the test is re-run.
Let's make a copy of the asset so that the one in the cache is
preserved and the cache sees a hit on re-runs.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240726134438.14720-9-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
tests/avocado/boot_linux_console.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index b8b0a4df10..2929aa042d 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -401,14 +401,16 @@ def test_arm_emcraft_sf2(self):
'fe371d32e50ca682391e1e70ab98c2942aeffb01/spi.bin')
spi_hash = '65523a1835949b6f4553be96dec1b6a38fb05501'
spi_path = self.fetch_asset(spi_url, asset_hash=spi_hash)
+ spi_path_rw = os.path.join(self.workdir, os.path.basename(spi_path))
+ shutil.copy(spi_path, spi_path_rw)
- file_truncate(spi_path, 16 << 20) # Spansion S25FL128SDPBHICO is 16 MiB
+ file_truncate(spi_path_rw, 16 << 20) # Spansion S25FL128SDPBHICO is 16 MiB
self.vm.set_console()
kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE
self.vm.add_args('-kernel', uboot_path,
'-append', kernel_command_line,
- '-drive', 'file=' + spi_path + ',if=mtd,format=raw',
+ '-drive', 'file=' + spi_path_rw + ',if=mtd,format=raw',
'-no-reboot')
self.vm.launch()
self.wait_for_console_pattern('Enter \'help\' for a list')
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PULL 00/14] Docs / testing patches for 2024-07-29
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
` (13 preceding siblings ...)
2024-07-29 15:27 ` [PULL 14/14] tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset Philippe Mathieu-Daudé
@ 2024-07-29 15:36 ` Daniel P. Berrangé
2024-07-30 10:04 ` Philippe Mathieu-Daudé
14 siblings, 1 reply; 19+ messages in thread
From: Daniel P. Berrangé @ 2024-07-29 15:36 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: qemu-devel
On Mon, Jul 29, 2024 at 05:27:00PM +0200, Philippe Mathieu-Daudé wrote:
> The following changes since commit 93b799fafd9170da3a79a533ea6f73a18de82e22:
>
> Merge tag 'pull-ppc-for-9.1-2-20240726-1' of https://gitlab.com/npiggin/qemu into staging (2024-07-26 15:10:45 +1000)
>
> are available in the Git repository at:
>
> https://github.com/philmd/qemu.git tags/docs-testing-20240729
>
> for you to fetch changes up to 8e2275592299b637e87cc905d20bfdeffa586a83:
>
> tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset (2024-07-29 16:33:24 +0200)
>
> Ignored warnings:
>
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> WARNING: line over 80 characters
>
> ----------------------------------------------------------------
> Docs & testing patch queue
>
> - Test QAPI firmware.json schema (Thomas)
> - Handle new env.doc2path() return value (Peter)
> - Convert some Avocado tests to simpler python-based functional testing (Thomas)
> - Improve how assets are used by some Avocado tests (Cleber)
>
> ----------------------------------------------------------------
>
> Cleber Rosa (3):
> tests/avocado: mips: fallback to HTTP given certificate expiration
> tests/avocado: mips: add hint for fetchasset plugin
> tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset
>
> Peter Maydell (1):
> docs/sphinx/depfile.py: Handle env.doc2path() returning a Path not a
> str
>
> Thomas Huth (9):
> python: Install pycotap in our venv if necessary
> tests/functional: Add base classes for the upcoming pytest-based tests
> tests/Makefile.include: Increase the level of indentation in the help
> text
> tests/functional: Prepare the meson build system for the functional
> tests
> tests/functional: Convert simple avocado tests into standalone python
> tests
> tests/functional: Convert avocado tests that just need a small
> adjustment
> tests/functional: Convert the x86_cpu_model_versions test
> tests/functional: Convert the riscv_opensbi avocado test into a
> standalone test
> gitlab-ci: Add "check-functional" to the build tests
On the avocado review I mentioned that I didn't think we should
be rebasing avocado during freeze. By the same token, I'm not
convinced we should be introducing a new test framework during
the freeze period. This is alot simpler than avocado, but at
the same time this small subset of Thomas' patches isn't really
fixing any problem on its own, as avocado still exists in
parallel.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PULL 00/14] Docs / testing patches for 2024-07-29
2024-07-29 15:36 ` [PULL 00/14] Docs / testing patches for 2024-07-29 Daniel P. Berrangé
@ 2024-07-30 10:04 ` Philippe Mathieu-Daudé
2024-07-30 10:14 ` Peter Maydell
0 siblings, 1 reply; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-30 10:04 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel
On 29/7/24 17:36, Daniel P. Berrangé wrote:
> On Mon, Jul 29, 2024 at 05:27:00PM +0200, Philippe Mathieu-Daudé wrote:
>> Thomas Huth (9):
>> python: Install pycotap in our venv if necessary
>> tests/functional: Add base classes for the upcoming pytest-based tests
>> tests/Makefile.include: Increase the level of indentation in the help
>> text
>> tests/functional: Prepare the meson build system for the functional
>> tests
>> tests/functional: Convert simple avocado tests into standalone python
>> tests
>> tests/functional: Convert avocado tests that just need a small
>> adjustment
>> tests/functional: Convert the x86_cpu_model_versions test
>> tests/functional: Convert the riscv_opensbi avocado test into a
>> standalone test
>> gitlab-ci: Add "check-functional" to the build tests
>
> On the avocado review I mentioned that I didn't think we should
> be rebasing avocado during freeze. By the same token, I'm not
> convinced we should be introducing a new test framework during
> the freeze period. This is alot simpler than avocado, but at
> the same time this small subset of Thomas' patches isn't really
> fixing any problem on its own, as avocado still exists in
> parallel.
- As you noticed, Avocado isn't rebased here
- The python file added is almost a copy of the avocado base class,
already well tested
- Test are moved from Avocado, so no waste of time running them
twice
- macOS developers can run 178 functional tests. This is the big
win and time saver for me.
Do you really see merging these few patches now as being
counter productive for the community?
Regards,
Phil.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PULL 00/14] Docs / testing patches for 2024-07-29
2024-07-30 10:04 ` Philippe Mathieu-Daudé
@ 2024-07-30 10:14 ` Peter Maydell
2024-07-30 18:31 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 19+ messages in thread
From: Peter Maydell @ 2024-07-30 10:14 UTC (permalink / raw)
To: Philippe Mathieu-Daudé; +Cc: Daniel P. Berrangé, qemu-devel
On Tue, 30 Jul 2024 at 11:05, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>
> On 29/7/24 17:36, Daniel P. Berrangé wrote:
> > On Mon, Jul 29, 2024 at 05:27:00PM +0200, Philippe Mathieu-Daudé wrote:
>
> >> Thomas Huth (9):
> >> python: Install pycotap in our venv if necessary
> >> tests/functional: Add base classes for the upcoming pytest-based tests
> >> tests/Makefile.include: Increase the level of indentation in the help
> >> text
> >> tests/functional: Prepare the meson build system for the functional
> >> tests
> >> tests/functional: Convert simple avocado tests into standalone python
> >> tests
> >> tests/functional: Convert avocado tests that just need a small
> >> adjustment
> >> tests/functional: Convert the x86_cpu_model_versions test
> >> tests/functional: Convert the riscv_opensbi avocado test into a
> >> standalone test
> >> gitlab-ci: Add "check-functional" to the build tests
> >
> > On the avocado review I mentioned that I didn't think we should
> > be rebasing avocado during freeze. By the same token, I'm not
> > convinced we should be introducing a new test framework during
> > the freeze period. This is alot simpler than avocado, but at
> > the same time this small subset of Thomas' patches isn't really
> > fixing any problem on its own, as avocado still exists in
> > parallel.
>
> - As you noticed, Avocado isn't rebased here
> - The python file added is almost a copy of the avocado base class,
> already well tested
> - Test are moved from Avocado, so no waste of time running them
> twice
> - macOS developers can run 178 functional tests. This is the big
> win and time saver for me.
>
> Do you really see merging these few patches now as being
> counter productive for the community?
I do feel from the discussion in other threads that "what is
the way forward for our currently-in-avocado test cases?" is
currently still an open question, so I tend to agree with Daniel
that it would be better to resolve that question first,
and we don't need to be in a hurry to change things in
our test framework during the freeze period.
thanks
-- PMM
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PULL 00/14] Docs / testing patches for 2024-07-29
2024-07-30 10:14 ` Peter Maydell
@ 2024-07-30 18:31 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2024-07-30 18:31 UTC (permalink / raw)
To: Peter Maydell
Cc: Daniel P. Berrangé, qemu-devel, Thomas Huth,
Richard Henderson
On 30/7/24 12:14, Peter Maydell wrote:
> On Tue, 30 Jul 2024 at 11:05, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>>
>> On 29/7/24 17:36, Daniel P. Berrangé wrote:
>>> On Mon, Jul 29, 2024 at 05:27:00PM +0200, Philippe Mathieu-Daudé wrote:
>>
>>>> Thomas Huth (9):
>>>> python: Install pycotap in our venv if necessary
>>>> tests/functional: Add base classes for the upcoming pytest-based tests
>>>> tests/Makefile.include: Increase the level of indentation in the help
>>>> text
>>>> tests/functional: Prepare the meson build system for the functional
>>>> tests
>>>> tests/functional: Convert simple avocado tests into standalone python
>>>> tests
>>>> tests/functional: Convert avocado tests that just need a small
>>>> adjustment
>>>> tests/functional: Convert the x86_cpu_model_versions test
>>>> tests/functional: Convert the riscv_opensbi avocado test into a
>>>> standalone test
>>>> gitlab-ci: Add "check-functional" to the build tests
>>>
>>> On the avocado review I mentioned that I didn't think we should
>>> be rebasing avocado during freeze. By the same token, I'm not
>>> convinced we should be introducing a new test framework during
>>> the freeze period. This is alot simpler than avocado, but at
>>> the same time this small subset of Thomas' patches isn't really
>>> fixing any problem on its own, as avocado still exists in
>>> parallel.
>>
>> - As you noticed, Avocado isn't rebased here
>> - The python file added is almost a copy of the avocado base class,
>> already well tested
>> - Test are moved from Avocado, so no waste of time running them
>> twice
>> - macOS developers can run 178 functional tests. This is the big
>> win and time saver for me.
>>
>> Do you really see merging these few patches now as being
>> counter productive for the community?
>
> I do feel from the discussion in other threads that "what is
> the way forward for our currently-in-avocado test cases?" is
> currently still an open question,
Thomas demonstrated with few LoC we can leverage some Avocado
libraries (util, archive, ...) without having to use Avocado
framework as our entry point to run acceptance tests - which
is what gave us headaches the last 2 years -.
Richard only requirement is a "fetch assets" equivalent. We'll
look at.
I don't see anything else blocking this test conversion.
> so I tend to agree with Daniel
> that it would be better to resolve that question first,
> and we don't need to be in a hurry to change things in
> our test framework during the freeze period.
OK. I'll repost without these patches.
Regards,
Phil.
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2024-07-30 18:32 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-29 15:27 [PULL 00/14] Docs / testing patches for 2024-07-29 Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 01/14] docs: add test for firmware.json QAPI Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 02/14] docs/sphinx/depfile.py: Handle env.doc2path() returning a Path not a str Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 03/14] python: Install pycotap in our venv if necessary Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 04/14] tests/functional: Add base classes for the upcoming pytest-based tests Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 05/14] tests/Makefile.include: Increase the level of indentation in the help text Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 06/14] tests/functional: Prepare the meson build system for the functional tests Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 07/14] tests/functional: Convert simple avocado tests into standalone python tests Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 08/14] tests/functional: Convert avocado tests that just need a small adjustment Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 09/14] tests/functional: Convert the x86_cpu_model_versions test Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 10/14] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 11/14] gitlab-ci: Add "check-functional" to the build tests Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 12/14] tests/avocado: mips: fallback to HTTP given certificate expiration Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 13/14] tests/avocado: mips: add hint for fetchasset plugin Philippe Mathieu-Daudé
2024-07-29 15:27 ` [PULL 14/14] tests/avocado: test_arm_emcraft_sf2: handle RW requirements for asset Philippe Mathieu-Daudé
2024-07-29 15:36 ` [PULL 00/14] Docs / testing patches for 2024-07-29 Daniel P. Berrangé
2024-07-30 10:04 ` Philippe Mathieu-Daudé
2024-07-30 10:14 ` Peter Maydell
2024-07-30 18:31 ` Philippe Mathieu-Daudé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).