Re: [PATCH] meson: use thorough test setup as default

[Thomas Huth <thuth@redhat.com>]

On 07/05/2025 20.45, Pierrick Bouvier wrote:
> On 5/7/25 12:39 AM, Thomas Huth wrote:
>>>>> Then I don't understand the previous argument from Thomas to not make
>>>>> thorough the default: "The thorough functional tests download a
>>>>> lot of assets from the internet, so if someone just runs "meson test"
>>>>> without further parameters, I think we should not trigger these
>>>>> downloads in that case". It's what precache-functional is doing.
>>>>
>>>> precache-functional is *only* called when you run "make check-functional",
>>>> i.e. when you know that you want to run the functional tests that might
>>>> download assets from the internet. It's not called when you run the normal
>>>> "make check".
>>>
>>> Are you sure ?  If that's the case it was *not* my intention when i
>>> added precaching - I thought that "make check"  would call
>>> "make check-functional" and thus trigger precaching ?
>>
>> "check-functional" is not part of the normal "check" target - on purpose
>> since we don't want to trigger downloads by surprise when people just run
>> "make check". That's also why we have separate "functional" CI jobs in the
>> gitlab CI, since otherwise this would be handled by the normal "check" jobs
>> already.
>>
> 
> `make check` calls build/pyvenv/bin/meson test --no-rebuild -t 1, which 
> triggers func-quick by default, triggering associated downloads, since 
> QEMU_TEST_NO_DOWNLOAD=1 is not set for this target, except if I missed 
> another hidden hack somewhere.

You missed the fact that the "quick" functional tests do not download any 
assets :-)

I know it's confusing since the name of the suites rather indicate something 
about the runtime of the tests and not about whether they trigger a download 
or not, but the suite names are so deeply glued into the "mtest2make" logic 
that I was not able to come up with a better solution. Maybe Paolo or some 
other Meson expert could clean that up, but for the time being, for the 
functional test we have:

- quick tests that can always run (also run during "make check")

- thorough tests that download assets from the internet (only run during 
"make check-functional")

I tried to document it in docs/devel/testing/functional.rst in the "Asset 
handling" section already, please have a look whether that's sufficient, or 
whether you have some ideas how to improve the situation.

  Thomas