Re: [PATCH] meson: use thorough test setup as default

[Thomas Huth <thuth@redhat.com>]

On 07/05/2025 09.14, Daniel P. Berrangé wrote:
> On Tue, May 06, 2025 at 07:12:53PM +0200, Thomas Huth wrote:
>> On 06/05/2025 17.31, Pierrick Bouvier wrote:
>>> On 5/6/25 2:33 AM, Daniel P. Berrangé wrote:
>>>> On Mon, May 05, 2025 at 10:46:52AM -0700, Pierrick Bouvier wrote:
>>>>> On 5/5/25 3:32 AM, Thomas Huth wrote:
>>>>>> On 03/05/2025 22.18, Pierrick Bouvier wrote:
>>>>>>> Allows all tests to be visible by default when using meson test
>>>>>>> directly.
>>>>>>>
>>>>>>> This has no impact on make check-* commands, which use SPEED=quick by
>>>>>>> default (see scripts/mtest2make.py).
>>>>>>>
>>>>>>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>>>>>>> ---
>>>>>>>      meson.build | 5 +++--
>>>>>>>      1 file changed, 3 insertions(+), 2 deletions(-)
>>>>>>>
>>>>>>> diff --git a/meson.build b/meson.build
>>>>>>> index f8bf6e7bb66..57ff3f722d8 100644
>>>>>>> --- a/meson.build
>>>>>>> +++ b/meson.build
>>>>>>> @@ -5,12 +5,13 @@ project('qemu', ['c'], meson_version: '>=1.5.0',
>>>>>>>      meson.add_devenv({ 'MESON_BUILD_ROOT' : meson.project_build_root() })
>>>>>>> -add_test_setup('quick', exclude_suites: ['slow',
>>>>>>> 'thorough'], is_default: true,
>>>>>>> +add_test_setup('quick', exclude_suites: ['slow', 'thorough'],
>>>>>>>                     env: ['RUST_BACKTRACE=1'])
>>>>>>>      add_test_setup('slow', exclude_suites: ['thorough'],
>>>>>>>                     env: ['G_TEST_SLOW=1', 'SPEED=slow',
>>>>>>> 'RUST_BACKTRACE=1'])
>>>>>>>      add_test_setup('thorough',
>>>>>>> -               env: ['G_TEST_SLOW=1', 'SPEED=thorough',
>>>>>>> 'RUST_BACKTRACE=1'])
>>>>>>> +               env: ['G_TEST_SLOW=1', 'SPEED=thorough',
>>>>>>> 'RUST_BACKTRACE=1'],
>>>>>>> +               is_default: true)
>>>>>>
>>>>>> I'd rather not make it the default: The thorough functional
>>>>>> tests download a
>>>>>> lot of assets from the internet, so if someone just runs "meson test"
>>>>>> without further parameters, I think we should not trigger
>>>>>> these downloads in
>>>>>> that case.
>>>>>>
>>>>>
>>>>> $ cat tests/Makefile.include
>>>>> ...
>>>>> check-functional:
>>>>>      @$(NINJA) precache-functional
>>>>>      @QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func
>>>>> ...
>>>>>
>>>>> What's the rationale to run check-func with SPEED=thorough and
>>>>> QEMU_TEST_NO_DOWNLOAD=1 with precache-functional having cached only quick
>>>>> tests by default?
>>>>
>>>> Are you sure about that ? The precache logic uses custom_target() in
>>>> meson and so does not filters on "suites" used to define quick vs
>>>> thorough tests.
>>>>
>>>
>>> My bad, I missed the "foreach speed : ['quick', 'thorough']" wrapping
>>> all this in tests/functional/meson.build.
>>>
>>>> IOW, precache should always be downloading all assets.
>>>>
>>>
>>> Then I don't understand the previous argument from Thomas to not make
>>> thorough the default: "The thorough functional tests download a
>>> lot of assets from the internet, so if someone just runs "meson test"
>>> without further parameters, I think we should not trigger these
>>> downloads in that case". It's what precache-functional is doing.
>>
>> precache-functional is *only* called when you run "make check-functional",
>> i.e. when you know that you want to run the functional tests that might
>> download assets from the internet. It's not called when you run the normal
>> "make check".
> 
> Are you sure ?  If that's the case it was *not* my intention when i
> added precaching - I thought that "make check"  would call
> "make check-functional" and thus trigger precaching ?

"check-functional" is not part of the normal "check" target - on purpose 
since we don't want to trigger downloads by surprise when people just run 
"make check". That's also why we have separate "functional" CI jobs in the 
gitlab CI, since otherwise this would be handled by the normal "check" jobs 
already.

  Thomas