Re: [PATCH] Add option to disable building tests

[Fabiano Rosas <farosas@suse.de>]

Philippe Mathieu-Daudé <philmd@oss.qualcomm.com> writes:

> On 27/7/26 13:19, BALATON Zoltan wrote:
>> On Mon, 27 Jul 2026, Thomas Huth wrote:
>>> On 27/07/2026 10.12, Philippe Mathieu-Daudé wrote:
>>>> On 24/7/26 18:45, Pierrick Bouvier wrote:
>>>>> On 7/24/2026 6:54 AM, Florian Schmidt wrote:
>>>>>> There are situations in which you might want to build QEMU without
>>>>>> building the full test suite, which in some configurations can take a
>>>>>> considerable amount of time to build. This is especially true when 
>>>>>> using
>>>>>> LTO with clang. For example:
>>>>>>
>>>>>> $ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto- 
>>>>>> objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target- 
>>>>>> list=x86_64-softmmu
>>>>>> [...]
>>>>>> $ time make -j8
>>>>>> [...]
>>>>>> [3078/3078] Linking target tests/qtest/qos-test
>>>>>> real    6m43.250s
>>>>>> user    101m15.967s
>>>>>> sys     4m3.813s
>>>>>>
>>>>>> $ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto- 
>>>>>> objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target- 
>>>>>> list=x86_64- softmmu --disable-tests
>>>>>> [...]
>>>>>> $ time make -j8
>>>>>> [...]
>>>>>> [2024/2024] Linking target qemu-system-x86_64
>>>>>>
>>>>>> real    3m14.277s
>>>>>> user    33m13.174s
>>>>>> sys     1m36.642s
>>>>>>
>>>>>> Add a toggle to optionally disable building tests.
>>>>>>
>>>>>> Signed-off-by: Florian Schmidt <flosch@nutanix.com>
>>>>>> ---
>>>>>>   meson.build                   | 2 +-
>>>>>>   meson_options.txt             | 2 ++
>>>>>>   scripts/meson-buildoptions.sh | 3 +++
>>>>>>   3 files changed, 6 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/meson.build b/meson.build
>>>>>> index 49a5baf5b5..ad728c26ec 100644
>>>>>> --- a/meson.build
>>>>>> +++ b/meson.build
>>>>>> @@ -4605,7 +4605,7 @@ subdir('docs')
>>>>>>   subdir('pyvenv')
>>>>>>   # Tests are disabled on emscripten because they rely on host 
>>>>>> features that aren't
>>>>>>   # supported by emscripten (e.g. fork and unix socket).
>>>>>> -if host_os != 'emscripten'
>>>>>> +if get_option('tests').allowed() and host_os != 'emscripten'
>>>>>>     subdir('tests')
>>>>>>   endif
>>>>>>   if gtk.found()
>>>>>> diff --git a/meson_options.txt b/meson_options.txt
>>>>>> index a07cb47d35..68333ca402 100644
>>>>>> --- a/meson_options.txt
>>>>>> +++ b/meson_options.txt
>>>>>> @@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '',
>>>>>>   # on the configure script command line.  After adding an option
>>>>>>   # here make sure to run "make update-buildoptions".
>>>>>> +option('tests', type: 'feature', value: 'auto',
>>>>>> +       description: 'Build the test suite')
>>>>>>   option('docs', type : 'feature', value : 'auto',
>>>>>>          description: 'Documentations build support')
>>>>>>   option('fuzzing', type : 'boolean', value: false,
>>>>>> diff --git a/scripts/meson-buildoptions.sh b/scripts/meson- 
>>>>>> buildoptions.sh
>>>>>> index c003985047..3fec13a336 100644
>>>>>> --- a/scripts/meson-buildoptions.sh
>>>>>> +++ b/scripts/meson-buildoptions.sh
>>>>>> @@ -194,6 +194,7 @@ meson_options_help() {
>>>>>>     printf "%s\n" '  spice-protocol  Spice protocol support'
>>>>>>     printf "%s\n" '  stack-protector compiler-provided stack 
>>>>>> protection'
>>>>>>     printf "%s\n" '  tcg             TCG support'
>>>>>> +  printf "%s\n" '  tests           build test suite'
>>>>>>     printf "%s\n" '  tools           build support utilities that 
>>>>>> come with QEMU'
>>>>>>     printf "%s\n" '  tpm             TPM support'
>>>>>>     printf "%s\n" '  u2f             U2F emulation support'
>>>>>> @@ -515,6 +516,8 @@ _meson_option_parse() {
>>>>>>       --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;;
>>>>>>       --disable-tcg-interpreter) printf "%s" - 
>>>>>> Dtcg_interpreter=false ;;
>>>>>>       --tls-priority=*) quote_sh "-Dtls_priority=$2" ;;
>>>>>> +    --enable-tests) printf "%s" -Dtests=enabled ;;
>>>>>> +    --disable-tests) printf "%s" -Dtests=disabled ;;
>>>>>>       --enable-tools) printf "%s" -Dtools=enabled ;;
>>>>>>       --disable-tools) printf "%s" -Dtools=disabled ;;
>>>>>>       --enable-tpm) printf "%s" -Dtpm=enabled ;;
>>>>>
>>>>> That's a great addition, including just for speeding up normal builds.
>>>>>
>>>>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>>>> Tested-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>>>>
>>>>> If we want to overengineer the thing, it could be possible to still
>>>>> declare all tests, but not build them by default. However, it's 
>>>>> probably
>>>>> too error prone and much less simple than this patch. So I don't think
>>>>> it's a good idea.
>>>>>
>>>>> Regards,
>>>>> Pierrick
>>>>>
>>>>
>>>> When is it useful to build without the provided test suite?
>>> I sometimes wished indeed for a --disable-tests switch in the past 
>>> already (was just too lazy to contribute a patch): I'm sometimes 
>>> building QEMU in a separate directory, just for a very special case 
>>> like testing a patch with an --enable-asan build, or for doing a "git 
>>> bisect". In such cases, I never want to run the normal tests since 
>>> it's simply not necessary. I run the tests from my main build 
>>> directory instead once I have a final patch and want to contribute it 
>>> to upstream. So IMHO this patch is a good idea.
>> 
>> I similarly have a local patch for this for such usage, that I never 
>> cleaned up to submit but I think it's a good idea.
>
> I'm not saying this patch is a bad idea, I'll likely use it too;
> I want to clarify what are the valid use cases the community sees
> here. Building pointless things clearly has a negative impact on
> resources and our time, but not testing changes also has.

Perhaps we could add something to checkpatch that emits an error if
config-status has --disable-tests.

> IOW in somes cases this change is acceptable, but I'm a little wary
> on massive use by default.
>
> Anyway I didn't wanted to start yet another endless discussion so I
> won't intervene further in this thread, sorry for the usual noise.

I think we'd rather have your comments and everyone else's, there's no
deadline for anything.