[PATCH] tests: Add a target to run all tests in thorough mode

[PATCH] tests: Add a target to run all tests in thorough mode

[Thomas Huth]

"make check SPEED=thorough" is a little bit clunky to type and many
people are not aware of this nice possibility to run almost all test
in parallel this way. So add a "check-thorough" target that makes this
way of running the tests a little bit more accessible.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/Makefile.include | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 9f82552ab54..d9d2b7ef13b 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -4,6 +4,7 @@
 check-help:
 	@echo "Regression testing targets:"
 	@echo " $(MAKE) check                         Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
+	@echo " $(MAKE) check-thorough                Download assets and run almost all non-flaky tests"
 	@echo " $(MAKE) bench                         Run speed tests"
 	@echo
 	@echo "Individual test suites:"
@@ -46,10 +47,13 @@ FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$
 $(FUNCTIONAL_TARGETS):
 	@$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
 
-.PHONY: check-functional
-check-functional:
+.PHONY: check-functional-precache
+check-functional-precache:
 	@$(NINJA) precache-functional
 	@$(PYTHON) $(SRC_PATH)/scripts/clean_functional_cache.py
+
+.PHONY: check-functional
+check-functional: check-functional-precache
 	@QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
 
 .PHONY: check-func check-func-quick
@@ -57,9 +61,12 @@ check-func check-func-quick:
 
 # Consolidated targets
 
-.PHONY: check check-clean
+.PHONY: check check-thorough check-clean
 check:
 
+check-thorough: check-functional-precache
+	$(MAKE) check SPEED=thorough QEMU_TEST_NO_DOWNLOAD=1
+
 check-build: run-ninja
 
 check-clean:
-- 
2.56.0

Re: [PATCH] tests: Add a target to run all tests in thorough mode

[Philippe Mathieu-Daudé]

On 26/10/26 10:30, Thomas Huth wrote:
> "make check SPEED=thorough" is a little bit clunky to type and many
> people are not aware of this nice possibility to run almost all test
> in parallel this way. So add a "check-thorough" target that makes this
> way of running the tests a little bit more accessible.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/Makefile.include | 13 ++++++++++---
>   1 file changed, 10 insertions(+), 3 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>

Re: [PATCH] tests: Add a target to run all tests in thorough mode

[Daniel P. Berrangé]

On Mon, Oct 26, 2026 at 10:30:34AM +0100, Thomas Huth wrote:
> "make check SPEED=thorough" is a little bit clunky to type and many
> people are not aware of this nice possibility to run almost all test
> in parallel this way. So add a "check-thorough" target that makes this
> way of running the tests a little bit more accessible.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/Makefile.include | 13 ++++++++++---
>  1 file changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 9f82552ab54..d9d2b7ef13b 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -4,6 +4,7 @@
>  check-help:
>  	@echo "Regression testing targets:"
>  	@echo " $(MAKE) check                         Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
> +	@echo " $(MAKE) check-thorough                Download assets and run almost all non-flaky tests"
>  	@echo " $(MAKE) bench                         Run speed tests"
>  	@echo
>  	@echo "Individual test suites:"
> @@ -46,10 +47,13 @@ FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$
>  $(FUNCTIONAL_TARGETS):
>  	@$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
>  
> -.PHONY: check-functional
> -check-functional:
> +.PHONY: check-functional-precache
> +check-functional-precache:
>  	@$(NINJA) precache-functional
>  	@$(PYTHON) $(SRC_PATH)/scripts/clean_functional_cache.py
> +
> +.PHONY: check-functional
> +check-functional: check-functional-precache
>  	@QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
>  
>  .PHONY: check-func check-func-quick
> @@ -57,9 +61,12 @@ check-func check-func-quick:
>  
>  # Consolidated targets
>  
> -.PHONY: check check-clean
> +.PHONY: check check-thorough check-clean
>  check:
>  
> +check-thorough: check-functional-precache
> +	$(MAKE) check SPEED=thorough QEMU_TEST_NO_DOWNLOAD=1

I'm not understanding why this commit has the refactoring present
too ? Wasn't it enough to do

 +check-thorough:
 +	$(MAKE) check SPEED=thorough

the precaching download would just do the right thing already


With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH] tests: Add a target to run all tests in thorough mode

[Thomas Huth]

On 26/10/2026 11.21, Daniel P. Berrangé wrote:
> On Mon, Oct 26, 2026 at 10:30:34AM +0100, Thomas Huth wrote:
>> "make check SPEED=thorough" is a little bit clunky to type and many
>> people are not aware of this nice possibility to run almost all test
>> in parallel this way. So add a "check-thorough" target that makes this
>> way of running the tests a little bit more accessible.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>   tests/Makefile.include | 13 ++++++++++---
>>   1 file changed, 10 insertions(+), 3 deletions(-)
>>
>> diff --git a/tests/Makefile.include b/tests/Makefile.include
>> index 9f82552ab54..d9d2b7ef13b 100644
>> --- a/tests/Makefile.include
>> +++ b/tests/Makefile.include
>> @@ -4,6 +4,7 @@
>>   check-help:
>>   	@echo "Regression testing targets:"
>>   	@echo " $(MAKE) check                         Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
>> +	@echo " $(MAKE) check-thorough                Download assets and run almost all non-flaky tests"
>>   	@echo " $(MAKE) bench                         Run speed tests"
>>   	@echo
>>   	@echo "Individual test suites:"
>> @@ -46,10 +47,13 @@ FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$
>>   $(FUNCTIONAL_TARGETS):
>>   	@$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
>>   
>> -.PHONY: check-functional
>> -check-functional:
>> +.PHONY: check-functional-precache
>> +check-functional-precache:
>>   	@$(NINJA) precache-functional
>>   	@$(PYTHON) $(SRC_PATH)/scripts/clean_functional_cache.py
>> +
>> +.PHONY: check-functional
>> +check-functional: check-functional-precache
>>   	@QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
>>   
>>   .PHONY: check-func check-func-quick
>> @@ -57,9 +61,12 @@ check-func check-func-quick:
>>   
>>   # Consolidated targets
>>   
>> -.PHONY: check check-clean
>> +.PHONY: check check-thorough check-clean
>>   check:
>>   
>> +check-thorough: check-functional-precache
>> +	$(MAKE) check SPEED=thorough QEMU_TEST_NO_DOWNLOAD=1
> 
> I'm not understanding why this commit has the refactoring present
> too ? Wasn't it enough to do
> 
>   +check-thorough:
>   +	$(MAKE) check SPEED=thorough
> 
> the precaching download would just do the right thing already
No, that way, the precaching (i.e. the downloads before the tests start) are 
not triggered at all, and assets will be downloaded while the tests run 
instead. That means you'd easily run into timeout problems when the download 
takes too long. We need the check-functional-precache target for 
check-through to make sure that the assets are downloaded before the tests 
start.

  Thomas

Re: [PATCH] tests: Add a target to run all tests in thorough mode

[Daniel P. Berrangé]

On Mon, Oct 26, 2026 at 10:30:34AM +0100, Thomas Huth wrote:
> "make check SPEED=thorough" is a little bit clunky to type and many
> people are not aware of this nice possibility to run almost all test
> in parallel this way. So add a "check-thorough" target that makes this
> way of running the tests a little bit more accessible.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/Makefile.include | 13 ++++++++++---
>  1 file changed, 10 insertions(+), 3 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|