[PATCH] tests/qtest/npcm7xx_watchdog_timer: Only test the corner cases by default

[PATCH] tests/qtest/npcm7xx_watchdog_timer: Only test the corner cases by default

[Thomas Huth]

The test_prescaler() part in the npcm7xx_watchdog_timer test is quite
repetive, testing all possible combinations of the WTCLK and WTIS
bitfields. Since each test spins up a new instance of QEMU, this is
rather an expensive test, especially on loaded host systems.
For the normal quick test mode, it should be sufficient to test the
corner settings of these fields (i.e. 0 and 3), so we can speed up
this test in the default mode quite a bit.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/npcm7xx_watchdog_timer-test.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/tests/qtest/npcm7xx_watchdog_timer-test.c b/tests/qtest/npcm7xx_watchdog_timer-test.c
index 4773a673b2..981b853c99 100644
--- a/tests/qtest/npcm7xx_watchdog_timer-test.c
+++ b/tests/qtest/npcm7xx_watchdog_timer-test.c
@@ -172,9 +172,10 @@ static void test_reset_action(gconstpointer watchdog)
 static void test_prescaler(gconstpointer watchdog)
 {
     const Watchdog *wd = watchdog;
+    int inc = g_test_quick() ? 3 : 1;
 
-    for (int wtclk = 0; wtclk < 4; ++wtclk) {
-        for (int wtis = 0; wtis < 4; ++wtis) {
+    for (int wtclk = 0; wtclk < 4; wtclk += inc) {
+        for (int wtis = 0; wtis < 4; wtis += inc) {
             QTestState *qts = qtest_init("-machine quanta-gsj");
 
             qtest_irq_intercept_in(qts, "/machine/soc/a9mpcore/gic");
-- 
2.43.0

Re: [PATCH] tests/qtest/npcm7xx_watchdog_timer: Only test the corner cases by default

[Philippe Mathieu-Daudé]

On 15/1/24 08:02, Thomas Huth wrote:
> The test_prescaler() part in the npcm7xx_watchdog_timer test is quite
> repetive, testing all possible combinations of the WTCLK and WTIS
> bitfields. Since each test spins up a new instance of QEMU, this is
> rather an expensive test, especially on loaded host systems.
> For the normal quick test mode, it should be sufficient to test the
> corner settings of these fields (i.e. 0 and 3), so we can speed up
> this test in the default mode quite a bit.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/qtest/npcm7xx_watchdog_timer-test.c | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)

Good idea.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH] tests/qtest/npcm7xx_watchdog_timer: Only test the corner cases by default

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> The test_prescaler() part in the npcm7xx_watchdog_timer test is quite
> repetive, testing all possible combinations of the WTCLK and WTIS
> bitfields. Since each test spins up a new instance of QEMU, this is
> rather an expensive test, especially on loaded host systems.

I'm not against the change but I do not my home machine runs these tests
in:

  1/1 qemu:qtest+qtest-arm / qtest-arm/npcm7xx_timer-test        OK              0.18s   180 subtests passed

so I do wonder how the system load can cause such a dramatic increase
for a comparatively simple test.

> For the normal quick test mode, it should be sufficient to test the
> corner settings of these fields (i.e. 0 and 3), so we can speed up
> this test in the default mode quite a bit.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/qtest/npcm7xx_watchdog_timer-test.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/tests/qtest/npcm7xx_watchdog_timer-test.c b/tests/qtest/npcm7xx_watchdog_timer-test.c
> index 4773a673b2..981b853c99 100644
> --- a/tests/qtest/npcm7xx_watchdog_timer-test.c
> +++ b/tests/qtest/npcm7xx_watchdog_timer-test.c
> @@ -172,9 +172,10 @@ static void test_reset_action(gconstpointer watchdog)
>  static void test_prescaler(gconstpointer watchdog)
>  {
>      const Watchdog *wd = watchdog;
> +    int inc = g_test_quick() ? 3 : 1;
>  
> -    for (int wtclk = 0; wtclk < 4; ++wtclk) {
> -        for (int wtis = 0; wtis < 4; ++wtis) {
> +    for (int wtclk = 0; wtclk < 4; wtclk += inc) {
> +        for (int wtis = 0; wtis < 4; wtis += inc) {
>              QTestState *qts = qtest_init("-machine quanta-gsj");
>  
>              qtest_irq_intercept_in(qts, "/machine/soc/a9mpcore/gic");

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH] tests/qtest/npcm7xx_watchdog_timer: Only test the corner cases by default

[Thomas Huth]

On 15/01/2024 12.32, Alex Bennée wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> The test_prescaler() part in the npcm7xx_watchdog_timer test is quite
>> repetive, testing all possible combinations of the WTCLK and WTIS
>> bitfields. Since each test spins up a new instance of QEMU, this is
>> rather an expensive test, especially on loaded host systems.
> 
> I'm not against the change but I do not my home machine runs these tests
> in:
> 
>    1/1 qemu:qtest+qtest-arm / qtest-arm/npcm7xx_timer-test        OK              0.18s   180 subtests passed

That's a different test, I think. Look for "watchdog" in its name, please.

> so I do wonder how the system load can cause such a dramatic increase
> for a comparatively simple test.

The watchdog test is executing a new QEMU instance for each test in the 
loop, and that can be quite expensive on a loaded system, I think.

  Thomas

Re: [PATCH] tests/qtest/npcm7xx_watchdog_timer: Only test the corner cases by default

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 15/01/2024 12.32, Alex Bennée wrote:
>> Thomas Huth <thuth@redhat.com> writes:
>> 
>>> The test_prescaler() part in the npcm7xx_watchdog_timer test is quite
>>> repetive, testing all possible combinations of the WTCLK and WTIS
>>> bitfields. Since each test spins up a new instance of QEMU, this is
>>> rather an expensive test, especially on loaded host systems.
>> I'm not against the change but I do not my home machine runs these
>> tests
>> in:
>>    1/1 qemu:qtest+qtest-arm / qtest-arm/npcm7xx_timer-test        OK
>> 0.18s   180 subtests passed
>
> That's a different test, I think. Look for "watchdog" in its name,
> please.

Ahh -ETOOMANYNPCMS...

So yes I get a drop in time by half:

  1/1 qemu:qtest+qtest-arm / qtest-arm/npcm7xx_watchdog_timer-test        OK             10.35s   15 subtests passed

to

  1/1 qemu:qtest+qtest-arm / qtest-arm/npcm7xx_watchdog_timer-test        OK              4.68s   15 subtests passed


Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

>> so I do wonder how the system load can cause such a dramatic increase
>> for a comparatively simple test.
>
> The watchdog test is executing a new QEMU instance for each test in
> the loop, and that can be quite expensive on a loaded system, I think.
>
>  Thomas

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro