Re: [LTP] [RFC PATCh] lib: redefine the overall timeout logic of test

[Cyril Hrubis <chrubis@suse.cz>]

Hi!
> Key Concepts:
> 
>  timeout: Represents the time allocated for setup, cleanup, or other
>           non-runtime parts of the test.
> 
>  runtime: The maximum runtime of the test's main execution loop. If
>           exceeded, the test will exit early.

This is not entirely correct. I guess that we still didn't understand
each other. Let me try to explain once more.

The runtime is supposed to be used only in tests that call
tst_remaning_runtime() in the main loop to limit the test duration based
on runtime. This means that the test function execution time is always
the same (plus minus some rounding errors).

And the timeout is supposed to be upper bound for everything in the test
minus the part that is limited by the runtime.

So for the tests that use tst_remaining_runtime() the timeout guards the
setup and cleanup, but for tests that does not use
tst_remaining_runtime() the timeout limits the whole test run.

The reasoning behind this is that if for example we have a test where
the test function does I/O that is known to take up to 10s we want to
multiply this in the case of the debug kernel because the test function
may be slower.

On the other hand test function that has a loop that checks the remaning
runtime will do less iterations on debug kernel but the duration will be
the same. And we do not want to multiply this value because it will make
the test even slower on debug kernels.

The changes mostly looks okay, but some of the tests below that had
.max_runtime should replace that with .timeout instead. And we also
need tst_set_timeout() function for the testcases that try to figure out
the upper bound for the test duration in the test setup.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp