Re: [LTP] [RFC] Reduce .runtime for Long-Running Tests ?

[Cyril Hrubis <chrubis@suse.cz>]

Hi!
> After a round of experiments and internal discussions (thanks to
> Ian Wien for sharing his thoughts with me), we think that making
> LTP_RUNTIME_MUL support floating point numbers [0, 1] may
> a possible way to reduce the .runtime values set in tests.
> 
> For example, setting LTP_RUNTIME_MUL to 0.1 can obviously
> reduce the test time of 600 seconds to 60 seconds.
> 
> One may think that reducing the .runtime value in aproduction
> environment is potentially risky, and to some extent the answer
> is yes.
> 
> But looking back, LTP is triggered very frequently in CI and various
> production flows, so to compensate for this loss, we can use floating
> point LTP_RUNTIME_MUL only in designated quick CI, instead of
> using it in daily tests. This will help cover more scenarios.

Having reduced runtime for CI makes sense, as you said you are making up
for the reduction by the number of testruns. It may not work well for
fuzzy sync though where we depend on having good enough sampling period.

Also limiting the smallest multiplier to be 0.1 or so does make sense. I
assume that if we set it to 0.01 or smaller most tests with runtime
woudn't even execute.

> From our CI report, use 0.1 in runtime_mul find a few failures in the round
> down problem with nice05.c (.runtime = 3), this is a defect of
> multply_runtime().
> Also, another preadv203 possible failure related this. But they are tiny
> issues. And the rest .runtime tests so far no obvious problem on that.

I guess that we need to make sure that the runtime stays positive. So
maybe we need something as:

runtime = MAX(2, tst_test->runtime * runtime_mul)

> So I would like to start the work from this point to reduce execution time.

Sounds good to me.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp