Re: [LTP] [PATCH 1/1] net: tst_netload_compare(): Ignore performance failures

[Petr Vorel <pvorel@suse.cz>]

> Hi all,

[ Cc Alexey, see
https://lore.kernel.org/ltp/20231219123709.339435-1-pvorel@suse.cz/
https://patchwork.ozlabs.org/project/ltp/patch/20231219123709.339435-1-pvorel@suse.cz/
]

> > Performance failures in tests which use tst_netload_compare() (tests in
> > runtest/net.features) can hide a real error (e.g. test fails due missing
> > required kernel module). Best solution would be to have feature tests
> > (likely written in C API) and performance tests (the current ones).

> > But until it happens, just disable performance failure by default,
> > print TINFO message instead unless TST_NET_FEATURES_TEST_PERFORMANCE=1
> > environment variable is set.

> NOTE: why is this enabled by default? IMHO it's more important to have feature
> tests than performance tests often failing and thus hiding feature test.
> Even introducing some LTP_TIMEOUT_MUL like variable would not help in case SUT
> performance differs (sometimes SUTs are ok, other time slow due QEMU host which
> runs SUTs is overbooked).

I knew there is some variable for performance tuning of network feature
tests: VIRT_PERF_THRESHOLD_MIN. But again, if the performance of the QEMU host
varies, the value must be set too high, which can hide the failure on broken
system which is idle. IMHO these tests should be run 2x - just feature tests
then with properly VIRT_PERF_THRESHOLD_MIN.

And, also VIRT_PERF_THRESHOLD_MIN is used only in virt_lib.sh, but
tst_netload_compare() which can fail due performance is used also in other
tests, e.g. sctp01.sh. That's why I think it's better to disable performance by
default.

Kind regards,
Petr

> Kind regards,
> Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp