Re: [LTP] [PATCH 2/3] tst_fuzzy_sync: ingnore test result if loop termination occurs

[Petr Vorel <pvorel@suse.cz>]

Hi Li,

> This patch updates the behavior of tst_fuzzy_sync tests to address
> scenarios where test results may be unreliable due to resource constraints.

> Specifically, if pair->exit is set (indicating exceeded loop termination),
> the test result is ignored, and a TCONF status is returned. This ensures
> invalid results are not falsely reported as failures or successes.

>   tst_fuzzy_sync.h:654: TINFO: Exceeded execution loops, requesting exit
>   /lib/newlib_tests/tst_fuzzy_sync02.c:164: TFAIL: 2| =:15

LGTM, good catch.

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Below are noted typos, could you please fix them before merge?

> Signed-off-by: Li Wang <liwang@redhat.com>
> ---
>  lib/newlib_tests/tst_fuzzy_sync01.c | 21 +++++++++++++++++++++
>  lib/newlib_tests/tst_fuzzy_sync02.c | 21 +++++++++++++++++++++
>  2 files changed, 42 insertions(+)

> diff --git a/lib/newlib_tests/tst_fuzzy_sync01.c b/lib/newlib_tests/tst_fuzzy_sync01.c
> index d510bd2dd..c8c195859 100644
> --- a/lib/newlib_tests/tst_fuzzy_sync01.c
> +++ b/lib/newlib_tests/tst_fuzzy_sync01.c
> @@ -212,10 +212,31 @@ static void run(unsigned int i)

>  		if (critical > 100) {
>  			tst_fzsync_pair_cleanup(&pair);
> +			tst_atomic_store(0, &pair.exit);
>  			break;
>  		}
>  	}

> +	/*
> +	 * If `pair->exit` is true, the test may fail to meet expected
> +	 * results due to resource constraints in shared CI environments
> +	 * (e.g., GitHub Actions). Limited control over CPU allocation
> +	 * can cause delays or interruptions in CPU time slices due to
> +	 * contention with other jobs.
> +	 *
> +	 * Binding the test to a single CPU core (e.g., via `taskset -c 0`)
> +	 * can worsen this by increasing contention, leading to performance
> +	 * degradation and premature loop termination.
> +	 *
> +	 * To ensure valid and reliable results in scenarios (e.g., HW, VM, CI),
> +	 * it is best to ingore test result when loop termination occurs,
s/ingore/ignore/
> +	 * avoiding unnecessary false postive.
s/postive/positive/

> +	 */
> +	if (pair.exit) {
> +		tst_res(TCONF, "Test may not be able to generate a valid result");
> +		return;
> +	}
> +
>  	tst_res(critical > 50 ? TPASS : TFAIL,
>  		"acs:%-2d act:%-2d art:%-2d | =:%-4d -:%-4d +:%-4d",
>  		a.critical_s, a.critical_t, a.return_t,
> diff --git a/lib/newlib_tests/tst_fuzzy_sync02.c b/lib/newlib_tests/tst_fuzzy_sync02.c
> index f95394371..ad19a675e 100644
> --- a/lib/newlib_tests/tst_fuzzy_sync02.c
> +++ b/lib/newlib_tests/tst_fuzzy_sync02.c
> @@ -192,10 +192,31 @@ static void run(unsigned int i)

>  		if (critical > 100) {
>  			tst_fzsync_pair_cleanup(&pair);
> +			tst_atomic_store(0, &pair.exit);
>  			break;
>  		}
>  	}

> +	/*
> +	 * If `pair->exit` is true, the test may fail to meet expected
> +	 * results due to resource constraints in shared CI environments
> +	 * (e.g., GitHub Actions). Limited control over CPU allocation
> +	 * can cause delays or interruptions in CPU time slices due to
> +	 * contention with other jobs.
> +	 *
> +	 * Binding the test to a single CPU core (e.g., via `taskset -c 0`)
> +	 * can worsen this by increasing contention, leading to performance
> +	 * degradation and premature loop termination.
> +	 *
> +	 * To ensure valid and reliable results in scenarios (e.g., HW, VM, CI),
> +	 * it is best to ingore test result when loop termination occurs,
s/ingore/ignore/
> +	 * avoiding unnecessary false postive.
s/postive/positive/

Also there is original typo from Richie:
s/Emperically/Empirically/

Kind regards,
Petr

> +	 */
> +	if (pair.exit) {
> +		tst_res(TCONF, "Test may not be able to generate a valid result");
> +		return;
> +	}

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp