[LTP] [PATCH 2/3] tst_fuzzy_sync: ingnore test result if loop termination occurs

Li Wang
2024-12-22T07:52:07Z

This patch updates the behavior of tst_fuzzy_sync tests to address
scenarios where test results may be unreliable due to resource constraints.

Specifically, if pair->exit is set (indicating exceeded loop termination),
the test result is ignored, and a TCONF status is returned. This ensures
invalid results are not falsely reported as failures or successes.

tst_fuzzy_sync.h:654: TINFO: Exceeded execution loops, requesting exit
/lib/newlib_tests/tst_fuzzy_sync02.c:164: TFAIL: 2| =:15

Signed-off-by: Li Wang <liwang@redhat.com>
---
lib/newlib_tests/tst_fuzzy_sync01.c | 21 +++++++++++++++++++++
lib/newlib_tests/tst_fuzzy_sync02.c | 21 +++++++++++++++++++++
2 files changed, 42 insertions(+)

diff --git a/lib/newlib_tests/tst_fuzzy_sync01.c b/lib/newlib_tests/tst_fuzzy_sync01.c
index d510bd2dd..c8c195859 100644
--- a/lib/newlib_tests/tst_fuzzy_sync01.c
+++ b/lib/newlib_tests/tst_fuzzy_sync01.c
@@ -212,10 +212,31 @@ static void run(unsigned int i)

if (critical > 100) {
tst_fzsync_pair_cleanup(&pair);
+ tst_atomic_store(0, &pair.exit);
break;
}
}

+ /*
+ * If `pair->exit` is true, the test may fail to meet expected
+ * results due to resource constraints in shared CI environments
+ * (e.g., GitHub Actions). Limited control over CPU allocation
+ * can cause delays or interruptions in CPU time slices due to
+ * contention with other jobs.
+ *
+ * Binding the test to a single CPU core (e.g., via `taskset -c 0`)
+ * can worsen this by increasing contention, leading to performance
+ * degradation and premature loop termination.
+ *
+ * To ensure valid and reliable results in scenarios (e.g., HW, VM, CI),
+ * it is best to ingore test result when loop termination occurs,
+ * avoiding unnecessary false postive.
+ */
+ if (pair.exit) {
+ tst_res(TCONF, "Test may not be able to generate a valid result");
+ return;
+ }
+
tst_res(critical > 50 ? TPASS : TFAIL,
"acs:%-2d act:%-2d art:%-2d | =:%-4d -:%-4d +:%-4d",
a.critical_s, a.critical_t, a.return_t,
diff --git a/lib/newlib_tests/tst_fuzzy_sync02.c b/lib/newlib_tests/tst_fuzzy_sync02.c
index f95394371..ad19a675e 100644
--- a/lib/newlib_tests/tst_fuzzy_sync02.c
+++ b/lib/newlib_tests/tst_fuzzy_sync02.c
@@ -192,10 +192,31 @@ static void run(unsigned int i)

if (critical > 100) {
tst_fzsync_pair_cleanup(&pair);
+ tst_atomic_store(0, &pair.exit);
break;
}
}

+ /*
+ * If `pair->exit` is true, the test may fail to meet expected
+ * results due to resource constraints in shared CI environments
+ * (e.g., GitHub Actions). Limited control over CPU allocation
+ * can cause delays or interruptions in CPU time slices due to
+ * contention with other jobs.
+ *
+ * Binding the test to a single CPU core (e.g., via `taskset -c 0`)
+ * can worsen this by increasing contention, leading to performance
+ * degradation and premature loop termination.
+ *
+ * To ensure valid and reliable results in scenarios (e.g., HW, VM, CI),
+ * it is best to ingore test result when loop termination occurs,
+ * avoiding unnecessary false postive.
+ */
+ if (pair.exit) {
+ tst_res(TCONF, "Test may not be able to generate a valid result");
+ return;
+ }
+
tst_res(critical > 50 ? TPASS : TFAIL, "%d| =:%-4d", i, critical);
}

--
2.47.1