public inbox for ltp@lists.linux.it
 help / color / mirror / Atom feed
From: Li Wang <liwang@redhat.com>
To: ltp@lists.linux.it
Subject: [LTP] [PATCH 2/3] tst_fuzzy_sync: ingnore test result if loop termination occurs
Date: Sun, 22 Dec 2024 15:51:01 +0800	[thread overview]
Message-ID: <20241222075102.14536-2-liwang@redhat.com> (raw)
In-Reply-To: <20241222075102.14536-1-liwang@redhat.com>

This patch updates the behavior of tst_fuzzy_sync tests to address
scenarios where test results may be unreliable due to resource constraints.

Specifically, if pair->exit is set (indicating exceeded loop termination),
the test result is ignored, and a TCONF status is returned. This ensures
invalid results are not falsely reported as failures or successes.

  tst_fuzzy_sync.h:654: TINFO: Exceeded execution loops, requesting exit
  /lib/newlib_tests/tst_fuzzy_sync02.c:164: TFAIL: 2| =:15

Signed-off-by: Li Wang <liwang@redhat.com>
---
 lib/newlib_tests/tst_fuzzy_sync01.c | 21 +++++++++++++++++++++
 lib/newlib_tests/tst_fuzzy_sync02.c | 21 +++++++++++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/lib/newlib_tests/tst_fuzzy_sync01.c b/lib/newlib_tests/tst_fuzzy_sync01.c
index d510bd2dd..c8c195859 100644
--- a/lib/newlib_tests/tst_fuzzy_sync01.c
+++ b/lib/newlib_tests/tst_fuzzy_sync01.c
@@ -212,10 +212,31 @@ static void run(unsigned int i)
 
 		if (critical > 100) {
 			tst_fzsync_pair_cleanup(&pair);
+			tst_atomic_store(0, &pair.exit);
 			break;
 		}
 	}
 
+	/*
+	 * If `pair->exit` is true, the test may fail to meet expected
+	 * results due to resource constraints in shared CI environments
+	 * (e.g., GitHub Actions). Limited control over CPU allocation
+	 * can cause delays or interruptions in CPU time slices due to
+	 * contention with other jobs.
+	 *
+	 * Binding the test to a single CPU core (e.g., via `taskset -c 0`)
+	 * can worsen this by increasing contention, leading to performance
+	 * degradation and premature loop termination.
+	 *
+	 * To ensure valid and reliable results in scenarios (e.g., HW, VM, CI),
+	 * it is best to ingore test result when loop termination occurs,
+	 * avoiding unnecessary false postive.
+	 */
+	if (pair.exit) {
+		tst_res(TCONF, "Test may not be able to generate a valid result");
+		return;
+	}
+
 	tst_res(critical > 50 ? TPASS : TFAIL,
 		"acs:%-2d act:%-2d art:%-2d | =:%-4d -:%-4d +:%-4d",
 		a.critical_s, a.critical_t, a.return_t,
diff --git a/lib/newlib_tests/tst_fuzzy_sync02.c b/lib/newlib_tests/tst_fuzzy_sync02.c
index f95394371..ad19a675e 100644
--- a/lib/newlib_tests/tst_fuzzy_sync02.c
+++ b/lib/newlib_tests/tst_fuzzy_sync02.c
@@ -192,10 +192,31 @@ static void run(unsigned int i)
 
 		if (critical > 100) {
 			tst_fzsync_pair_cleanup(&pair);
+			tst_atomic_store(0, &pair.exit);
 			break;
 		}
 	}
 
+	/*
+	 * If `pair->exit` is true, the test may fail to meet expected
+	 * results due to resource constraints in shared CI environments
+	 * (e.g., GitHub Actions). Limited control over CPU allocation
+	 * can cause delays or interruptions in CPU time slices due to
+	 * contention with other jobs.
+	 *
+	 * Binding the test to a single CPU core (e.g., via `taskset -c 0`)
+	 * can worsen this by increasing contention, leading to performance
+	 * degradation and premature loop termination.
+	 *
+	 * To ensure valid and reliable results in scenarios (e.g., HW, VM, CI),
+	 * it is best to ingore test result when loop termination occurs,
+	 * avoiding unnecessary false postive.
+	 */
+	if (pair.exit) {
+		tst_res(TCONF, "Test may not be able to generate a valid result");
+		return;
+	}
+
 	tst_res(critical > 50 ? TPASS : TFAIL, "%d| =:%-4d", i, critical);
 }
 
-- 
2.47.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

  reply	other threads:[~2024-12-22  7:52 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-12-22  7:51 [LTP] [PATCH 1/3] tst_fuzzy_sync: refine code and comment Li Wang
2024-12-22  7:51 ` Li Wang [this message]
2025-01-02 14:50   ` [LTP] [PATCH 2/3] tst_fuzzy_sync: ingnore test result if loop termination occurs Petr Vorel
2024-12-22  7:51 ` [LTP] [PATCH 3/3] newlib_test: enable all tst_fuzzy_sync tests Li Wang
2025-01-02 14:53   ` Petr Vorel
2025-01-02 14:38 ` [LTP] [PATCH 1/3] tst_fuzzy_sync: refine code and comment Petr Vorel
2025-01-02 14:41 ` Petr Vorel
2025-01-08  2:45   ` Li Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241222075102.14536-2-liwang@redhat.com \
    --to=liwang@redhat.com \
    --cc=ltp@lists.linux.it \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox