[PATCH i-g-t v2] tests/kms_setmode: Relax vblank timing tolerance from 1 scanline to 1%

[Jason-JH Lin <jason-jh.lin@mediatek.com>]

The original check_timings() validation required vblank intervals to be
within 1 scanline (~15us for 1080p60) which is overly strict and causes
false failures due to userspace measurement overhead.

Changes:
- Keep original strict checks (1 scanline, 1.718 sigma) as warnings only
- Add relaxed validation with 1% relative tolerance for pass/fail
- Use 3 sigma threshold (99.7% confidence) instead of 1.718 sigma (90%)

Rationale:
Hardware specs (VESA/HDMI/DP) allow ±0.5% pixel clock tolerance, which
translates to ±0.5% frame time due to inverse relationship. However,
userspace measurement via read() syscall adds ~0.3-0.5% overhead from:
- Interrupt latency and scheduling delays (5-50 us)
- System call overhead (1-5 us)
- Variable delays depending on system load

Total tolerance: ±0.5% (hardware) + ~0.5% (software) = ±1.0%

This makes the test ~11x more tolerant while still detecting real timing
issues and aligning with industry hardware specifications.

Signed-off-by: Jason-JH Lin <jason-jh.lin@mediatek.com>
Suggested-by: Jeevain B <jeevan.b@intel.com>
---
 tests/kms_setmode.c | 71 +++++++++++++++++++++++++++++++++++++--------
 1 file changed, 59 insertions(+), 12 deletions(-)

diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
index 1f2849bc26cf..b452f8a9e997 100644
--- a/tests/kms_setmode.c
+++ b/tests/kms_setmode.c
@@ -495,6 +495,7 @@ static bool check_timings(int crtc_idx, const drmModeModeInfo *kmode)
 	double mean;
 	double stddev;
 	int n;
+	bool strict_failed = false;
 
 	memset(&wait, 0, sizeof(wait));
 	wait.request.type = kmstest_get_vbl_flag(crtc_idx);
@@ -563,10 +564,9 @@ static bool check_timings(int crtc_idx, const drmModeModeInfo *kmode)
 
 	/* 99.7% samples within one scanline on each side of mean */
 	if (accuracy >= line_time(kmode)) {
-		igt_info("vblank accuracy (%.3fus, %.1f%%) worse than a scanline (%.3fus)\n",
-			  accuracy, 100 * accuracy / mean, line_time(kmode));
-
-		return false;
+		igt_warn("vblank accuracy (%.3fus, %.1f%%) worse than a scanline (%.3fus)\n",
+			 accuracy, 100 * accuracy / mean, line_time(kmode));
+		strict_failed = true;
 	}
 
 	/* At least 90% of frame times fall within the one scanline on each
@@ -591,14 +591,61 @@ static bool check_timings(int crtc_idx, const drmModeModeInfo *kmode)
 	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
 	 */
 	if (fabs(mean - expected) >= max(line_time(kmode), 1.718 * stddev)) {
-		igt_info("vblank interval differs from modeline! expected %.1fus, "
-			 "measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
-			  expected, mean, stddev,
-			  fabs(mean - expected),
-			  fabs(mean - expected) / stddev,
-			  fabs(mean - expected) / line_time(kmode));
-
-		return false;
+		igt_warn("vblank interval differs from modeline! expected %.1fus, "
+			 "measured %1.fus +- %.3fus, difference %.1fus "
+			 "(%.1f sigma, %.1f scanlines)\n",
+			 expected, mean, stddev,
+			 fabs(mean - expected),
+			 fabs(mean - expected) / stddev,
+			 fabs(mean - expected) / line_time(kmode));
+		strict_failed = true;
+	}
+
+	if (strict_failed) {
+		/* Relaxed validation: 1% relative tolerance for accuracy and deviation
+		 *
+		 * Hardware specifications (VESA/HDMI/DP) allow ±0.5% pixel clock tolerance.
+		 * Since frame_time = (htotal × vtotal) / pixel_clock, the relationship is
+		 * inverse: ±0.5% pixel clock → ±0.5% frame time (in opposite direction).
+		 *
+		 * However, userspace measurement introduces additional overhead (~0.3-0.5%):
+		 *   - Interrupt latency and scheduling delays (5-50 μs)
+		 *   - System call overhead for read() (1-5 μs)
+		 *   - Variable delays depending on system load
+		 *
+		 * Total tolerance: ±0.5% (hardware) + ~0.5% (software) ≈ ±1.0%
+		 *
+		 * For stricter validation matching hardware spec only, use 0.005 (0.5%),
+		 * but expect potential false failures due to measurement noise.
+		 */
+		double tolerance_pct = 0.01;  /* 1% tolerance */
+		double tolerance_us = expected * tolerance_pct;
+		double relative_accuracy = accuracy / expected;
+
+		/* Check 1: 99.7% samples within 1% of the mean */
+		if (accuracy >= tolerance_us) {
+			igt_info("FAIL: vblank accuracy (%.3fus, %.2f%%) exceeds 1%% tolerance "
+				 "(%.3fus)\n",
+				 accuracy, 100 * relative_accuracy, tolerance_us);
+			return false;
+		}
+
+		/* Check 2: Mean within 1% of expected, or within 3 sigma (99.7% confidence) */
+		if (fabs(mean - expected) >= max(tolerance_us, 3.0 * stddev)) {
+			igt_info("FAIL: vblank interval differs from expected! expected %.1fus, "
+				 "measured %.1fus +- %.3fus, difference %.1fus "
+				 "(%.1f sigma, %.2f%%), "
+				 "exceeds 1%% tolerance or 3-sigma threshold\n",
+				 expected, mean, stddev,
+				 fabs(mean - expected),
+				 fabs(mean - expected) / stddev,
+				 100 * fabs(mean - expected) / expected);
+			return false;
+		}
+
+		igt_info("Passed relaxed validation (strict failed)\n");
+	} else {
+		igt_info("Passed strict validation\n");
 	}
 
 	return true;
-- 
2.43.0