Re: [PATCH i-g-t v2] tests/kms_setmode: Relax vblank timing tolerance from 1 scanline to 1%

["Ville Syrjälä" <ville.syrjala@linux.intel.com>]

On Thu, Apr 16, 2026 at 06:00:58PM +0300, Ville Syrjälä wrote:
> On Thu, Apr 16, 2026 at 10:45:28PM +0800, Jason-JH Lin wrote:
> > The original check_timings() validation required vblank intervals to be
> > within 1 scanline (~15us for 1080p60) which is overly strict and causes
> > false failures due to userspace measurement overhead.
> > 
> > Changes:
> > - Keep original strict checks (1 scanline, 1.718 sigma) as warnings only
> > - Add relaxed validation with 1% relative tolerance for pass/fail
> > - Use 3 sigma threshold (99.7% confidence) instead of 1.718 sigma (90%)
> > 
> > Rationale:
> > Hardware specs (VESA/HDMI/DP) allow ±0.5% pixel clock tolerance, which
> > translates to ±0.5% frame time due to inverse relationship. However,
> > userspace measurement via read() syscall adds ~0.3-0.5% overhead from:
> 
> read() overhead is irrelevant here. This is measuring the accuracy
> of the kernel generated vblank event timestamps. If you are having
> problems with this then that would indicate your kernel driver isn't
> generating particularly good timestamps, or it's using a dotclock that
> doesn't quite match what userspace specified originally.
> 
> > - Interrupt latency and scheduling delays (5-50 us)
> > - System call overhead (1-5 us)
> > - Variable delays depending on system load
> > 
> > Total tolerance: ±0.5% (hardware) + ~0.5% (software) = ±1.0%
> > 
> > This makes the test ~11x more tolerant while still detecting real timing
> > issues and aligning with industry hardware specifications.
> > 
> > Signed-off-by: Jason-JH Lin <jason-jh.lin@mediatek.com>
> > Suggested-by: Jeevain B <jeevan.b@intel.com>
> > ---
> >  tests/kms_setmode.c | 71 +++++++++++++++++++++++++++++++++++++--------
> >  1 file changed, 59 insertions(+), 12 deletions(-)
> > 
> > diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
> > index 1f2849bc26cf..b452f8a9e997 100644
> > --- a/tests/kms_setmode.c
> > +++ b/tests/kms_setmode.c
> > @@ -495,6 +495,7 @@ static bool check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> >  	double mean;
> >  	double stddev;
> >  	int n;
> > +	bool strict_failed = false;
> >  
> >  	memset(&wait, 0, sizeof(wait));
> >  	wait.request.type = kmstest_get_vbl_flag(crtc_idx);
> > @@ -563,10 +564,9 @@ static bool check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> >  
> >  	/* 99.7% samples within one scanline on each side of mean */
> >  	if (accuracy >= line_time(kmode)) {
> > -		igt_info("vblank accuracy (%.3fus, %.1f%%) worse than a scanline (%.3fus)\n",
> > -			  accuracy, 100 * accuracy / mean, line_time(kmode));
> > -
> > -		return false;
> > +		igt_warn("vblank accuracy (%.3fus, %.1f%%) worse than a scanline (%.3fus)\n",
> > +			 accuracy, 100 * accuracy / mean, line_time(kmode));
> > +		strict_failed = true;

Failing this would indicate the driver isn't generating particularly
good timestamps.

> >  	}
> >  
> >  	/* At least 90% of frame times fall within the one scanline on each
> > @@ -591,14 +591,61 @@ static bool check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> >  	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
> >  	 */
> >  	if (fabs(mean - expected) >= max(line_time(kmode), 1.718 * stddev)) {

And assuming the timestamps were good, failing this would indicate the
actual timings don't quite match what userspace specified (eg. dotclock
could be a bit off). I suppose we should allow that VESA specified 
.5% error here.

> > -		igt_info("vblank interval differs from modeline! expected %.1fus, "
> > -			 "measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> > -			  expected, mean, stddev,
> > -			  fabs(mean - expected),
> > -			  fabs(mean - expected) / stddev,
> > -			  fabs(mean - expected) / line_time(kmode));
> > -
> > -		return false;
> > +		igt_warn("vblank interval differs from modeline! expected %.1fus, "
> > +			 "measured %1.fus +- %.3fus, difference %.1fus "
> > +			 "(%.1f sigma, %.1f scanlines)\n",
> > +			 expected, mean, stddev,
> > +			 fabs(mean - expected),
> > +			 fabs(mean - expected) / stddev,
> > +			 fabs(mean - expected) / line_time(kmode));
> > +		strict_failed = true;
> > +	}
> > +
> > +	if (strict_failed) {
> > +		/* Relaxed validation: 1% relative tolerance for accuracy and deviation
> > +		 *
> > +		 * Hardware specifications (VESA/HDMI/DP) allow ±0.5% pixel clock tolerance.
> > +		 * Since frame_time = (htotal × vtotal) / pixel_clock, the relationship is
> > +		 * inverse: ±0.5% pixel clock → ±0.5% frame time (in opposite direction).
> > +		 *
> > +		 * However, userspace measurement introduces additional overhead (~0.3-0.5%):
> > +		 *   - Interrupt latency and scheduling delays (5-50 μs)
> > +		 *   - System call overhead for read() (1-5 μs)
> > +		 *   - Variable delays depending on system load
> > +		 *
> > +		 * Total tolerance: ±0.5% (hardware) + ~0.5% (software) ≈ ±1.0%
> > +		 *
> > +		 * For stricter validation matching hardware spec only, use 0.005 (0.5%),
> > +		 * but expect potential false failures due to measurement noise.
> > +		 */
> > +		double tolerance_pct = 0.01;  /* 1% tolerance */
> > +		double tolerance_us = expected * tolerance_pct;
> > +		double relative_accuracy = accuracy / expected;
> > +
> > +		/* Check 1: 99.7% samples within 1% of the mean */
> > +		if (accuracy >= tolerance_us) {
> > +			igt_info("FAIL: vblank accuracy (%.3fus, %.2f%%) exceeds 1%% tolerance "
> > +				 "(%.3fus)\n",
> > +				 accuracy, 100 * relative_accuracy, tolerance_us);
> > +			return false;
> > +		}
> > +
> > +		/* Check 2: Mean within 1% of expected, or within 3 sigma (99.7% confidence) */
> > +		if (fabs(mean - expected) >= max(tolerance_us, 3.0 * stddev)) {
> > +			igt_info("FAIL: vblank interval differs from expected! expected %.1fus, "
> > +				 "measured %.1fus +- %.3fus, difference %.1fus "
> > +				 "(%.1f sigma, %.2f%%), "
> > +				 "exceeds 1%% tolerance or 3-sigma threshold\n",
> > +				 expected, mean, stddev,
> > +				 fabs(mean - expected),
> > +				 fabs(mean - expected) / stddev,
> > +				 100 * fabs(mean - expected) / expected);
> > +			return false;
> > +		}
> > +
> > +		igt_info("Passed relaxed validation (strict failed)\n");
> > +	} else {
> > +		igt_info("Passed strict validation\n");
> >  	}
> >  
> >  	return true;
> > -- 
> > 2.43.0
> 
> -- 
> Ville Syrjälä
> Intel

-- 
Ville Syrjälä
Intel