fix for rounding errors in cvt/gtf calculation

fix for rounding errors in cvt/gtf calculation

[Prashant Laddha]

While I was testing cvt / gtf timings against the timings given by
standards timing generator spreadsheets, there were differences in 
results for some of the formats. Those differences could be traced 
back to the rounding used to compute some of the intermediate values.
 
Following two patches fixes two such rounding errors.

[PATCH 1/2] v4l2-dv-timings: fix rounding error in vsync_bp
[PATCH 2/2] v4l2-dv-timings: fix rounding in hblank and hsync

Please review and share your comments.

Regards,
Prashant

[PATCH 1/2] v4l2-dv-timings: fix rounding error in vsync_bp calculation

[Prashant Laddha]

Changed the rounding offsets used in vsync_bp calculation in cvt and
gtf timings. The results for vsync_bp should now match with results
from timing generator spreadsheets for cvt and gtf standards.

In the vsync_bp calculation for cvt, always round down the value of
(CVT_MIN_VSYNC_BP / h_period_est) and then add 1. It thus, reflects
the equation used in timing generator spreadsheet. Using 1999999 as
rounding offset, could pontentially lead to bumping up the vsync_bp
value by extra 1.

In the vsync_bp calculations for gtf, instead of round up or round
down, round the (CVT_MIN_VSYNC_BP / h_period_est) to the nearest
integer.

Cc: Hans Verkuil <hans.verkuil@cisco.com>
Cc: Martin Bugge <marbugge@cisco.com>

Thanks to Martin Bugge <marbugge@cisco.com> for validating with
standards and suggestions on equations.

Signed-off-by: Prashant Laddha <prladdha@cisco.com>
---
 drivers/media/v4l2-core/v4l2-dv-timings.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-dv-timings.c b/drivers/media/v4l2-core/v4l2-dv-timings.c
index 80e4722..5e114ee 100644
--- a/drivers/media/v4l2-core/v4l2-dv-timings.c
+++ b/drivers/media/v4l2-core/v4l2-dv-timings.c
@@ -367,14 +367,14 @@ bool v4l2_detect_cvt(unsigned frame_height, unsigned hfreq, unsigned vsync,
 	/* Vertical */
 	if (reduced_blanking) {
 		v_fp = CVT_RB_V_FPORCH;
-		v_bp = (CVT_RB_MIN_V_BLANK * hfreq + 1999999) / 1000000;
+		v_bp = (CVT_RB_MIN_V_BLANK * hfreq) / 1000000 + 1;
 		v_bp -= vsync + v_fp;
 
 		if (v_bp < CVT_RB_MIN_V_BPORCH)
 			v_bp = CVT_RB_MIN_V_BPORCH;
 	} else {
 		v_fp = CVT_MIN_V_PORCH_RND;
-		v_bp = (CVT_MIN_VSYNC_BP * hfreq + 1999999) / 1000000 - vsync;
+		v_bp = (CVT_MIN_VSYNC_BP * hfreq) / 1000000 + 1 - vsync;
 
 		if (v_bp < CVT_MIN_V_BPORCH)
 			v_bp = CVT_MIN_V_BPORCH;
@@ -543,7 +543,7 @@ bool v4l2_detect_gtf(unsigned frame_height,
 
 	/* Vertical */
 	v_fp = GTF_V_FP;
-	v_bp = (GTF_MIN_VSYNC_BP * hfreq + 999999) / 1000000 - vsync;
+	v_bp = (GTF_MIN_VSYNC_BP * hfreq + 500000) / 1000000 - vsync;
 
 	if (scan == V4L2_DV_INTERLACED)
 		image_height = (frame_height - 2 * v_fp - 2 * vsync - 2 * v_bp)
-- 
1.9.1

[PATCH 2/2] v4l2-dv-timings: fix rounding in hblank and hsync calculation

[Prashant Laddha]

Changed the rounding calculation for hblank and hsync to match it
to equations in standards. Currently hblank and hsync are rounded
down. hblank needs to be rounded to nearest multiple of twice the
cell granularity. hsync needs to be rounded to nearest multiple of
cell granularity.

Cc: Hans Verkuil <hans.verkuil@cisco.com>
Cc: Martin Bugge <marbugge@cisco.com>
Signed-off-by: Prashant Laddha <prladdha@cisco.com>
---
 drivers/media/v4l2-core/v4l2-dv-timings.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-dv-timings.c b/drivers/media/v4l2-core/v4l2-dv-timings.c
index 5e114ee..4b8ec4e 100644
--- a/drivers/media/v4l2-core/v4l2-dv-timings.c
+++ b/drivers/media/v4l2-core/v4l2-dv-timings.c
@@ -570,14 +570,15 @@ bool v4l2_detect_gtf(unsigned frame_height,
 			(hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000) / 2) /
 			(hfreq * (100 - GTF_S_C_PRIME) + GTF_S_M_PRIME * 1000);
 
-	h_blank = h_blank - h_blank % (2 * GTF_CELL_GRAN);
+	h_blank = ((h_blank + GTF_CELL_GRAN) / (2 * GTF_CELL_GRAN)) *
+		  (2 * GTF_CELL_GRAN);
 	frame_width = image_width + h_blank;
 
 	pix_clk = (image_width + h_blank) * hfreq;
 	pix_clk = pix_clk / GTF_PXL_CLK_GRAN * GTF_PXL_CLK_GRAN;
 
 	hsync = (frame_width * 8 + 50) / 100;
-	hsync = hsync - hsync % GTF_CELL_GRAN;
+	hsync = ((hsync + GTF_CELL_GRAN / 2) / GTF_CELL_GRAN) * GTF_CELL_GRAN;
 
 	h_fp = h_blank / 2 - hsync;
 
-- 
1.9.1