[PATCH] omap v4l2: fix wrong resizer calculation

[PATCH] omap v4l2: fix wrong resizer calculation

[Vladimir Pantelic]

[-- Attachment #1: Type: text/plain, Size: 0 bytes --]



[-- Attachment #2: 0001-omap-v4l2-fix-wrong-resizer-calculation.patch --]
[-- Type: text/x-diff, Size: 1722 bytes --]

>From 47473f26bda2ef2e3a5e8f0203fde0c74c412c36 Mon Sep 17 00:00:00 2001
From: Vladimir Pantelic <vladoman@gmail.com>
Date: Tue, 26 Apr 2011 09:28:11 +0200
Subject: [PATCH] omap v4l2: fix wrong resizer calculation

in omap_vout_new_crop(), "crop" is the output parameter and
therefore new_crop/try_crop should be used to calculate the
resizer values

Signed-off-by: Vladimir Pantelic <vladoman@gmail.com>
---
 drivers/media/video/omap/omap_voutlib.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/media/video/omap/omap_voutlib.c b/drivers/media/video/omap/omap_voutlib.c
index 2aa6a76..8ae7481 100644
--- a/drivers/media/video/omap/omap_voutlib.c
+++ b/drivers/media/video/omap/omap_voutlib.c
@@ -193,7 +193,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		return -EINVAL;
 
 	if (cpu_is_omap24xx()) {
-		if (crop->height != win->w.height) {
+		if (try_crop.height != win->w.height) {
 			/* If we're resizing vertically, we can't support a
 			 * crop width wider than 768 pixels.
 			 */
@@ -202,7 +202,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		}
 	}
 	/* vertical resizing */
-	vresize = (1024 * crop->height) / win->w.height;
+	vresize = (1024 * try_crop.height) / win->w.height;
 	if (cpu_is_omap24xx() && (vresize > 2048))
 		vresize = 2048;
 	else if (cpu_is_omap34xx() && (vresize > 4096))
@@ -221,7 +221,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 			try_crop.height = 2;
 	}
 	/* horizontal resizing */
-	hresize = (1024 * crop->width) / win->w.width;
+	hresize = (1024 * try_crop.width) / win->w.width;
 	if (cpu_is_omap24xx() && (hresize > 2048))
 		hresize = 2048;
 	else if (cpu_is_omap34xx() && (hresize > 4096))
-- 
1.6.0.2

Re: [PATCH] omap v4l2: fix wrong resizer calculation

[Vladimir Pantelic]

[-- Attachment #1: Type: text/plain, Size: 5 bytes --]

ping

[-- Attachment #2: 0001-omap-v4l2-fix-wrong-resizer-calculation.patch --]
[-- Type: text/x-diff, Size: 1722 bytes --]

>From 47473f26bda2ef2e3a5e8f0203fde0c74c412c36 Mon Sep 17 00:00:00 2001
From: Vladimir Pantelic <vladoman@gmail.com>
Date: Tue, 26 Apr 2011 09:28:11 +0200
Subject: [PATCH] omap v4l2: fix wrong resizer calculation

in omap_vout_new_crop(), "crop" is the output parameter and
therefore new_crop/try_crop should be used to calculate the
resizer values

Signed-off-by: Vladimir Pantelic <vladoman@gmail.com>
---
 drivers/media/video/omap/omap_voutlib.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/media/video/omap/omap_voutlib.c b/drivers/media/video/omap/omap_voutlib.c
index 2aa6a76..8ae7481 100644
--- a/drivers/media/video/omap/omap_voutlib.c
+++ b/drivers/media/video/omap/omap_voutlib.c
@@ -193,7 +193,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		return -EINVAL;
 
 	if (cpu_is_omap24xx()) {
-		if (crop->height != win->w.height) {
+		if (try_crop.height != win->w.height) {
 			/* If we're resizing vertically, we can't support a
 			 * crop width wider than 768 pixels.
 			 */
@@ -202,7 +202,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 		}
 	}
 	/* vertical resizing */
-	vresize = (1024 * crop->height) / win->w.height;
+	vresize = (1024 * try_crop.height) / win->w.height;
 	if (cpu_is_omap24xx() && (vresize > 2048))
 		vresize = 2048;
 	else if (cpu_is_omap34xx() && (vresize > 4096))
@@ -221,7 +221,7 @@ int omap_vout_new_crop(struct v4l2_pix_format *pix,
 			try_crop.height = 2;
 	}
 	/* horizontal resizing */
-	hresize = (1024 * crop->width) / win->w.width;
+	hresize = (1024 * try_crop.width) / win->w.width;
 	if (cpu_is_omap24xx() && (hresize > 2048))
 		hresize = 2048;
 	else if (cpu_is_omap34xx() && (hresize > 4096))
-- 
1.6.0.2

RE: [PATCH] omap v4l2: fix wrong resizer calculation

[Hiremath, Vaibhav]

> -----Original Message-----
> From: Vladimir Pantelic [mailto:vladoman@gmail.com]
> Sent: Wednesday, June 15, 2011 3:46 PM
> To: Kernel development list
> Cc: linux-omap@vger.kernel.org; Hiremath, Vaibhav
> Subject: Re: [PATCH] omap v4l2: fix wrong resizer calculation
> 
> ping
[Hiremath, Vaibhav] Sorry for delayed response, somehow I completely missed this patch.

Thanks for finding this bug, the patch looks ok to me. Only one suggestion I have is to update the description of patch commit, request you to change like following - 

----
The omap_vout_new_crop() function has possible bug, uses uninitialized
variable "crop.width/height" which is actually output of the function. 
Instead we should be using "try_crop.width/height" to calculate the resizer value.
----

Rest everything looks ok, while submitting patch again you can add my ack; I will queue to my Arago repo.

Also I have tested this patch on OMAP3EVM platform, and it seems to be working fine.

Thanks,
Vaibhav