[PATCH] media: atomisp: Avoid picking too big sensor resolution

[PATCH] media: atomisp: Avoid picking too big sensor resolution

[Hans de Goede]

atomisp_try_fmt() is limiting the width of the requested resolution to 1920
before calling the sensor's try_fmt() method. But it is not limiting
the height. In case of the old mode-list based t4ka3 driver which has
a mode list of:

736x496
896x736
1936x1096
3280x2464

This results in 3280x2464 being selected when try_fmt is called
with a requested resolution of 3280x2464, which is not supported because
its width > 1920 .

Fix this by also limiting the height when in preview mode.

Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/staging/media/atomisp/pci/atomisp_cmd.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.c b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
index 02ccf80e6559..10462c5db36f 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_cmd.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
@@ -3784,9 +3784,14 @@ int atomisp_try_fmt(struct atomisp_device *isp, struct v4l2_pix_format *f,
 			return -EINVAL;
 	}
 
-	/* The preview pipeline does not support width > 1920 */
-	if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW)
+	/*
+	 * The preview pipeline does not support width > 1920. Also limit height
+	 * to avoid sensor drivers still picking a too wide resolution.
+	 */
+	if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW) {
 		f->width = min_t(u32, f->width, 1920);
+		f->height = min_t(u32, f->height, 1440);
+	}
 
 	/*
 	 * atomisp_set_fmt() will set the sensor resolution to the requested
-- 
2.47.0

Re: [PATCH] media: atomisp: Avoid picking too big sensor resolution

[Andy Shevchenko]

On Wed, Nov 06, 2024 at 10:55:09PM +0100, Hans de Goede wrote:
> atomisp_try_fmt() is limiting the width of the requested resolution to 1920
> before calling the sensor's try_fmt() method. But it is not limiting
> the height. In case of the old mode-list based t4ka3 driver which has
> a mode list of:
> 
> 736x496
> 896x736
> 1936x1096
> 3280x2464
> 
> This results in 3280x2464 being selected when try_fmt is called
> with a requested resolution of 3280x2464, which is not supported because
> its width > 1920 .
> 
> Fix this by also limiting the height when in preview mode.

Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

...

> +	/*
> +	 * The preview pipeline does not support width > 1920. Also limit height
> +	 * to avoid sensor drivers still picking a too wide resolution.
> +	 */
> +	if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW) {
>  		f->width = min_t(u32, f->width, 1920);
> +		f->height = min_t(u32, f->height, 1440);

Perhaps umin() instead of min_t() in both cases?

> +	}

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH] media: atomisp: Avoid picking too big sensor resolution

[Andy Shevchenko]

On Thu, Nov 07, 2024 at 10:09:26AM +0200, Andy Shevchenko wrote:
> On Wed, Nov 06, 2024 at 10:55:09PM +0100, Hans de Goede wrote:

...

> >  		f->width = min_t(u32, f->width, 1920);
> > +		f->height = min_t(u32, f->height, 1440);
> 
> Perhaps umin() instead of min_t() in both cases?

...or min(..., 1920U); and similar for the second one.

-- 
With Best Regards,
Andy Shevchenko