[PATCH] media: mali-c55: Fix scaler factor overflow for large crop sizes

[PATCH] media: mali-c55: Fix scaler factor overflow for large crop sizes

[David Carlier]

The horizontal and vertical scaling factors multiply the crop dimensions
by MALI_C55_RSZ_SCALER_FACTOR, a Q4.20 factor of (1 << 20). Both operands
are 32-bit, so the multiplication wraps before the result is stored in
the u64 scale variables. For any crop dimension of 4096 or more (the
maximum is 8192) the value overflows; an 8192 to 4096 downscale yields a
TINC of zero, so the scaler never advances and the output is corrupted.

Cast the crop dimensions to u64 before the multiplication.

Fixes: d5f281f3dd29 ("media: mali-c55: Add Mali-C55 ISP driver")
Cc: stable@vger.kernel.org
Signed-off-by: David Carlier <devnexen@gmail.com>
---
 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
index c4f46651d..0713e7d43 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
@@ -422,8 +422,8 @@ static int mali_c55_rsz_program_resizer(struct mali_c55_resizer *rsz,
 	mali_c55_resizer_program_coefficients(rsz);
 
 	/* Program the V/H scaling factor in Q4.20 format. */
-	h_scale = crop->width * MALI_C55_RSZ_SCALER_FACTOR;
-	v_scale = crop->height * MALI_C55_RSZ_SCALER_FACTOR;
+	h_scale = (u64)crop->width * MALI_C55_RSZ_SCALER_FACTOR;
+	v_scale = (u64)crop->height * MALI_C55_RSZ_SCALER_FACTOR;
 
 	do_div(h_scale, scale->width);
 	do_div(v_scale, scale->height);
-- 
2.53.0