[PATCH v2 1/4] drm/rect: Avoid division by zero

[PATCH v2 1/4] drm/rect: Avoid division by zero

[Ville Syrjala]

From: Ville Syrjälä <ville.syrjala@linux.intel.com>

Check for zero width/height destination rectangle in
drm_rect_clip_scaled() to avoid a division by zero.

Cc: stable@vger.kernel.org
Fixes: f96bdf564f3e ("drm/rect: Handle rounding errors in drm_rect_clip_scaled, v3.")
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Benjamin Gaignard <benjamin.gaignard@st.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Testcase: igt/kms_selftest/drm_rect_clip_scaled_div_by_zero
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
 drivers/gpu/drm/drm_rect.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/drm_rect.c b/drivers/gpu/drm/drm_rect.c
index b8363aaa9032..818738e83d06 100644
--- a/drivers/gpu/drm/drm_rect.c
+++ b/drivers/gpu/drm/drm_rect.c
@@ -54,7 +54,12 @@ EXPORT_SYMBOL(drm_rect_intersect);
 
 static u32 clip_scaled(u32 src, u32 dst, u32 clip)
 {
-	u64 tmp = mul_u32_u32(src, dst - clip);
+	u64 tmp;
+
+	if (dst == 0)
+		return 0;
+
+	tmp = mul_u32_u32(src, dst - clip);
 
 	/*
 	 * Round toward 1.0 when clipping so that we don't accidentally
-- 
2.23.0

Re: [PATCH v2 1/4] drm/rect: Avoid division by zero

[Daniel Vetter]

On Fri, Nov 22, 2019 at 07:56:20PM +0200, Ville Syrjala wrote:
> From: Ville Syrjälä <ville.syrjala@linux.intel.com>
> 
> Check for zero width/height destination rectangle in
> drm_rect_clip_scaled() to avoid a division by zero.
> 
> Cc: stable@vger.kernel.org
> Fixes: f96bdf564f3e ("drm/rect: Handle rounding errors in drm_rect_clip_scaled, v3.")
> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@st.com>
> Cc: Daniel Vetter <daniel@ffwll.ch>
> Testcase: igt/kms_selftest/drm_rect_clip_scaled_div_by_zero
> Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>

Clamping src to 0 if dst is 0 makes sense to me.

Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>

> ---
>  drivers/gpu/drm/drm_rect.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/drm_rect.c b/drivers/gpu/drm/drm_rect.c
> index b8363aaa9032..818738e83d06 100644
> --- a/drivers/gpu/drm/drm_rect.c
> +++ b/drivers/gpu/drm/drm_rect.c
> @@ -54,7 +54,12 @@ EXPORT_SYMBOL(drm_rect_intersect);
>  
>  static u32 clip_scaled(u32 src, u32 dst, u32 clip)
>  {
> -	u64 tmp = mul_u32_u32(src, dst - clip);
> +	u64 tmp;
> +
> +	if (dst == 0)
> +		return 0;
> +
> +	tmp = mul_u32_u32(src, dst - clip);
>  
>  	/*
>  	 * Round toward 1.0 when clipping so that we don't accidentally
> -- 
> 2.23.0
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch