[PATCH] drm: cast calculation to __u64 to fix potential integer overflow

[PATCH] drm: cast calculation to __u64 to fix potential integer overflow

[Gax-c]

From: Zichen Xie <zichenxie0106@gmail.com>

Like commit b0b0d811eac6 ("drm/mediatek: Fix coverity issue with
unintentional integer overflow"), directly multiply pitch and
height may lead to integer overflow. Add a cast to avoid it.

Fixes: 6d1782919dc9 ("drm/cma: Introduce drm_gem_cma_dumb_create_internal()")
Fixes: dc5698e80cf7 ("Add virtio gpu driver.")
Fixes: dc6057ecb39e ("drm/tegra: gem: dumb: pitch and size are outputs")
Signed-off-by: Zichen Xie <zichenxie0106@gmail.com>
Cc: stable@vger.kernel.org
---
 drivers/gpu/drm/drm_gem_dma_helper.c | 6 +++---
 drivers/gpu/drm/tegra/gem.c          | 2 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_dma_helper.c b/drivers/gpu/drm/drm_gem_dma_helper.c
index 870b90b78bc4..020c3b17fc02 100644
--- a/drivers/gpu/drm/drm_gem_dma_helper.c
+++ b/drivers/gpu/drm/drm_gem_dma_helper.c
@@ -272,8 +272,8 @@ int drm_gem_dma_dumb_create_internal(struct drm_file *file_priv,
 	if (args->pitch < min_pitch)
 		args->pitch = min_pitch;
 
-	if (args->size < args->pitch * args->height)
-		args->size = args->pitch * args->height;
+	if (args->size < (__u64)args->pitch * args->height)
+		args->size = (__u64)args->pitch * args->height;
 
 	dma_obj = drm_gem_dma_create_with_handle(file_priv, drm, args->size,
 						 &args->handle);
@@ -306,7 +306,7 @@ int drm_gem_dma_dumb_create(struct drm_file *file_priv,
 	struct drm_gem_dma_object *dma_obj;
 
 	args->pitch = DIV_ROUND_UP(args->width * args->bpp, 8);
-	args->size = args->pitch * args->height;
+	args->size = (__u64)args->pitch * args->height;
 
 	dma_obj = drm_gem_dma_create_with_handle(file_priv, drm, args->size,
 						 &args->handle);
diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
index d275404ad0e9..a84acdbbbe3f 100644
--- a/drivers/gpu/drm/tegra/gem.c
+++ b/drivers/gpu/drm/tegra/gem.c
@@ -548,7 +548,7 @@ int tegra_bo_dumb_create(struct drm_file *file, struct drm_device *drm,
 	struct tegra_bo *bo;
 
 	args->pitch = round_up(min_pitch, tegra->pitch_align);
-	args->size = args->pitch * args->height;
+	args->size = (__u64)args->pitch * args->height;
 
 	bo = tegra_bo_create_with_handle(file, drm, args->size, 0,
 					 &args->handle);
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 7db48d17ee3a..d5407316b12e 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -72,7 +72,7 @@ int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
 		return -EINVAL;
 
 	pitch = args->width * 4;
-	args->size = pitch * args->height;
+	args->size = (__u64)pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
 
 	params.format = virtio_gpu_translate_format(DRM_FORMAT_HOST_XRGB8888);
-- 
2.34.1

Re: [PATCH] drm: cast calculation to __u64 to fix potential integer overflow

[Thierry Reding]

[-- Attachment #1: Type: text/plain, Size: 986 bytes --]

On Tue, Dec 03, 2024 at 10:02:00AM -0600, Gax-c wrote:
> From: Zichen Xie <zichenxie0106@gmail.com>
> 
> Like commit b0b0d811eac6 ("drm/mediatek: Fix coverity issue with
> unintentional integer overflow"), directly multiply pitch and
> height may lead to integer overflow. Add a cast to avoid it.
> 
> Fixes: 6d1782919dc9 ("drm/cma: Introduce drm_gem_cma_dumb_create_internal()")
> Fixes: dc5698e80cf7 ("Add virtio gpu driver.")
> Fixes: dc6057ecb39e ("drm/tegra: gem: dumb: pitch and size are outputs")
> Signed-off-by: Zichen Xie <zichenxie0106@gmail.com>
> Cc: stable@vger.kernel.org
> ---
>  drivers/gpu/drm/drm_gem_dma_helper.c | 6 +++---
>  drivers/gpu/drm/tegra/gem.c          | 2 +-
>  drivers/gpu/drm/virtio/virtgpu_gem.c | 2 +-
>  3 files changed, 5 insertions(+), 5 deletions(-)

I don't think this can ever happen. All of these functions should only
ever be called via drm_mode_create_dumb(), which already ensures that
these won't overflow.

Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]