[PATCH 0/2] drm/amdgpu,radeon: fix integer overflow in pitch alignment

[PATCH 0/2] drm/amdgpu,radeon: fix integer overflow in pitch alignment

[Werner Kasselman]

Both amdgpu_gem_align_pitch() and radeon_align_pitch() use signed int
for the pitch calculation. When alignment rounding pushes the width to
a boundary value, 'aligned * cpp' overflows signed 32-bit int to 0.

This defeats the overflow guards in drm_mode_create_dumb() because
these drivers bypass drm_mode_size_dumb() and perform their own
alignment rounding, which can push the pitch past the pre-validated
range.

A zero pitch propagates to a zero-size GEM object allocation reachable
from unprivileged userspace via DRM_IOCTL_MODE_CREATE_DUMB on the
render node.

Both drivers need the same fix: add an overflow check in the alignment
function and reject zero pitch/size in the dumb_create callback. The
proper long-term fix is to convert both drivers to use
drm_mode_size_dumb() as Thomas Zimmermann's series is doing for other
drivers.

Werner Kasselman (2):
  drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()
  drm/radeon: fix integer overflow in radeon_align_pitch()

 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 13 +++++++++++++
 drivers/gpu/drm/radeon/radeon_gem.c     |  9 +++++++++
 2 files changed, 22 insertions(+)

-- 
2.43.0

[PATCH 1/2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

[Werner Kasselman]

amdgpu_gem_align_pitch() uses signed int for the pitch calculation.
When alignment rounding pushes the width to a boundary value (e.g.,
2^30 for cpp=4), the multiplication 'aligned * cpp' overflows signed
32-bit int, producing 0 or a negative value.

The overflow guard in drm_mode_create_dumb() validates width * cpp
BEFORE the driver callback, but amdgpu_mode_dumb_create() bypasses the
generic drm_mode_size_dumb() helper and performs its own alignment
rounding, which can push the pitch past the pre-validated range.

A zero pitch propagates to a zero-size GEM object allocation via
amdgpu_gem_object_create(). The 0-byte BO passes
amdgpu_bo_validate_size() (since 0 < man->size) and is returned to
userspace with a valid handle. This object can then be mmap'd or
referenced in GPU command submissions, potentially causing out-of-bounds
access to adjacent slab memory.

DRM_IOCTL_MODE_CREATE_DUMB requires no DRM authentication, so any local
user with access to /dev/dri/renderD* can trigger this with e.g.
width=1073741760, bpp=32, height=1.

Add an overflow check in amdgpu_gem_align_pitch() to detect when
'aligned * cpp' would exceed INT_MAX, returning 0 in that case. Add
corresponding checks in amdgpu_mode_dumb_create() to reject pitch=0
and size=0 with -EINVAL.

The proper long-term fix is to convert amdgpu to use
drm_mode_size_dumb() which centralizes pitch/size calculation with
proper overflow guards, as is being done for other drivers in Thomas
Zimmermann's dumb-buffer series.

Found via AST-based call-graph analysis using sqry.

Fixes: 087451f372bf ("drm/amdgpu: use generic fb helpers instead of setting up AMD own's.")
Cc: stable@vger.kernel.org
Signed-off-by: Werner Kasselman <werner@verivus.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index a6107109a2b8..b4341abba20c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -1246,6 +1246,15 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
 
 	aligned += pitch_mask;
 	aligned &= ~pitch_mask;
+
+	/* Sanity check to avoid integer overflow in aligned * cpp.
+	 * The caller (drm_mode_create_dumb) validates width * cpp fits
+	 * in u32 before alignment, but rounding up can push aligned
+	 * past INT_MAX / cpp, causing signed overflow to 0 or negative.
+	 */
+	if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+		return 0;
+
 	return aligned * cpp;
 }
 
@@ -1273,8 +1282,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = amdgpu_gem_align_pitch(adev, args->width,
 					     DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 	domain = amdgpu_bo_get_preferred_domain(adev,
 				amdgpu_display_supported_domains(adev, flags));
 	r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
-- 
2.43.0

Re: [PATCH 1/2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

[Alex Deucher]

On Tue, Apr 7, 2026 at 3:41 AM Werner Kasselman <werner@verivus.ai> wrote:
>
> amdgpu_gem_align_pitch() uses signed int for the pitch calculation.

Can you convert amdgpu_gem_align_pitch() to use unsigned ints?  The
width passed to it is always unsigned.  That would simplify things.

Alex

> When alignment rounding pushes the width to a boundary value (e.g.,
> 2^30 for cpp=4), the multiplication 'aligned * cpp' overflows signed
> 32-bit int, producing 0 or a negative value.
>
> The overflow guard in drm_mode_create_dumb() validates width * cpp
> BEFORE the driver callback, but amdgpu_mode_dumb_create() bypasses the
> generic drm_mode_size_dumb() helper and performs its own alignment
> rounding, which can push the pitch past the pre-validated range.
>
> A zero pitch propagates to a zero-size GEM object allocation via
> amdgpu_gem_object_create(). The 0-byte BO passes
> amdgpu_bo_validate_size() (since 0 < man->size) and is returned to
> userspace with a valid handle. This object can then be mmap'd or
> referenced in GPU command submissions, potentially causing out-of-bounds
> access to adjacent slab memory.
>
> DRM_IOCTL_MODE_CREATE_DUMB requires no DRM authentication, so any local
> user with access to /dev/dri/renderD* can trigger this with e.g.
> width=1073741760, bpp=32, height=1.
>
> Add an overflow check in amdgpu_gem_align_pitch() to detect when
> 'aligned * cpp' would exceed INT_MAX, returning 0 in that case. Add
> corresponding checks in amdgpu_mode_dumb_create() to reject pitch=0
> and size=0 with -EINVAL.
>
> The proper long-term fix is to convert amdgpu to use
> drm_mode_size_dumb() which centralizes pitch/size calculation with
> proper overflow guards, as is being done for other drivers in Thomas
> Zimmermann's dumb-buffer series.
>
> Found via AST-based call-graph analysis using sqry.
>
> Fixes: 087451f372bf ("drm/amdgpu: use generic fb helpers instead of setting up AMD own's.")
> Cc: stable@vger.kernel.org
> Signed-off-by: Werner Kasselman <werner@verivus.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> index a6107109a2b8..b4341abba20c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> @@ -1246,6 +1246,15 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
>
>         aligned += pitch_mask;
>         aligned &= ~pitch_mask;
> +
> +       /* Sanity check to avoid integer overflow in aligned * cpp.
> +        * The caller (drm_mode_create_dumb) validates width * cpp fits
> +        * in u32 before alignment, but rounding up can push aligned
> +        * past INT_MAX / cpp, causing signed overflow to 0 or negative.
> +        */
> +       if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
> +               return 0;
> +
>         return aligned * cpp;
>  }
>
> @@ -1273,8 +1282,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
>
>         args->pitch = amdgpu_gem_align_pitch(adev, args->width,
>                                              DIV_ROUND_UP(args->bpp, 8), 0);
> +       if (!args->pitch)
> +               return -EINVAL;
>         args->size = (u64)args->pitch * args->height;
>         args->size = ALIGN(args->size, PAGE_SIZE);
> +       if (!args->size)
> +               return -EINVAL;
>         domain = amdgpu_bo_get_preferred_domain(adev,
>                                 amdgpu_display_supported_domains(adev, flags));
>         r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
> --
> 2.43.0
>

[PATCH v2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

[Werner Kasselman]

amdgpu_gem_align_pitch() is passed u32 width and cpp from dumb buffer
creation but uses signed int internally.  The round-up add and the
aligned * cpp multiplication can overflow, returning zero or a negative
pitch.  A zero pitch propagates to a zero-sized GEM object allocation
that reaches userspace via DRM_IOCTL_MODE_CREATE_DUMB.

Switch the helper to unsigned int and use check_add_overflow() /
check_mul_overflow() so wraparound returns zero.  Reject a zero pitch
or size in amdgpu_mode_dumb_create() rather than allocating a zero-
byte BO.

Fixes: 8e911ab770f7 ("drm: amdgpu: Replace drm_fb_get_bpp_depth() with drm_format_plane_cpp()")
Cc: stable@vger.kernel.org
Signed-off-by: Werner Kasselman <werner@verivus.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index a6107109a2b8..0d9309f792a4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -27,6 +27,7 @@
  */
 #include <linux/ktime.h>
 #include <linux/module.h>
+#include <linux/overflow.h>
 #include <linux/pagemap.h>
 #include <linux/pci.h>
 #include <linux/dma-buf.h>
@@ -1223,13 +1224,14 @@ int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
 	return ret;
 }
 
-static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
-				  int width,
-				  int cpp,
-				  bool tiled)
+static unsigned int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
+					   unsigned int width,
+					   unsigned int cpp,
+					   bool tiled)
 {
-	int aligned = width;
-	int pitch_mask = 0;
+	unsigned int aligned = width;
+	unsigned int pitch_mask = 0;
+	unsigned int pitch;
 
 	switch (cpp) {
 	case 1:
@@ -1244,9 +1246,12 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
 		break;
 	}
 
-	aligned += pitch_mask;
+	if (check_add_overflow(aligned, pitch_mask, &aligned))
+		return 0;
 	aligned &= ~pitch_mask;
-	return aligned * cpp;
+	if (check_mul_overflow(aligned, cpp, &pitch))
+		return 0;
+	return pitch;
 }
 
 int amdgpu_mode_dumb_create(struct drm_file *file_priv,
@@ -1273,8 +1278,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = amdgpu_gem_align_pitch(adev, args->width,
 					     DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 	domain = amdgpu_bo_get_preferred_domain(adev,
 				amdgpu_display_supported_domains(adev, flags));
 	r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
-- 
2.43.0

Re: [PATCH v2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

[Christian König]

On 4/14/26 07:08, Werner Kasselman wrote:
> amdgpu_gem_align_pitch() is passed u32 width and cpp from dumb buffer
> creation but uses signed int internally.  The round-up add and the
> aligned * cpp multiplication can overflow, returning zero or a negative
> pitch.  A zero pitch propagates to a zero-sized GEM object allocation
> that reaches userspace via DRM_IOCTL_MODE_CREATE_DUMB.
> 
> Switch the helper to unsigned int and use check_add_overflow() /
> check_mul_overflow() so wraparound returns zero.  Reject a zero pitch
> or size in amdgpu_mode_dumb_create() rather than allocating a zero-
> byte BO.
> 
> Fixes: 8e911ab770f7 ("drm: amdgpu: Replace drm_fb_get_bpp_depth() with drm_format_plane_cpp()")
> Cc: stable@vger.kernel.org

The patch looks valid to me, but I think we can drop this CC: stable.

Since dump buffers are a kernel only interface the fix has no practical relevance and is only of cosmetic nature.

Regards,
Christian.

> Signed-off-by: Werner Kasselman <werner@verivus.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 25 +++++++++++++++++--------
>  1 file changed, 17 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> index a6107109a2b8..0d9309f792a4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> @@ -27,6 +27,7 @@
>   */
>  #include <linux/ktime.h>
>  #include <linux/module.h>
> +#include <linux/overflow.h>
>  #include <linux/pagemap.h>
>  #include <linux/pci.h>
>  #include <linux/dma-buf.h>
> @@ -1223,13 +1224,14 @@ int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
>  	return ret;
>  }
>  
> -static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
> -				  int width,
> -				  int cpp,
> -				  bool tiled)
> +static unsigned int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
> +					   unsigned int width,
> +					   unsigned int cpp,
> +					   bool tiled)
>  {
> -	int aligned = width;
> -	int pitch_mask = 0;
> +	unsigned int aligned = width;
> +	unsigned int pitch_mask = 0;
> +	unsigned int pitch;
>  
>  	switch (cpp) {
>  	case 1:
> @@ -1244,9 +1246,12 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
>  		break;
>  	}
>  
> -	aligned += pitch_mask;
> +	if (check_add_overflow(aligned, pitch_mask, &aligned))
> +		return 0;
>  	aligned &= ~pitch_mask;
> -	return aligned * cpp;
> +	if (check_mul_overflow(aligned, cpp, &pitch))
> +		return 0;
> +	return pitch;
>  }
>  
>  int amdgpu_mode_dumb_create(struct drm_file *file_priv,
> @@ -1273,8 +1278,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
>  
>  	args->pitch = amdgpu_gem_align_pitch(adev, args->width,
>  					     DIV_ROUND_UP(args->bpp, 8), 0);
> +	if (!args->pitch)
> +		return -EINVAL;
>  	args->size = (u64)args->pitch * args->height;
>  	args->size = ALIGN(args->size, PAGE_SIZE);
> +	if (!args->size)
> +		return -EINVAL;
>  	domain = amdgpu_bo_get_preferred_domain(adev,
>  				amdgpu_display_supported_domains(adev, flags));
>  	r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,

[PATCH 2/2] drm/radeon: fix integer overflow in radeon_align_pitch()

[Werner Kasselman]

radeon_align_pitch() has the same integer overflow as amdgpu's variant:
'aligned * cpp' can overflow signed int to 0 when alignment rounding
pushes the width past INT_MAX/cpp. This produces a 0-byte GEM buffer
via radeon_mode_dumb_create(), reachable from unprivileged userspace
via DRM_IOCTL_MODE_CREATE_DUMB on the render node.

Add an overflow check in radeon_align_pitch() and reject zero pitch/size
in radeon_mode_dumb_create().

Found via AST-based call-graph analysis using sqry.

Fixes: ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")
Cc: stable@vger.kernel.org
Signed-off-by: Werner Kasselman <werner@verivus.com>
---
 drivers/gpu/drm/radeon/radeon_gem.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index 20fc87409f2e..2cd179fef347 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -828,6 +828,11 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 
 	aligned += pitch_mask;
 	aligned &= ~pitch_mask;
+
+	/* Guard against integer overflow in aligned * cpp. */
+	if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+		return 0;
+
 	return aligned * cpp;
 }
 
@@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = radeon_align_pitch(rdev, args->width,
 					 DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 
 	r = radeon_gem_object_create(rdev, args->size, 0,
 				     RADEON_GEM_DOMAIN_VRAM, 0,
-- 
2.43.0

Re: [PATCH 2/2] drm/radeon: fix integer overflow in radeon_align_pitch()

[Alex Deucher]

On Tue, Apr 7, 2026 at 3:41 AM Werner Kasselman <werner@verivus.ai> wrote:
>
> radeon_align_pitch() has the same integer overflow as amdgpu's variant:
> 'aligned * cpp' can overflow signed int to 0 when alignment rounding
> pushes the width past INT_MAX/cpp. This produces a 0-byte GEM buffer
> via radeon_mode_dumb_create(), reachable from unprivileged userspace
> via DRM_IOCTL_MODE_CREATE_DUMB on the render node.
>
> Add an overflow check in radeon_align_pitch() and reject zero pitch/size
> in radeon_mode_dumb_create().
>
> Found via AST-based call-graph analysis using sqry.
>
> Fixes: ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")
> Cc: stable@vger.kernel.org
> Signed-off-by: Werner Kasselman <werner@verivus.com>

Can you fix this up similar to the amdgpu patch?

Thanks,

Alex

> ---
>  drivers/gpu/drm/radeon/radeon_gem.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
> index 20fc87409f2e..2cd179fef347 100644
> --- a/drivers/gpu/drm/radeon/radeon_gem.c
> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
> @@ -828,6 +828,11 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
>
>         aligned += pitch_mask;
>         aligned &= ~pitch_mask;
> +
> +       /* Guard against integer overflow in aligned * cpp. */
> +       if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
> +               return 0;
> +
>         return aligned * cpp;
>  }
>
> @@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
>
>         args->pitch = radeon_align_pitch(rdev, args->width,
>                                          DIV_ROUND_UP(args->bpp, 8), 0);
> +       if (!args->pitch)
> +               return -EINVAL;
>         args->size = (u64)args->pitch * args->height;
>         args->size = ALIGN(args->size, PAGE_SIZE);
> +       if (!args->size)
> +               return -EINVAL;
>
>         r = radeon_gem_object_create(rdev, args->size, 0,
>                                      RADEON_GEM_DOMAIN_VRAM, 0,
> --
> 2.43.0
>

[PATCH v2] drm/radeon: fix integer overflow in radeon_align_pitch()

[Werner Kasselman]

radeon_align_pitch() has the same kind of overflow issue as the old
amdgpu helper: the alignment round-up add and the final 'aligned * cpp'
calculation can overflow signed int.

If that wraps to 0, radeon_mode_dumb_create() can end up with an invalid
pitch value from DRM_IOCTL_MODE_CREATE_DUMB.

Fix this by using check_add_overflow() for the alignment round-up and
check_mul_overflow() for the final pitch calculation, returning 0 on
overflow.

Found via AST-based call-graph analysis using sqry.

Fixes: ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")
Cc: stable@vger.kernel.org
Signed-off-by: Werner Kasselman <werner@verivus.com>
---
v2:
- Use overflow helpers like amdgpu.
- Drop the stale zero pitch/size change from the original submission.
- Fix the changelog wording around reachability.

 drivers/gpu/drm/radeon/radeon_gem.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index 2cd179fef347..8ce180e22d1d 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -28,6 +28,7 @@
 
 #include <linux/debugfs.h>
 #include <linux/iosys-map.h>
+#include <linux/overflow.h>
 #include <linux/pci.h>
 
 #include <drm/drm_device.h>
@@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 	int aligned = width;
 	int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
 	int pitch_mask = 0;
+	int pitch;
 
 	switch (cpp) {
 	case 1:
@@ -826,14 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 		break;
 	}
 
-	aligned += pitch_mask;
+	if (check_add_overflow(aligned, pitch_mask, &aligned))
+		return 0;
 	aligned &= ~pitch_mask;
-
-	/* Guard against integer overflow in aligned * cpp. */
-	if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+	if (check_mul_overflow(aligned, cpp, &pitch))
 		return 0;
-
-	return aligned * cpp;
+	return pitch;
 }
 
 int radeon_mode_dumb_create(struct drm_file *file_priv,
-- 
2.43.0

Re: [PATCH v2] drm/radeon: fix integer overflow in radeon_align_pitch()

[Alex Deucher]

Can you squash this with the previous radeon patch?  I only applied
the amdgpu patch at this point.

Alex

On Tue, Apr 14, 2026 at 5:14 PM Werner Kasselman <werner@verivus.ai> wrote:
>
> radeon_align_pitch() has the same kind of overflow issue as the old
> amdgpu helper: the alignment round-up add and the final 'aligned * cpp'
> calculation can overflow signed int.
>
> If that wraps to 0, radeon_mode_dumb_create() can end up with an invalid
> pitch value from DRM_IOCTL_MODE_CREATE_DUMB.
>
> Fix this by using check_add_overflow() for the alignment round-up and
> check_mul_overflow() for the final pitch calculation, returning 0 on
> overflow.
>
> Found via AST-based call-graph analysis using sqry.
>
> Fixes: ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")
> Cc: stable@vger.kernel.org
> Signed-off-by: Werner Kasselman <werner@verivus.com>
> ---
> v2:
> - Use overflow helpers like amdgpu.
> - Drop the stale zero pitch/size change from the original submission.
> - Fix the changelog wording around reachability.
>
>  drivers/gpu/drm/radeon/radeon_gem.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
> index 2cd179fef347..8ce180e22d1d 100644
> --- a/drivers/gpu/drm/radeon/radeon_gem.c
> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
> @@ -28,6 +28,7 @@
>
>  #include <linux/debugfs.h>
>  #include <linux/iosys-map.h>
> +#include <linux/overflow.h>
>  #include <linux/pci.h>
>
>  #include <drm/drm_device.h>
> @@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
>         int aligned = width;
>         int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
>         int pitch_mask = 0;
> +       int pitch;
>
>         switch (cpp) {
>         case 1:
> @@ -826,14 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
>                 break;
>         }
>
> -       aligned += pitch_mask;
> +       if (check_add_overflow(aligned, pitch_mask, &aligned))
> +               return 0;
>         aligned &= ~pitch_mask;
> -
> -       /* Guard against integer overflow in aligned * cpp. */
> -       if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
> +       if (check_mul_overflow(aligned, cpp, &pitch))
>                 return 0;
> -
> -       return aligned * cpp;
> +       return pitch;
>  }
>
>  int radeon_mode_dumb_create(struct drm_file *file_priv,
> --
> 2.43.0

[PATCH v3] drm/radeon: fix integer overflow in radeon_align_pitch()

[Werner Kasselman]

radeon_align_pitch() has the same kind of overflow issue as the old
amdgpu helper: both the alignment round-up add and the final
'aligned * cpp' calculation can overflow signed int.

If that wraps, radeon_mode_dumb_create() can end up returning an
invalid pitch or creating a zero-sized dumb buffer.

Fix this by using check_add_overflow() for the alignment round-up and
check_mul_overflow() for the final pitch calculation, returning 0 on
overflow. Also reject zero pitch and size in
radeon_mode_dumb_create().

Found via AST-based call-graph analysis using sqry.

Fixes: ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")
Cc: stable@vger.kernel.org
Signed-off-by: Werner Kasselman <werner@verivus.com>
---
v3:
- Squash this fix with the earlier zero pitch/size validation change.
- Use overflow helpers for both the alignment round-up and final
  pitch calculation.

 drivers/gpu/drm/radeon/radeon_gem.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index 20fc87409f2e..8ce180e22d1d 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -28,6 +28,7 @@
 
 #include <linux/debugfs.h>
 #include <linux/iosys-map.h>
+#include <linux/overflow.h>
 #include <linux/pci.h>
 
 #include <drm/drm_device.h>
@@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 	int aligned = width;
 	int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
 	int pitch_mask = 0;
+	int pitch;
 
 	switch (cpp) {
 	case 1:
@@ -826,9 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 		break;
 	}
 
-	aligned += pitch_mask;
+	if (check_add_overflow(aligned, pitch_mask, &aligned))
+		return 0;
 	aligned &= ~pitch_mask;
-	return aligned * cpp;
+	if (check_mul_overflow(aligned, cpp, &pitch))
+		return 0;
+	return pitch;
 }
 
 int radeon_mode_dumb_create(struct drm_file *file_priv,
@@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = radeon_align_pitch(rdev, args->width,
 					 DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 
 	r = radeon_gem_object_create(rdev, args->size, 0,
 				     RADEON_GEM_DOMAIN_VRAM, 0,
-- 
2.43.0

Re: [PATCH v3] drm/radeon: fix integer overflow in radeon_align_pitch()

[Alex Deucher]

Applied.  Thanks!

On Wed, Apr 15, 2026 at 6:14 PM Werner Kasselman <werner@verivus.ai> wrote:
>
> radeon_align_pitch() has the same kind of overflow issue as the old
> amdgpu helper: both the alignment round-up add and the final
> 'aligned * cpp' calculation can overflow signed int.
>
> If that wraps, radeon_mode_dumb_create() can end up returning an
> invalid pitch or creating a zero-sized dumb buffer.
>
> Fix this by using check_add_overflow() for the alignment round-up and
> check_mul_overflow() for the final pitch calculation, returning 0 on
> overflow. Also reject zero pitch and size in
> radeon_mode_dumb_create().
>
> Found via AST-based call-graph analysis using sqry.
>
> Fixes: ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")
> Cc: stable@vger.kernel.org
> Signed-off-by: Werner Kasselman <werner@verivus.com>
> ---
> v3:
> - Squash this fix with the earlier zero pitch/size validation change.
> - Use overflow helpers for both the alignment round-up and final
>   pitch calculation.
>
>  drivers/gpu/drm/radeon/radeon_gem.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
> index 20fc87409f2e..8ce180e22d1d 100644
> --- a/drivers/gpu/drm/radeon/radeon_gem.c
> +++ b/drivers/gpu/drm/radeon/radeon_gem.c
> @@ -28,6 +28,7 @@
>
>  #include <linux/debugfs.h>
>  #include <linux/iosys-map.h>
> +#include <linux/overflow.h>
>  #include <linux/pci.h>
>
>  #include <drm/drm_device.h>
> @@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
>         int aligned = width;
>         int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
>         int pitch_mask = 0;
> +       int pitch;
>
>         switch (cpp) {
>         case 1:
> @@ -826,9 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
>                 break;
>         }
>
> -       aligned += pitch_mask;
> +       if (check_add_overflow(aligned, pitch_mask, &aligned))
> +               return 0;
>         aligned &= ~pitch_mask;
> -       return aligned * cpp;
> +       if (check_mul_overflow(aligned, cpp, &pitch))
> +               return 0;
> +       return pitch;
>  }
>
>  int radeon_mode_dumb_create(struct drm_file *file_priv,
> @@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
>
>         args->pitch = radeon_align_pitch(rdev, args->width,
>                                          DIV_ROUND_UP(args->bpp, 8), 0);
> +       if (!args->pitch)
> +               return -EINVAL;
>         args->size = (u64)args->pitch * args->height;
>         args->size = ALIGN(args->size, PAGE_SIZE);
> +       if (!args->size)
> +               return -EINVAL;
>
>         r = radeon_gem_object_create(rdev, args->size, 0,
>                                      RADEON_GEM_DOMAIN_VRAM, 0,
> --
> 2.43.0