[PATCH] drm/modes: reject out-of-range GTF blanking duty cycle

[PATCH] drm/modes: reject out-of-range GTF blanking duty cycle

[Xiang Mei]

drm_gtf_mode_complex() derives the blanking duty cycle from the GTF
C/M/K/J coefficients and divides by its complement:

	ideal_duty_cycle = GTF_C_PRIME * 1000 -
				(GTF_M_PRIME * 1000000 / hfreq_est);
	hblank = total_active_pixels * ideal_duty_cycle /
			(100000 - ideal_duty_cycle);

ideal_duty_cycle is an unsigned per-mille percentage that must stay below
100000, but nothing enforces it. Via drm_gtf2_mode() the coefficients come
straight from an EDID secondary-GTF range descriptor, so a crafted monitor
(or an edid_override re-probed with DRM_IOCTL_MODE_GETCONNECTOR) can set
ideal_duty_cycle to exactly 100000, making the divisor 0 and trapping with
a divide error; a larger value wraps the unsigned subtraction and yields a
garbage modeline.

Reject a duty cycle of 100% or more, which is never a valid GTF result.
drm_gtf2_mode() and the inferred-mode helpers already handle a NULL return.

Crash:

  Oops: divide error: 0000 [#1] SMP KASAN NOPTI
  RIP: 0010:drm_gtf_mode_complex (drivers/gpu/drm/drm_modes.c:972)
  Call Trace:
   drm_gtf2_mode (drivers/gpu/drm/drm_edid.c:3365)
   drm_mode_std (drivers/gpu/drm/drm_edid.c:3464)
   _drm_edid_connector_add_modes.part.0 (drivers/gpu/drm/drm_edid.c:4024)
   drm_edid_connector_add_modes (drivers/gpu/drm/drm_edid.c:7020)
   bochs_connector_helper_get_modes (drivers/gpu/drm/tiny/bochs.c:575)
   drm_helper_probe_single_connector_modes (drivers/gpu/drm/drm_probe_helper.c:426)
   drm_mode_getconnector (drivers/gpu/drm/drm_connector.c:3376)
   ...
   __x64_sys_ioctl (fs/ioctl.c:597)
   do_syscall_64 (arch/x86/entry/syscall_64.c:94)
   entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
  Kernel panic - not syncing: Fatal exception

Fixes: 26bbdadad356 ("drm/mode: add the GTF algorithm in kernel space")
Reported-by: Weiming Shi <bestswngs@gmail.com>
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Xiang Mei <xmei5@asu.edu>
---
 drivers/gpu/drm/drm_modes.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
index 3f8e025fd6d9..b6f263e9ba86 100644
--- a/drivers/gpu/drm/drm_modes.c
+++ b/drivers/gpu/drm/drm_modes.c
@@ -966,6 +966,10 @@ drm_gtf_mode_complex(struct drm_device *dev, int hdisplay, int vdisplay,
 	/* 18.Find the ideal blanking duty cycle from blanking duty cycle */
 	ideal_duty_cycle = GTF_C_PRIME * 1000 -
 				(GTF_M_PRIME * 1000000 / hfreq_est);
+	if (ideal_duty_cycle >= 100000) {
+		drm_mode_destroy(dev, drm_mode);
+		return NULL;
+	}
 	/* 19.Find the number of pixels in the blanking time to the nearest
 	 * double character cell: */
 	hblank = total_active_pixels * ideal_duty_cycle /
-- 
2.43.0

Re: [PATCH] drm/modes: reject out-of-range GTF blanking duty cycle

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Medium] Unchecked `vrefresh` parameter from kernel command line can cause multiple divide-by-zero panics.
--

--- Patch [1]: [PATCH] drm/modes: reject out-of-range GTF blanking duty cycle ---
commit c78fc29b3d453bf33d5cd88746572b5f1e9a5c60
Author: Xiang Mei <xmei5@asu.edu>

drm/modes: reject out-of-range GTF blanking duty cycle

This commit rejects GTF blanking duty cycles of 100% or more to prevent a
divide-by-zero error that occurs when the divisor (100000 - ideal_duty_cycle)
evaluates to 0.

> diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
> index e1eed13a8e94f..291037706b587 100644
> --- a/drivers/gpu/drm/drm_modes.c
> +++ b/drivers/gpu/drm/drm_modes.c
> @@ -966,6 +966,10 @@ drm_gtf_mode_complex(struct drm_device *dev, int hdisplay, int vdisplay,
>  	/* 18.Find the ideal blanking duty cycle from blanking duty cycle */
>  	ideal_duty_cycle = GTF_C_PRIME * 1000 -
>  				(GTF_M_PRIME * 1000000 / hfreq_est);

[Severity: Medium]
This is a pre-existing issue, but while fixing divide-by-zero crashes here,
could an unvalidated vrefresh parameter cause other divide-by-zero panics
in this function?

For example, if an administrator provides a kernel command line parameter
with a refresh rate of @0 (e.g., video=1024x768@0), vfieldrate_rqd
evaluates to 0. This would cause a panic earlier in drm_gtf_mode_complex():

    hperiod = 1000000 / (vfieldrate_rqd * vtotal_lines);

Also, if the refresh rate is @1818, tmp1 evaluates to 0:

    tmp1 = (1000000  - MIN_VSYNC_PLUS_BP * vfieldrate_rqd) / 500;
    // (1000000 - 550 * 1818) / 500 = 0

This leads to another direct division by zero:

    hfreq_est = (tmp2 * 1000 * vfieldrate_rqd) / tmp1;

And if hfreq_est evaluates to 0, it causes a third divide-by-zero on the
ideal_duty_cycle assignment above.

Should drm_gtf_mode_complex() validate the vrefresh parameter before
performing these calculations?

> +	if (ideal_duty_cycle >= 100000) {
> +		drm_mode_destroy(dev, drm_mode);
> +		return NULL;
> +	}
>  	/* 19.Find the number of pixels in the blanking time to the nearest
>  	 * double character cell: */
>  	hblank = total_active_pixels * ideal_duty_cycle /

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260711230728.2250485-1-xmei5@asu.edu?part=1