Intel-GFX Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/2] drm/i915/guc: Fix requests of a banned context never completing
@ 2026-11-10 12:24 Javier MV
  2026-11-10 12:24 ` [PATCH 1/2] drm/i915/guc: Complete all requests of a non-schedulable context Javier MV
                   ` (3 more replies)
  0 siblings, 4 replies; 7+ messages in thread
From: Javier MV @ 2026-11-10 12:24 UTC (permalink / raw)
  To: intel-gfx
  Cc: Javier MV, jani.nikula, joonas.lahtinen, rodrigo.vivi, tursulin,
	matthew.brost, daniele.ceraolospurio, umesh.nerlige.ramappa

From: Javier MV <javiermv@duck.com>

On ADL-P with GuC submission, a GuC context reset of a context whose requests
are being cancelled by the request watchdog can leave that context's requests
incomplete forever. The client waits on their fences forever, and if it is
killed its buffers stay pinned until reboot (here 7.5 GB of Shmem/Unevictable,
with the process gone from DRM clients).
Reported in https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/15301.

The sequence, from instrumented runs (a llama.cpp Vulkan compute job,
request_timeout_ms=2000 via debugfs, preempt_timeout_ms=7500):

 1. The watchdog cancels a request: guc_context_cancel_request() blocks the
    context (schedule disable) to skip it.
 2. GuC can't preempt the compute work (no mid-thread preemption on Gen12) and
    resets the context instead. The reset handler captures, bans (revoke finds
    the disable pending, so it only sets a 1 ms preemption timeout) and replays,
    which unwinds the incomplete requests back to the scheduler queue.
 3. The late schedule disable done G2H clears SCHED_STATE_BANNED and cancels
    nothing (the list is already empty), then guc_context_unblock() enables
    scheduling of the banned context.
 4. Either the resubmitted requests run as NOPs and everything retires
    (harmless), or GuC resets the context again at the first preemption (1 ms
    timeout). That second notification is ignored for a banned context, so its
    submitted requests never complete. In one run this was followed by G2H
    action 0x6000 rejected with -EOPNOTSUPP, a stopped rcs0 heartbeat and a
    full GT reset; after it the context was no longer registered, and the
    dequeue only marks the last of a banned context's merged requests
    complete, so the other 9 never completed.

Patch 1 completes all requests of a non-schedulable context on resubmission
(once it is off the HW), and patch 2 stops unblock from re-enabling a banned
context.

A reset notification for an already banned context is still ignored. I didn't
see one with this series, so it doesn't change that, but if one can still
happen some other way, that context's requests could stay incomplete. I have
an additional patch for that case, which cancels the requests on such a
notification. It has never run in my tests because I couldn't reach that path
with these patches applied, so I left it out, but I can send it if you want it.

Tested on i5-1235U (ADL-P GT2, 8086:46a8), GuC 70.36.0, on top of Ubuntu's
7.0.0-42 (mainline 7.0.14; the touched code is identical in mainline as of
2026-10-29). Without the series: 5 hangs in 17 runs with the setup above
(some with other GPU or CPU load), one of them leaked as described (as did
an earlier run on the stock module). With this series: 30 runs, 1 hang (the
ecode of the run that leaked): no full GT reset, the client got
VK_ERROR_DEVICE_LOST instead of waiting forever, nothing stayed pinned.
Instrumented builds that also carried the additional patch above, which never
fired: 20 runs with both patches, 2 hangs of the leaking kind (same ecodes as
before), handled the same way, and the logs show no second reset; 20 runs with
patch 1 alone, 1 hang, also contained.

I can reproduce this here, though a hang takes 10 to 30 runs. If it helps, I
can send the full log of a hang from the instrumented build, attach error
states, or test a different approach if you'd rather fix it another way.

Tool disclosure: I did this work with Claude Code (Opus 5.5), over several
sessions. I reported the symptoms, ran it on my machine and decided what to
pursue. The assistant read the i915 sources and the logs, wrote the debug
instrumentation and test scripts, worked out the sequence above, and wrote both
patches, their changelogs and most of this letter. The logs and results quoted
here come from runs on my machine. I have checked the sequence and the patches
against the code and can answer for them.

Javier MV (2):
  drm/i915/guc: Complete all requests of a non-schedulable context
  drm/i915/guc: Don't unblock a banned context

 .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)


base-commit: d89e1d186ec9271bf78e62771779e9fc03bf0fc3
-- 
2.51.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 1/2] drm/i915/guc: Complete all requests of a non-schedulable context
  2026-11-10 12:24 [PATCH 0/2] drm/i915/guc: Fix requests of a banned context never completing Javier MV
@ 2026-11-10 12:24 ` Javier MV
  2026-11-10 12:56   ` sashiko-bot
  2026-11-10 12:24 ` [PATCH 2/2] drm/i915/guc: Don't unblock a banned context Javier MV
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 7+ messages in thread
From: Javier MV @ 2026-11-10 12:24 UTC (permalink / raw)
  To: intel-gfx
  Cc: Javier MV, jani.nikula, joonas.lahtinen, rodrigo.vivi, tursulin,
	matthew.brost, daniele.ceraolospurio, umesh.nerlige.ramappa

From: Javier MV <javiermv@duck.com>

When the context of a request being submitted is banned or exiting,
__guc_add_request() marks the request as complete with -EIO instead of
submitting it. But guc_dequeue_one_context() merges all queued requests
of a context and only passes the last one to __guc_add_request(). The
earlier ones were moved to ce->guc_state.requests by
__i915_request_submit(), which only skips their payload: they still
need the GPU to run the context and write their breadcrumbs.

After a full GT reset the banned context is no longer registered with
GuC and is never registered again, so those requests never complete.
Nothing retires them, the buffers they reference stay pinned, and the
client waits on their fences forever.

Seen on ADL-P with a Vulkan compute job (llama.cpp) and a short
request_timeout_ms: a GuC context reset, the ban, the replay moving
10 requests back to the queue, then a full GT reset after a stopped
heartbeat. Afterwards the context was unregistered and the GuC submit
queue was empty, but 9 of its requests never completed: 7.5 GB of
objects stayed pinned after the process was killed.

Complete every request on the context's list, as
guc_cancel_context_requests() does, but only when the context is off
the HW (not enabled, no enable or disable in flight), so a request that
is still executing is never marked complete.

Found and fixed with Claude Code (AI coding assistant); details in the
cover letter.

Fixes: ae8ac10dfd2a ("drm/i915/guc: Implement banned contexts for GuC submission")
Link: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/15301
Signed-off-by: Javier MV <javiermv@duck.com>
---
 drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 788e59c..555ca8f 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -709,6 +709,20 @@ static int __guc_add_request(struct intel_guc *guc, struct i915_request *rq)
 	 * request resubmitted after the context was banned.
 	 */
 	if (unlikely(!intel_context_is_schedulable(ce))) {
+		struct i915_request *pos;
+
+		/*
+		 * The dequeue only passes the last of the merged requests, and
+		 * the context won't run again to write the others' breadcrumbs:
+		 * complete them all, once the context is off the HW.
+		 */
+		spin_lock(&ce->guc_state.lock);
+		if (!context_enabled(ce) && !context_pending_enable(ce) &&
+		    !context_pending_disable(ce))
+			list_for_each_entry(pos, &ce->guc_state.requests, sched.link)
+				i915_request_put(i915_request_mark_eio(pos));
+		spin_unlock(&ce->guc_state.lock);
+
 		i915_request_put(i915_request_mark_eio(rq));
 		intel_engine_signal_breadcrumbs(ce->engine);
 		return 0;
-- 
2.51.1


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 2/2] drm/i915/guc: Don't unblock a banned context
  2026-11-10 12:24 [PATCH 0/2] drm/i915/guc: Fix requests of a banned context never completing Javier MV
  2026-11-10 12:24 ` [PATCH 1/2] drm/i915/guc: Complete all requests of a non-schedulable context Javier MV
@ 2026-11-10 12:24 ` Javier MV
  2026-11-10 13:07   ` sashiko-bot
  2026-11-10 15:11 ` ✗ LGCI.VerificationFailed: failure for drm/i915/guc: Fix requests of a banned context never completing Patchwork
  2026-11-11 15:29 ` [PATCH 0/2] " Javier MV
  3 siblings, 1 reply; 7+ messages in thread
From: Javier MV @ 2026-11-10 12:24 UTC (permalink / raw)
  To: intel-gfx
  Cc: Javier MV, jani.nikula, joonas.lahtinen, rodrigo.vivi, tursulin,
	matthew.brost, daniele.ceraolospurio, umesh.nerlige.ramappa

From: Javier MV <javiermv@duck.com>

guc_context_cancel_request() blocks the context, skips the request and
unblocks the context again. If the context gets banned meanwhile (the
blocked request hits a GuC context reset), guc_context_revoke() finds
the schedule disable of the block still pending, so it only sets the
preemption timeout to 1 ms and leaves the cancellation to the schedule
disable done handler. That handler clears SCHED_STATE_BANNED, and then
guc_context_unblock() finds nothing in context_cant_unblock() that
stops it from enabling scheduling again.

GuC then runs a banned context with a 1 ms preemption timeout and
resets it again at the first preemption. Seen on ADL-P: the second
reset notification was ignored, GuC sent G2H messages the driver
rejected (action 0x6000, -EOPNOTSUPP), the rcs0 heartbeat stopped and
the whole GT was reset, which also kills every other client's work.

Also check intel_context_is_banned(), which stays set. The requests the
replay put back in the queue are then completed on resubmission, as for
any banned context.

Found and fixed with Claude Code (AI coding assistant); details in the
cover letter.

Fixes: 9888beaaf118 ("drm/i915/guc: Don't enable scheduling on a banned context, guc_id invalid, not registered")
Link: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/15301
Signed-off-by: Javier MV <javiermv@duck.com>
---
 drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 555ca8f..eb1d010 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -3182,7 +3182,12 @@ static bool context_cant_unblock(struct intel_context *ce)
 {
 	lockdep_assert_held(&ce->guc_state.lock);
 
+	/*
+	 * SCHED_STATE_BANNED alone isn't enough: the sched disable done
+	 * handler clears it.
+	 */
 	return (ce->guc_state.sched_state & SCHED_STATE_NO_UNBLOCK) ||
+		intel_context_is_banned(ce) ||
 		context_guc_id_invalid(ce) ||
 		!ctx_id_mapped(ce_to_guc(ce), ce->guc_id.id) ||
 		!intel_context_is_pinned(ce);
-- 
2.51.1


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 1/2] drm/i915/guc: Complete all requests of a non-schedulable context
  2026-11-10 12:24 ` [PATCH 1/2] drm/i915/guc: Complete all requests of a non-schedulable context Javier MV
@ 2026-11-10 12:56   ` sashiko-bot
  0 siblings, 0 replies; 7+ messages in thread
From: sashiko-bot @ 2026-11-10 12:56 UTC (permalink / raw)
  To: Javier MV, intel-gfx

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] drm/i915/guc: use-after-free in __guc_add_request() on
  completed rq
- [High] drm/i915/guc: out-of-order completion of rq in
  __guc_add_request()
- [Medium] drm/i915/guc: uncompleted requests in __guc_add_request() on
  pending enable
--

commit 75c751fdc5e9421329da29b1d39d5306a72785b3
Author: Javier MV <javiermv@duck.com>

drm/i915/guc: Complete all requests of a non-schedulable context

This commit completes all requests on ce->guc_state.requests with -EIO in
__guc_add_request() when a context is not schedulable and is off the
hardware.

Link: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/15301

> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 788e59cdfac9b..555ca8fdf64af 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -709,6 +709,20 @@ static int __guc_add_request(struct intel_guc *guc, struct i915_request *rq)
>  	 * request resubmitted after the context was banned.
>  	 */
>  	if (unlikely(!intel_context_is_schedulable(ce))) {
> +		struct i915_request *pos;
> +
> +		/*
> +		 * The dequeue only passes the last of the merged requests, and
> +		 * the context won't run again to write the others' breadcrumbs:
> +		 * complete them all, once the context is off the HW.
> +		 */
> +		spin_lock(&ce->guc_state.lock);
> +		if (!context_enabled(ce) && !context_pending_enable(ce) &&
> +		    !context_pending_disable(ce))
> +			list_for_each_entry(pos, &ce->guc_state.requests, sched.link)
> +				i915_request_put(i915_request_mark_eio(pos));

[Severity: Medium]
Can checking !context_pending_enable(ce) leave requests on
ce->guc_state.requests incomplete forever?

When GuC resets a context before an in-flight schedule-enable G2H is
processed, __guc_reset_context() calls clr_context_enabled(ce) while
SCHED_STATE_PENDING_ENABLE remains set and SCHED_STATE_PENDING_DISABLE is
clear.  In that state guc_context_revoke() treats the context as already off
the hardware and does not send a disable H2G:

drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c:guc_context_revoke() {
	...
	if (submission_disabled(guc) ||
	    (!context_enabled(ce) && !context_pending_disable(ce))) {
		spin_unlock_irqrestore(&ce->guc_state.lock, flags);

		guc_cancel_context_requests(ce);
		intel_engine_signal_breadcrumbs(ce->engine);
	...
}

When the pending enable G2H later arrives in
intel_guc_sched_done_process_msg(), it only calls clr_context_pending_enable()
and never calls guc_cancel_context_requests(ce), so any requests skipped here
because of context_pending_enable(ce) will never be completed.

> +		spin_unlock(&ce->guc_state.lock);
> +
> +		i915_request_put(i915_request_mark_eio(rq));
> +		intel_engine_signal_breadcrumbs(ce->engine);

[Severity: High]
Can this trigger a use-after-free on rq (and ce->engine) after dropping
ce->guc_state.lock?

Both callers of guc_add_request() first run __i915_request_submit(rq) ->
add_to_context(rq), which moves rq onto ce->guc_state.requests without taking
an extra reference beyond the timeline reference on tl->requests.  When the
loop above runs, it visits pos == rq and marks rq complete under
ce->guc_state.lock.  In GuC mode, request retirement does not take
sched_engine->lock; it only serializes on tl->mutex and ce->guc_state.lock in
remove_from_context():

CPU0                                    CPU1
__guc_add_request(guc, rq)
  spin_lock(&ce->guc_state.lock)
  list_for_each_entry(pos, ...)
    i915_request_mark_eio(rq)
    i915_request_put(rq)
  spin_unlock(&ce->guc_state.lock)
                                        i915_request_retire(rq)
                                          remove_from_context(rq)
                                          intel_context_unpin(ce)
                                          i915_request_put(rq)  <-- frees rq
  i915_request_mark_eio(rq)  <-- use-after-free
  intel_engine_signal_breadcrumbs(ce->engine)

If rq is recycled from the SLAB_TYPESAFE_BY_RCU pool by another context on
CPU1 before CPU0 reaches the second i915_request_mark_eio(rq), CPU0 will mark
that unrelated request complete with -EIO, or access freed memory if the slab
or virtual engine context was freed.

[Severity: High]
When the context is still enabled or has a schedule transition in flight,
does marking rq complete outside the if condition complete rq out of order
ahead of earlier requests on the same timeline?

guc_dequeue_one_context() merges all queued requests of a context onto
ce->guc_state.requests and passes only the last request rq to
__guc_add_request().  If context_enabled(ce), context_pending_enable(ce), or
context_pending_disable(ce) is true, the loop is skipped and earlier requests
on the timeline remain incomplete, but rq at the tail of the timeline is
still marked complete here.

Because __i915_active_fence_set() and dma_resv replace earlier timeline
fences with the newest request's fence, completing rq while earlier requests
are still running on the GPU allows VMAs to be unbound and backing pages
freed during active GPU DMA, and triggers
GEM_BUG_ON(!i915_request_completed(tmp)) in i915_request_retire_upto().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261110122534.12989-1-javiermv@gmail.com?part=1

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] drm/i915/guc: Don't unblock a banned context
  2026-11-10 12:24 ` [PATCH 2/2] drm/i915/guc: Don't unblock a banned context Javier MV
@ 2026-11-10 13:07   ` sashiko-bot
  0 siblings, 0 replies; 7+ messages in thread
From: sashiko-bot @ 2026-11-10 13:07 UTC (permalink / raw)
  To: Javier MV, intel-gfx

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] drm/i915/guc: context_cant_unblock() misses exiting contexts
  on unblock
--

commit f3c1b0cc59baf35aaa07862b720ef5529fd5baed
Author: Javier MV <javiermv@duck.com>

drm/i915/guc: Don't unblock a banned context

This commit adds an intel_context_is_banned() check to
context_cant_unblock() so that guc_context_unblock() does not re-enable
scheduling on a banned context after the schedule-disable done handler
clears SCHED_STATE_BANNED.

Link: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/15301

> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 555ca8fdf64af..eb1d010ed3dcc 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -3182,7 +3182,12 @@ static bool context_cant_unblock(struct intel_context *ce)
>  {
>  	lockdep_assert_held(&ce->guc_state.lock);
>  
> +	/*
> +	 * SCHED_STATE_BANNED alone isn't enough: the sched disable done
> +	 * handler clears it.
> +	 */
>  	return (ce->guc_state.sched_state & SCHED_STATE_NO_UNBLOCK) ||
> +		intel_context_is_banned(ce) ||
>  		context_guc_id_invalid(ce) ||
>  		!ctx_id_mapped(ce_to_guc(ce), ce->guc_id.id) ||
>  		!intel_context_is_pinned(ce);

[Severity: Medium]
Should this check !intel_context_is_schedulable(ce) instead of
intel_context_is_banned(ce)?

intel_context_revoke() sets CONTEXT_EXITING (not CONTEXT_BANNED) via
intel_context_set_exiting() before calling guc_context_revoke(), which
then takes the same SCHED_STATE_BANNED path as a banned context.  If
intel_context_revoke() (for example on context close or process exit for
a non-persistent context) races with guc_context_cancel_request() while
the block schedule-disable is pending, intel_guc_sched_done_process_msg()
clears SCHED_STATE_BANNED and cancels the inflight requests with -EIO.

When guc_context_cancel_request() resumes, guc_reset_state() sees
!intel_context_is_schedulable(ce) and returns early without scrubbing the
LRC or advancing the ring head, and then guc_context_unblock() re-enables
scheduling in GuC for the revoked context with an un-scrubbed ring and
already-completed requests.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261110122534.12989-1-javiermv@gmail.com?part=2

^ permalink raw reply	[flat|nested] 7+ messages in thread

* ✗ LGCI.VerificationFailed: failure for drm/i915/guc: Fix requests of a banned context never completing
  2026-11-10 12:24 [PATCH 0/2] drm/i915/guc: Fix requests of a banned context never completing Javier MV
  2026-11-10 12:24 ` [PATCH 1/2] drm/i915/guc: Complete all requests of a non-schedulable context Javier MV
  2026-11-10 12:24 ` [PATCH 2/2] drm/i915/guc: Don't unblock a banned context Javier MV
@ 2026-11-10 15:11 ` Patchwork
  2026-11-11 15:29 ` [PATCH 0/2] " Javier MV
  3 siblings, 0 replies; 7+ messages in thread
From: Patchwork @ 2026-11-10 15:11 UTC (permalink / raw)
  To: Javier MV; +Cc: intel-gfx

== Series Details ==

Series: drm/i915/guc: Fix requests of a banned context never completing
URL   : https://patchwork.freedesktop.org/series/178201/
State : failure

== Summary ==

Series author address 'javiermv@gmail.com' is not on the allowlist, which prevents CI from being automatically triggered.
If you want CI to run for this series, ask Patchwork project owners to click 'retest' on the series in Patchwork.
Exception occurred during validation, bailing out!
Build URL: http://gfx-ci.igk.intel.com:8080/job/CI_PW_kernel/192948/ (on built-in)



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/2] drm/i915/guc: Fix requests of a banned context never completing
  2026-11-10 12:24 [PATCH 0/2] drm/i915/guc: Fix requests of a banned context never completing Javier MV
                   ` (2 preceding siblings ...)
  2026-11-10 15:11 ` ✗ LGCI.VerificationFailed: failure for drm/i915/guc: Fix requests of a banned context never completing Patchwork
@ 2026-11-11 15:29 ` Javier MV
  3 siblings, 0 replies; 7+ messages in thread
From: Javier MV @ 2026-11-11 15:29 UTC (permalink / raw)
  To: intel-gfx
  Cc: javiermv, jani.nikula, joonas.lahtinen, rodrigo.vivi, tursulin,
	matthew.brost, daniele.ceraolospurio, umesh.nerlige.ramappa

This series is superseded by v2, which fixes the use-after-free in patch 1
and covers exiting contexts in patch 2, both found by the Sashiko review:

https://lore.kernel.org/intel-gfx/20261111135753.11071-1-javiermv@gmail.com/

Please review v2 instead.

Javier MV

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-11-11 15:30 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-10 12:24 [PATCH 0/2] drm/i915/guc: Fix requests of a banned context never completing Javier MV
2026-11-10 12:24 ` [PATCH 1/2] drm/i915/guc: Complete all requests of a non-schedulable context Javier MV
2026-11-10 12:56   ` sashiko-bot
2026-11-10 12:24 ` [PATCH 2/2] drm/i915/guc: Don't unblock a banned context Javier MV
2026-11-10 13:07   ` sashiko-bot
2026-11-10 15:11 ` ✗ LGCI.VerificationFailed: failure for drm/i915/guc: Fix requests of a banned context never completing Patchwork
2026-11-11 15:29 ` [PATCH 0/2] " Javier MV

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox