From: sashiko-bot@kernel.org
To: "Arvind Yadav" <arvind.yadav@intel.com>, intel-xe@lists.freedesktop.org
Subject: Re: [PATCH] CI_ONLY: Defer TDR execution while submission is suspended
Date: Wed, 18 Nov 2026 10:46:22 +0000 [thread overview]
Message-ID: <sashiko-outbox-220743@kernel.org> (raw)
In-Reply-To: <20261118101543.2210570-1-arvind.yadav@intel.com>
Thank you for your contribution! Sashiko AI review found 4 potential
issues to consider:
- [Medium] drm/xe: broken TDR and submission recovery on xe_uc_suspend()
failure
- [Medium] drm/xe: work_tdr left disabled on resume and wedge failure
paths
- [Medium] drm/xe: infinite busy loop in uc_reset_wait() when suspending
wedged device
- [Low] drm/xe: CI_ONLY commit markers and undocumented
xe_device_resume() change
It also noticed 1 related pre-existing issue this patch does not
introduce:
- [High] drm/xe: infinite loop in uc_reset_wait() [1]
--
commit dc721c0cb578b8e8efd82320e9f07c85073891f4
Author: Arvind Yadav <arvind.yadav@intel.com>
> CI_ONLY: Defer TDR execution while submission is suspended
>
> CI ONLY...
> guc_exec_queue_stop() can ban a queue with an incomplete job and request
> immediate TDR cleanup. During suspend this runs from the PM thread,
> while the timeout worker runs on gt->ordered_wq. The timeout worker can
> therefore capture engine registers and restart scheduler submission
> while GuC submission is being stopped and the device is powering down.
>
> Disable each queue's timeout work before stopping submission. Do this
> from gt->ordered_wq so any running drm_sched_job_timedout() has finished,
> including the pending-list updates outside the Xe timeout callback.
> Keep the GuC stop and suspend operations on the PM caller's thread.
>
> Keep timeout work disabled until all queues have been restarted. Also
> disable timeout work on queues created while submission is suspended.
> Re-enable it on suspend failure and teardown abort paths so the new
> disable count cannot prevent cleanup indefinitely.
>
> Leave the scheduler timeout value unchanged. An immediate cleanup
> request made while timeout work is disabled still sets it to zero,
> so restarting submission re-arms that cleanup without an extra delay.
>
> Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
> Link: https://sashiko.dev/#/bug/linux-449968a5-0b4a-4c5d-acd3-7ced29f0129b
> Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
[Severity: Low]
This isn't a bug, but the commit subject and body still contain temporary
"CI_ONLY" markers, and the changelog does not mention or explain the
xe_wedge_declare() change in xe_device_resume().
> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> index 6911c2ac39f82..2de67e4119539 100644
> --- a/drivers/gpu/drm/xe/xe_device.c
> +++ b/drivers/gpu/drm/xe/xe_device.c
> @@ -1509,8 +1509,13 @@ int xe_device_resume(struct xe_device *xe, bool flr)
>
> for_each_gt(gt, xe, id) {
> err = xe_gt_resume(gt);
> - if (err)
> + if (err) {
> + /* GuC boot failed before submission could restart. */
> + if (!flr && err == -ENOEXEC)
> + xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_REBIND |
> + DRM_WEDGE_RECOVERY_BUS_RESET);
[Severity: Medium]
Can declaring the device wedged here while guc->submission_state.stopped
remains 1 cause a later xe_uc_suspend() call to spin forever in
uc_reset_wait()?
When xe_device_resume() fails after xe_uc_suspend() has set
guc->submission_state.stopped to 1, xe_wedge_declare() marks the device
wedged without clearing stopped. On a subsequent system suspend,
xe_uc_suspend() calls uc_reset_wait(), where xe_guc_submit_reset_wait()
returns immediately because xe_wedged() is true while xe_uc_reset_prepare()
keeps returning 1 because stopped is still set, looping indefinitely
without sleeping.
> break;
> + }
> }
[Severity: Medium]
Does work_tdr remain permanently disabled if resume fails along any other
path before xe_guc_submit_start()?
Only !flr && err == -ENOEXEC in xe_device_resume() calls xe_wedge_declare().
If xe_pcode_ready() or xe_bo_restore_early() fails earlier, if
xe_gt_resume() fails with another error such as -ETIMEDOUT or -EIO (or skips
remaining GTs after break), if D3Cold runtime resume fails in
xe_pm_runtime_resume(), or if xe->wedged.mode is
XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET or XE_WEDGED_MODE_NEVER,
guc_submit_resume_tdr() is never reached. Pending jobs on banned or killed
queues then never time out or signal their fences, hanging xe_vm_close() and
show_run_ticks().
[ ... ]
> diff --git a/drivers/gpu/drm/xe/xe_uc.c b/drivers/gpu/drm/xe/xe_uc.c
> index b32ba504041c5..aa154800a8815 100644
> --- a/drivers/gpu/drm/xe/xe_uc.c
> +++ b/drivers/gpu/drm/xe/xe_uc.c
> @@ -362,17 +362,41 @@ void xe_uc_suspend_prepare(struct xe_uc *uc)
> xe_guc_stop_prepare(&uc->guc);
> }
>
> +struct xe_uc_suspend_work {
> + struct work_struct work;
> + struct xe_uc *uc;
> +};
> +
> +static void uc_suspend_work(struct work_struct *w)
> +{
> + struct xe_uc_suspend_work *suspend =
> + container_of(w, struct xe_uc_suspend_work, work);
> + xe_guc_submit_suspend_tdr(&suspend->uc->guc);
> +}
> +
> int xe_uc_suspend(struct xe_uc *uc)
> {
> + struct xe_uc_suspend_work suspend = { .uc = uc };
> + int err;
> +
> /* GuC submission not enabled, nothing to do */
> if (!xe_device_uc_enabled(uc_to_xe(uc)))
> return 0;
>
> uc_reset_wait(uc);
[Severity: High]
This problem wasn't introduced by this patch, but can uc_reset_wait() spin
in an infinite loop [1] when suspending a wedged device?
drivers/gpu/drm/xe/xe_uc.c:uc_reset_wait() {
int ret;
again:
xe_guc_reset_wait(&uc->guc);
ret = xe_uc_reset_prepare(uc);
if (ret)
goto again;
}
If the device is already wedged with guc->submission_state.stopped set to 1,
xe_guc_submit_reset_wait() returns immediately because xe_wedged() is true,
while xe_uc_reset_prepare() atomically sets stopped and returns its previous
value of 1, jumping back to again in a tight non-sleeping loop.
> +
> + /* Drain the complete timeout worker before examining queue state. */
> + INIT_WORK_ONSTACK(&suspend.work, uc_suspend_work);
> + queue_work(uc_to_gt(uc)->ordered_wq, &suspend.work);
> + flush_work(&suspend.work);
> + destroy_work_on_stack(&suspend.work);
>
> xe_uc_stop(uc);
> + err = xe_guc_suspend(&uc->guc);
> + if (err)
> + xe_guc_submit_resume_tdr(&uc->guc);
>
> - return xe_guc_suspend(&uc->guc);
> + return err;
> }
[Severity: Medium]
If xe_guc_suspend() fails here, is calling xe_guc_submit_resume_tdr() safe
after uc_reset_wait() and xe_uc_stop() have already stopped submission?
By the time xe_guc_suspend() runs, guc->submission_state.stopped is 1, CT is
stopped, and guc_exec_queue_stop() has stopped each queue's scheduler,
cleared EXEC_QUEUE_STATE_REGISTERED and EXEC_QUEUE_STATE_ENABLED, and banned
user queues with started jobs (setting base.timeout to 0). If
xe_guc_softreset() fails, GuC firmware is still running, so resuming TDR
immediately runs guc_exec_queue_timedout_job() on banned queues: because
REGISTERED and ENABLED were cleared, it skips disabling scheduling in GuC,
signals fences early, restarts the scheduler against a stopped CT, and
frees the queue's guc_id and LRC locally while GuC is still active.
In addition, idle queues remain stopped so later guc_exec_queue_destroy()
calls never process STATIC_MSG_CLEANUP, guc->submission_state.stopped stays
1 so subsequent suspends or GT resets stall, and on multi-GT systems a
suspend failure on a later GT in xe_device_suspend() or
xe_pm_runtime_suspend() leaves work_tdr disabled on earlier GTs that already
completed xe_gt_suspend().
[1] https://sashiko.dev/#/bug/linux-62eea921-894e-4f73-bbf0-ac02cad95b25
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118101543.2210570-1-arvind.yadav@intel.com?part=1
next prev parent reply other threads:[~2026-11-18 10:46 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 10:15 [PATCH] CI_ONLY: Defer TDR execution while submission is suspended Arvind Yadav
2026-11-18 10:46 ` sashiko-bot [this message]
2026-11-18 11:13 ` ✓ CI.KUnit: success for " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-220743@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=arvind.yadav@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=sashiko-reviews@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.