From: sashiko-bot@kernel.org
To: "Arvind Yadav" <arvind.yadav@intel.com>, intel-xe@lists.freedesktop.org
Subject: Re: [PATCH] CI_ONLY: Defer TDR execution while submission is suspended
Date: Wed, 18 Nov 2026 10:46:22 +0000 [thread overview]
Message-ID: <sashiko-outbox-220743@kernel.org> (raw)
In-Reply-To: <20261118101543.2210570-1-arvind.yadav@intel.com>
Thank you for your contribution! Sashiko AI review found 4 potential
issues to consider:
- [Medium] drm/xe: broken TDR and submission recovery on xe_uc_suspend()
failure
- [Medium] drm/xe: work_tdr left disabled on resume and wedge failure
paths
- [Medium] drm/xe: infinite busy loop in uc_reset_wait() when suspending
wedged device
- [Low] drm/xe: CI_ONLY commit markers and undocumented
xe_device_resume() change
It also noticed 1 related pre-existing issue this patch does not
introduce:
- [High] drm/xe: infinite loop in uc_reset_wait() [1]
--
commit dc721c0cb578b8e8efd82320e9f07c85073891f4
Author: Arvind Yadav <arvind.yadav@intel.com>
> CI_ONLY: Defer TDR execution while submission is suspended
>
> CI ONLY...
> guc_exec_queue_stop() can ban a queue with an incomplete job and request
> immediate TDR cleanup. During suspend this runs from the PM thread,
> while the timeout worker runs on gt->ordered_wq. The timeout worker can
> therefore capture engine registers and restart scheduler submission
> while GuC submission is being stopped and the device is powering down.
>
> Disable each queue's timeout work before stopping submission. Do this
> from gt->ordered_wq so any running drm_sched_job_timedout() has finished,
> including the pending-list updates outside the Xe timeout callback.
> Keep the GuC stop and suspend operations on the PM caller's thread.
>
> Keep timeout work disabled until all queues have been restarted. Also
> disable timeout work on queues created while submission is suspended.
> Re-enable it on suspend failure and teardown abort paths so the new
> disable count cannot prevent cleanup indefinitely.
>
> Leave the scheduler timeout value unchanged. An immediate cleanup
> request made while timeout work is disabled still sets it to zero,
> so restarting submission re-arms that cleanup without an extra delay.
>
> Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
> Link: https://sashiko.dev/#/bug/linux-449968a5-0b4a-4c5d-acd3-7ced29f0129b
> Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
[Severity: Low]
This isn't a bug, but the commit subject and body still contain temporary
"CI_ONLY" markers, and the changelog does not mention or explain the
xe_wedge_declare() change in xe_device_resume().
> diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> index 6911c2ac39f82..2de67e4119539 100644
> --- a/drivers/gpu/drm/xe/xe_device.c
> +++ b/drivers/gpu/drm/xe/xe_device.c
> @@ -1509,8 +1509,13 @@ int xe_device_resume(struct xe_device *xe, bool flr)
>
> for_each_gt(gt, xe, id) {
> err = xe_gt_resume(gt);
> - if (err)
> + if (err) {
> + /* GuC boot failed before submission could restart. */
> + if (!flr && err == -ENOEXEC)
> + xe_wedge_declare(xe, DRM_WEDGE_RECOVERY_REBIND |
> + DRM_WEDGE_RECOVERY_BUS_RESET);
[Severity: Medium]
Can declaring the device wedged here while guc->submission_state.stopped
remains 1 cause a later xe_uc_suspend() call to spin forever in
uc_reset_wait()?
When xe_device_resume() fails after xe_uc_suspend() has set
guc->submission_state.stopped to 1, xe_wedge_declare() marks the device
wedged without clearing stopped. On a subsequent system suspend,
xe_uc_suspend() calls uc_reset_wait(), where xe_guc_submit_reset_wait()
returns immediately because xe_wedged() is true while xe_uc_reset_prepare()
keeps returning 1 because stopped is still set, looping indefinitely
without sleeping.
> break;
> + }
> }
[Severity: Medium]
Does work_tdr remain permanently disabled if resume fails along any other
path before xe_guc_submit_start()?
Only !flr && err == -ENOEXEC in xe_device_resume() calls xe_wedge_declare().
If xe_pcode_ready() or xe_bo_restore_early() fails earlier, if
xe_gt_resume() fails with another error such as -ETIMEDOUT or -EIO (or skips
remaining GTs after break), if D3Cold runtime resume fails in
xe_pm_runtime_resume(), or if xe->wedged.mode is
XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET or XE_WEDGED_MODE_NEVER,
guc_submit_resume_tdr() is never reached. Pending jobs on banned or killed
queues then never time out or signal their fences, hanging xe_vm_close() and
show_run_ticks().
[ ... ]
> diff --git a/drivers/gpu/drm/xe/xe_uc.c b/drivers/gpu/drm/xe/xe_uc.c
> index b32ba504041c5..aa154800a8815 100644
> --- a/drivers/gpu/drm/xe/xe_uc.c
> +++ b/drivers/gpu/drm/xe/xe_uc.c
> @@ -362,17 +362,41 @@ void xe_uc_suspend_prepare(struct xe_uc *uc)
> xe_guc_stop_prepare(&uc->guc);
> }
>
> +struct xe_uc_suspend_work {
> + struct work_struct work;
> + struct xe_uc *uc;
> +};
> +
> +static void uc_suspend_work(struct work_struct *w)
> +{
> + struct xe_uc_suspend_work *suspend =
> + container_of(w, struct xe_uc_suspend_work, work);
> + xe_guc_submit_suspend_tdr(&suspend->uc->guc);
> +}
> +
> int xe_uc_suspend(struct xe_uc *uc)
> {
> + struct xe_uc_suspend_work suspend = { .uc = uc };
> + int err;
> +
> /* GuC submission not enabled, nothing to do */
> if (!xe_device_uc_enabled(uc_to_xe(uc)))
> return 0;
>
> uc_reset_wait(uc);
[Severity: High]
This problem wasn't introduced by this patch, but can uc_reset_wait() spin
in an infinite loop [1] when suspending a wedged device?
drivers/gpu/drm/xe/xe_uc.c:uc_reset_wait() {
int ret;
again:
xe_guc_reset_wait(&uc->guc);
ret = xe_uc_reset_prepare(uc);
if (ret)
goto again;
}
If the device is already wedged with guc->submission_state.stopped set to 1,
xe_guc_submit_reset_wait() returns immediately because xe_wedged() is true,
while xe_uc_reset_prepare() atomically sets stopped and returns its previous
value of 1, jumping back to again in a tight non-sleeping loop.
> +
> + /* Drain the complete timeout worker before examining queue state. */
> + INIT_WORK_ONSTACK(&suspend.work, uc_suspend_work);
> + queue_work(uc_to_gt(uc)->ordered_wq, &suspend.work);
> + flush_work(&suspend.work);
> + destroy_work_on_stack(&suspend.work);
>
> xe_uc_stop(uc);
> + err = xe_guc_suspend(&uc->guc);
> + if (err)
> + xe_guc_submit_resume_tdr(&uc->guc);
>
> - return xe_guc_suspend(&uc->guc);
> + return err;
> }
[Severity: Medium]
If xe_guc_suspend() fails here, is calling xe_guc_submit_resume_tdr() safe
after uc_reset_wait() and xe_uc_stop() have already stopped submission?
By the time xe_guc_suspend() runs, guc->submission_state.stopped is 1, CT is
stopped, and guc_exec_queue_stop() has stopped each queue's scheduler,
cleared EXEC_QUEUE_STATE_REGISTERED and EXEC_QUEUE_STATE_ENABLED, and banned
user queues with started jobs (setting base.timeout to 0). If
xe_guc_softreset() fails, GuC firmware is still running, so resuming TDR
immediately runs guc_exec_queue_timedout_job() on banned queues: because
REGISTERED and ENABLED were cleared, it skips disabling scheduling in GuC,
signals fences early, restarts the scheduler against a stopped CT, and
frees the queue's guc_id and LRC locally while GuC is still active.
In addition, idle queues remain stopped so later guc_exec_queue_destroy()
calls never process STATIC_MSG_CLEANUP, guc->submission_state.stopped stays
1 so subsequent suspends or GT resets stall, and on multi-GT systems a
suspend failure on a later GT in xe_device_suspend() or
xe_pm_runtime_suspend() leaves work_tdr disabled on earlier GTs that already
completed xe_gt_suspend().
[1] https://sashiko.dev/#/bug/linux-62eea921-894e-4f73-bbf0-ac02cad95b25
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118101543.2210570-1-arvind.yadav@intel.com?part=1
prev parent reply other threads:[~2026-11-18 10:46 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 10:15 [PATCH] CI_ONLY: Defer TDR execution while submission is suspended Arvind Yadav
2026-11-18 10:46 ` sashiko-bot [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-220743@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=arvind.yadav@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=sashiko-reviews@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox