* [PATCH] drm/xe/pf: Restore PF config before resuming submission after GT reset
@ 2026-11-18 23:39 FNU VISHWANATHA
2026-11-18 23:56 ` FNU VISHWANATHA
2026-11-19 0:02 ` sashiko-bot
0 siblings, 2 replies; 4+ messages in thread
From: FNU VISHWANATHA @ 2026-11-18 23:39 UTC (permalink / raw)
To: intel-xe
Cc: Matthew Brost, Thomas Hellström, Rodrigo Vivi,
Michal Wajdeczko, Piotr Piórkowski, FNU VISHWANATHA
A GT reset reloads the GuC, which loses the configuration the PF pushed
to it, including the PF's own scheduling parameters. It is restored by
the asynchronous PF restart worker, which is only queued after
xe_uc_start() has already resumed submission and re-registered the exec
queues.
Exec queues registered with the GuC in that window are acknowledged
(SCHED_CONTEXT_MODE_DONE) but are never dispatched, so their jobs never
start and their fences never signal. This shows up as repeated "Check
job timeout ... not started" errors after the first GT reset and,
eventually, as a hung task in ttm_bo_delayed_delete() waiting for a
fence that never signals.
Restore the configuration synchronously after the GuC has been reloaded
and before calling xe_uc_start(), using xe_gt_sriov_pf_config_restart().
That pushes the PF self configuration and all non-empty VF
configurations. The rest of the PF restart (policy and VF control state)
is still done by the asynchronous worker, which also pushes the
configuration again; this is harmless.
Fixes: 1c38dd6afa4a ("drm/xe/pf: Resend PF provisioning after GT reset")
Signed-off-by: FNU VISHWANATHA <fnu.vishwanatha@intel.com>
---
drivers/gpu/drm/xe/xe_gt.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
index fe71cb011d3a..a5f7a4bf4d94 100644
--- a/drivers/gpu/drm/xe/xe_gt.c
+++ b/drivers/gpu/drm/xe/xe_gt.c
@@ -35,6 +35,7 @@
#include "xe_gt_mcr.h"
#include "xe_gt_printk.h"
#include "xe_gt_sriov_pf.h"
+#include "xe_gt_sriov_pf_config.h"
#include "xe_gt_sriov_vf.h"
#include "xe_gt_stats.h"
#include "xe_gt_sysfs.h"
@@ -891,6 +892,18 @@ static int do_gt_restart(struct xe_gt *gt)
/* Get CCS mode in sync between sw/hw */
xe_gt_apply_ccs_mode(gt);
+ /*
+ * The GuC was reloaded and lost the PF configuration pushed to it,
+ * including the scheduling parameters. Restore it before
+ * xe_uc_start() resumes submission: exec queues re-registered with
+ * the GuC under its default configuration may be accepted but never
+ * dispatched, leaving their jobs unstarted and their fences unsignaled.
+ * The asynchronous PF restart below pushes the configuration again,
+ * which is harmless.
+ */
+ if (IS_SRIOV_PF(gt_to_xe(gt)))
+ xe_gt_sriov_pf_config_restart(gt);
+
err = xe_uc_start(>->uc);
if (err)
return err;
--
2.34.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] drm/xe/pf: Restore PF config before resuming submission after GT reset
2026-11-18 23:39 [PATCH] drm/xe/pf: Restore PF config before resuming submission after GT reset FNU VISHWANATHA
@ 2026-11-18 23:56 ` FNU VISHWANATHA
2026-11-19 1:23 ` Ceraolo Spurio, Daniele
2026-11-19 0:02 ` sashiko-bot
1 sibling, 1 reply; 4+ messages in thread
From: FNU VISHWANATHA @ 2026-11-18 23:56 UTC (permalink / raw)
To: intel-xe
Cc: Daniele Ceraolo Spurio, Piotr Piórkowski, Tomasz Lis,
Matthew Brost, Thomas Hellström, Rodrigo Vivi,
Michal Wajdeczko, FNU VISHWANATHA
Adding Daniele and Tomasz to Cc for review, together with Piotr and
Matthew. The patch moves the PF configuration restore in do_gt_restart()
ahead of xe_uc_start() after a GT reset.
Patchwork: https://patchwork.freedesktop.org/series/178984/
Thanks,
Vishwanatha
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] drm/xe/pf: Restore PF config before resuming submission after GT reset
2026-11-18 23:39 [PATCH] drm/xe/pf: Restore PF config before resuming submission after GT reset FNU VISHWANATHA
2026-11-18 23:56 ` FNU VISHWANATHA
@ 2026-11-19 0:02 ` sashiko-bot
1 sibling, 0 replies; 4+ messages in thread
From: sashiko-bot @ 2026-11-19 0:02 UTC (permalink / raw)
To: FNU VISHWANATHA, intel-xe
Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] drm/xe/pf: dma_fence deadlock and reclaim hazard in
do_gt_restart()
- [Medium] drm/xe/pf: do_gt_restart() restores PF/VF config before GuC
VGT policies
It also noticed 1 related pre-existing issue this patch does not
introduce:
- [High] drm/xe: loss of RING_IDLEDLY workarounds in adjust_idledly()
across GT reset and resume [1]
--
commit 19aac698e33fa931358e6f2c835ac921dba22d3f
Author: FNU VISHWANATHA <fnu.vishwanatha@intel.com>
drm/xe/pf: Restore PF config before resuming submission after GT reset
This commit calls xe_gt_sriov_pf_config_restart() synchronously in
do_gt_restart() before xe_uc_start() resumes GuC submission after a GT
reset.
> diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c
> index b751b5044f3ca..f0539440e7702 100644
> --- a/drivers/gpu/drm/xe/xe_gt.c
> +++ b/drivers/gpu/drm/xe/xe_gt.c
[ ... ]
> @@ -891,6 +892,18 @@ static int do_gt_restart(struct xe_gt *gt)
> /* Get CCS mode in sync between sw/hw */
> xe_gt_apply_ccs_mode(gt);
>
> + /*
> + * The GuC was reloaded and lost the PF configuration pushed to it,
> + * including the scheduling parameters. Restore it before
> + * xe_uc_start() resumes submission: exec queues re-registered with
> + * the GuC under its default configuration may be accepted but never
> + * dispatched, leaving their jobs unstarted and their fences unsignaled.
> + * The asynchronous PF restart below pushes the configuration again,
> + * which is harmless.
> + */
> + if (IS_SRIOV_PF(gt_to_xe(gt)))
> + xe_gt_sriov_pf_config_restart(gt);
> +
> err = xe_uc_start(>->uc);
> if (err)
> return err;
[Severity: High]
Can calling xe_gt_sriov_pf_config_restart() synchronously in do_gt_restart()
deadlock on xe->sriov.pf.master_lock and violate dma_fence reclaim rules?
Commit a4d1c5d0b99b ("drm/xe/pf: Move VFs reprovisioning to worker") moved
xe_gt_sriov_pf_config_restart() to an asynchronous worker because it takes
xe_gt_sriov_pf_master_mutex(gt), which is held across GFP_KERNEL allocations
and GPU migration dma_fence waits. During a GT reset, xe_uc_stop() stops
GuC submission without signaling pending jobs on kernel or VM queues such as
tile->migrate->q; those jobs only complete after do_gt_restart() reaches
xe_uc_start(). Meanwhile, xe_gt_sriov_pf_stop_prepare() cancels only the PF
restart worker and leaves other master_lock holders running:
CPU0 (VF provisioning / FLR) CPU1 (gt_reset_worker)
---------------------------- ----------------------
pf_provision_vf_lmem() /
xe_gt_sriov_pf_config_sanitize()
mutex_lock(master_lock)
gt_reset_worker()
xe_uc_stop()
xe_migrate_clear()
dma_fence_wait(...)
do_gt_restart()
xe_gt_sriov_pf_config_restart()
mutex_lock(master_lock)
...
xe_uc_start() <-- never reached
*** DEADLOCK ***
In pf_provision_vf_lmem() (via TTM BO creation and drm_suballoc_new()) and
in pf_sanitize_lmem() when suballocating batch buffers for large VF LMEM
clears, the wait under master_lock is uninterruptible with
MAX_SCHEDULE_TIMEOUT, wedging gt_reset_worker() on gt->ordered_wq.
[Severity: Medium]
Does calling xe_gt_sriov_pf_config_restart() and xe_uc_start() here before
xe_gt_sriov_pf_policy_restart() runs in the async pf_restart() worker invert
the policy-before-config restore order?
When the GuC is reloaded in xe_uc_load_hw(), VGT policies reset to hardware
defaults (sched_if_idle disabled, default priority GUC_SCHED_PRIORITY_LOW,
and scheduler groups disabled). However, encode_config_sched() consults the
software policy state when encoding PF and VF configs:
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c:custom_sched_priority() {
return xe_gt_sriov_pf_policy_get_sched_if_idle_locked(gt) ?
priority != GUC_SCHED_PRIORITY_NORMAL :
priority != GUC_SCHED_PRIORITY_LOW;
}
If sched_if_idle is enabled in software and a function uses
GUC_SCHED_PRIORITY_NORMAL, custom_sched_priority() returns false and omits
GUC_KLV_VF_CFG_SCHED_PRIORITY_KEY, assuming GUC_KLV_VGT_POLICY_SCHED_IF_IDLE_KEY
was already sent to the GuC. Conversely, if a function has a custom
priority while sched_if_idle is enabled, the later async
xe_gt_sriov_pf_policy_restart() sends SCHED_IF_IDLE=1 and resets the GuC's
priority back to GUC_SCHED_PRIORITY_NORMAL while submission is active.
In addition, xe_uc_start() re-registers contexts before
pf_reprovision_sched_groups() runs, and as noted in
pf_provision_sched_groups(), the GuC silently ignores scheduler group
provisioning if any MLRC contexts are already registered.
[Severity: High]
This problem wasn't introduced by this patch, but in do_gt_restart() the
engine register restoration loop just above only reapplies hwe->reg_sr via
xe_reg_sr_apply_mmio() and does not call adjust_idledly() [1].
During initial probe, hw_engine_init() calls adjust_idledly() to apply
Wa_16023105232 and Wa_14025941587 directly via xe_mmio_write32() on
RING_IDLEDLY without recording the change in hwe->reg_sr. Across a full GT
reset or system resume, RING_IDLEDLY reverts to hardware defaults and the
GuC ADS save/restore list is repopulated in xe_uc_load_hw(), so the bounded
IDLE_DELAY and cleared INHIBIT_SWITCH_UNTIL_PREEMPTED bit are lost, which
can cause preemption failures or GPU hangs.
[1] https://sashiko.dev/#/bug/linux-4061257d-241c-4f91-b214-9c272d968ab3
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118233946.275114-1-fnu.vishwanatha@intel.com?part=1
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] drm/xe/pf: Restore PF config before resuming submission after GT reset
2026-11-18 23:56 ` FNU VISHWANATHA
@ 2026-11-19 1:23 ` Ceraolo Spurio, Daniele
0 siblings, 0 replies; 4+ messages in thread
From: Ceraolo Spurio, Daniele @ 2026-11-19 1:23 UTC (permalink / raw)
To: FNU VISHWANATHA, intel-xe
Cc: Piotr Piórkowski, Tomasz Lis, Matthew Brost,
Thomas Hellström, Rodrigo Vivi, Michal Wajdeczko
The GuC should never silently drop submissions, so we might have a GuC
bug here. Even if the driver is doing the wrong thing, the GuC should at
least return an error to inform us.
I'll loop in one of the GuC devs and see if they have any feedback.
Daniele
On 11/18/2026 3:56 PM, FNU VISHWANATHA wrote:
> Adding Daniele and Tomasz to Cc for review, together with Piotr and
> Matthew. The patch moves the PF configuration restore in do_gt_restart()
> ahead of xe_uc_start() after a GT reset.
>
> Patchwork: https://patchwork.freedesktop.org/series/178984/
>
> Thanks,
> Vishwanatha
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-19 1:23 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 23:39 [PATCH] drm/xe/pf: Restore PF config before resuming submission after GT reset FNU VISHWANATHA
2026-11-18 23:56 ` FNU VISHWANATHA
2026-11-19 1:23 ` Ceraolo Spurio, Daniele
2026-11-19 0:02 ` sashiko-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox