On Thu, Feb 19, 2026 at 12:21:58AM +0100, Tomasz Lis wrote:When LRC is created during fixups, it may have invalid state. Ensure that all such situations are caught, so that LRC creation can be repeated. Due to VM having arbitrarly set amount of CPU cores, it is possible to limit the amount to 1. In such case, there is a possibility that kernel will switch CPU contexts in a way which makes previously used detection methods miss a VF migration recovery running in parallel (by simply not switching to the LRC creation thread during recovery). This possibility is not only theoretical, it was revealed by testing that in a small percentage of specially crafted test cases, the resulting LRC is damaged and causes GPU hang. With the additional atomic value increased after fixups, any VF migration that avoided the usual detection during LRC creation will be caught. Signed-off-by: Tomasz Lis <tomasz.lis@intel.com> --- drivers/gpu/drm/xe/xe_exec_queue.c | 6 +++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 +++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 1 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 2 ++ 4 files changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c index 2ebf25a35557..a8d26fece38a 100644 --- a/drivers/gpu/drm/xe/xe_exec_queue.c +++ b/drivers/gpu/drm/xe/xe_exec_queue.c @@ -308,15 +308,19 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q, u32 exec_queue_flags) */ for (i = 0; i < q->width; ++i) { struct xe_lrc *lrc; + int marker; xe_gt_sriov_vf_wait_valid_default_lrc(q->gt); + marker = xe_vf_migration_fixups_complete_count(q->gt); + lrc = xe_lrc_create(q->hwe, q->vm, q->replay_state, xe_lrc_ring_size(), q->msix_vec, flags); if (IS_ERR(lrc)) { err = PTR_ERR(lrc); goto err_lrc; } - if (!xe_gt_vf_valid_default_lrc(q->gt)) { + if (!xe_gt_vf_valid_default_lrc(q->gt) || + marker != xe_vf_migration_fixups_complete_count(q->gt)) { xe_lrc_put(lrc);What exactly does this marker buy us? Couldn't patch #3 just signal 'gt->sriov.vf.migration.default_lrcs_need_fixes' where 'gt->sriov.vf.migration.fixups_complete' is incremented in this patch? Then just drop this patch?
This solves an issue which was found by test fails, so it's not theoretical (though it is a rare sporadic):
Consider a VM with one-core CPU, where migration happened while __xe_exec_queue_init() was executing, during creation of LRCs - so after xe_gt_sriov_vf_wait_valid_default_lrc() has finished, stop was inside xe_lrc_create(). It is possible that this queue creation function will be preempted and will remain without progress during the whole migration recovery. When the function finally gets back to being executed, it is already past the recovery - and xe_gt_vf_valid_default_lrc() will return true. This means the whole function will run as normal, without any code flow change caused by migration. In particular, a LRC which was partially created before migration, and partially after recovery, will be kept. There are two problems with that: one is that depending on when the CPU context was switched, this LRC may have GGTT references and may have skipped fixups. The other is that LRC created during VF migration is sometimes damaged even after fixups, so it needs to be freed and re-created - and we did not detected that, leaving the LRC as is.
The `default_lrcs_need_fixes` tells us the recovery is still in progress, but it doesn't tell us whether it already finished before and we've missed it.
I originally didn't though it was achievable, as GuC communication is slow and something will be always executed while the VF recovery is waiting for GuC. But it turns out the CPU may get switched to other tasks, leaving the queue creation starving for the whole recovery.
What can I improve in the description to make this clearer?
-Tomasz
Matti--; continue; diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_vf.c b/drivers/gpu/drm/xe/xe_gt_sriov_vf.c index ff9fb9196486..240c53b07eb3 100644 --- a/drivers/gpu/drm/xe/xe_gt_sriov_vf.c +++ b/drivers/gpu/drm/xe/xe_gt_sriov_vf.c @@ -1254,6 +1254,11 @@ static size_t post_migration_scratch_size(struct xe_device *xe) return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE); } +int xe_vf_migration_fixups_complete_count(struct xe_gt *gt) +{ + return atomic_read(>->sriov.vf.migration.fixups_complete); +} + static int vf_post_migration_fixups(struct xe_gt *gt) { void *buf = gt->sriov.vf.migration.scratch; @@ -1274,6 +1279,8 @@ static int vf_post_migration_fixups(struct xe_gt *gt) if (err) return err; + atomic_inc(>->sriov.vf.migration.fixups_complete); + return 0; } diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_vf.h b/drivers/gpu/drm/xe/xe_gt_sriov_vf.h index 8c21b8ab2f16..4651c7f3335c 100644 --- a/drivers/gpu/drm/xe/xe_gt_sriov_vf.h +++ b/drivers/gpu/drm/xe/xe_gt_sriov_vf.h @@ -41,5 +41,6 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p); bool xe_gt_vf_valid_default_lrc(struct xe_gt *gt); void xe_gt_sriov_vf_wait_valid_default_lrc(struct xe_gt *gt); +int xe_vf_migration_fixups_complete_count(struct xe_gt *gt); #endif diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h b/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h index 8be181bf3cf3..41d6199e3508 100644 --- a/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +++ b/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h @@ -54,6 +54,8 @@ struct xe_gt_sriov_vf_migration { wait_queue_head_t wq; /** @scratch: Scratch memory for VF recovery */ void *scratch; + /** @fixups_complete: Counts completed fixups stages */ + atomic_t fixups_complete; /** @debug: Debug hooks for delaying migration */ struct { /** -- 2.25.1