Intel-XE Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>
To: Matthew Auld <matthew.auld@intel.com>,
	Matthew Brost	 <matthew.brost@intel.com>,
	Himal Prasad Ghimiray	 <himal.prasad.ghimiray@intel.com>,
	Stuart Summers <stuart.summers@intel.com>
Cc: intel-xe@lists.freedesktop.org
Subject: Re: [PATCH 2/5] drm/xe: Requeue the rebind worker instead of retrying in-line on -EAGAIN
Date: Thu, 19 Nov 2026 18:26:43 +0100	[thread overview]
Message-ID: <90e8b4228e4233d89c5295fec195f804a8a487de.camel@linux.intel.com> (raw)
In-Reply-To: <d88e9520-7e9e-45c2-9179-e5c50e86d2ae@intel.com>

On Thu, 2026-11-19 at 16:29 +0000, Matthew Auld wrote:
> On 19/11/2026 12:51, Thomas Hellström wrote:
> > On Thu, 2026-11-19 at 11:14 +0000, Matthew Auld wrote:
> > > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > > 
> > > When the preempt rebind worker needs to retry, it currently jumps
> > > back
> > > and keeps holding vm->lock in write mode across the retry. Only
> > > VFs
> > > with migration support drop the lock and requeue the worker.
> > > 
> > > Holding the vm lock across retries needlessly blocks other users
> > > of
> > > the
> > > lock and other work on the ordered workqueue, and makes the retry
> > > unfit
> > > for conditions that may last for a while, like repeatedly
> > > invalidated
> > > userptrs.
> > > 
> > > Always drop the vm lock and requeue the worker on -EAGAIN. This
> > > also
> > > fixes a leak of the preempt fences allocated in the failed
> > > attempt on
> > > the existing VF path.
> > 
> > Do we need a Fixes: here?
> 
> Yeah, should I also split out the leak fix? Feels like a separate 
> change? 

Yes please do.

> And do we want a fixes tag also for the new requeue behaviour?

Hm. You mean if we want to backport the whole series to fix the
timeout?

/Thomas


> 
> > 
> > Also needs a separate reviewer.
> 
> I don't think I touched anything other than adding my sob:
> 
> Reviewed-by: Matthew Auld <matthew.auld@intel.com>
> 
> > 
> > > 
> > > The userptr invalidation injection state that forces one retry
> > > per
> > > rebind sequence was a local variable that would be reset by a
> > > requeue.
> > > Move it to the vm.
> > > 
> > > Signed-off-by: Thomas Hellström
> > > <thomas.hellstrom@linux.intel.com>
> > > Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> > > ---
> > >   drivers/gpu/drm/xe/xe_vm.c       | 59
> > > ++++++++++++++++++++++++++----
> > > ----------
> > >   drivers/gpu/drm/xe/xe_vm_types.h |  8 ++++++
> > >   2 files changed, 46 insertions(+), 21 deletions(-)
> > > 
> > > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > > b/drivers/gpu/drm/xe/xe_vm.c
> > > index 75be5e5329bd..ff8d355728fc 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > @@ -542,6 +542,31 @@ void xe_vm_resume_rebind_worker(struct xe_vm
> > > *vm)
> > >   	queue_work(vm->xe->ordered_wq, &vm-
> > > >preempt.rebind_work);
> > >   }
> > >   
> > > +#if IS_ENABLED(CONFIG_DRM_XE_USERPTR_INVAL_INJECT)
> > > +/*
> > > + * Force one retry per rebind sequence. The sequence spans
> > > requeues
> > > of the
> > > + * worker, so the state must live in the vm.
> > > + */
> > > +static bool xe_vm_rebind_worker_inject_retry(struct xe_vm *vm)
> > > +{
> > > +	return !vm->preempt.inval_inject_tries++;
> > > +}
> > > +
> > > +static void xe_vm_rebind_worker_reset_inject(struct xe_vm *vm)
> > > +{
> > > +	vm->preempt.inval_inject_tries = 0;
> > > +}
> > > +#else
> > > +static bool xe_vm_rebind_worker_inject_retry(struct xe_vm *vm)
> > > +{
> > > +	return false;
> > > +}
> > > +
> > > +static void xe_vm_rebind_worker_reset_inject(struct xe_vm *vm)
> > > +{
> > > +}
> > > +#endif
> > > +
> > >   static void preempt_rebind_work_func(struct work_struct *w)
> > >   {
> > >   	struct xe_vm *vm = container_of(w, struct xe_vm,
> > > preempt.rebind_work);
> > > @@ -551,7 +576,6 @@ static void preempt_rebind_work_func(struct
> > > work_struct *w)
> > >   	LIST_HEAD(preempt_fences);
> > >   	int err = 0;
> > >   	long wait;
> > > -	int __maybe_unused tries = 0;
> > >   
> > >   	xe_assert(vm->xe, xe_vm_in_preempt_fence_mode(vm));
> > >   	trace_xe_vm_rebind_worker_enter(vm);
> > > @@ -564,7 +588,6 @@ static void preempt_rebind_work_func(struct
> > > work_struct *w)
> > >   		return;
> > >   	}
> > >   
> > > -retry:
> > >   	if (!try_wait_for_completion(&vm->xe->pm_block) &&
> > > vm_suspend_rebind_worker(vm)) {
> > >   		up_write(&vm->lock);
> > >   		/* We don't actually block but don't make
> > > progress.
> > > */
> > > @@ -614,20 +637,14 @@ static void preempt_rebind_work_func(struct
> > > work_struct *w)
> > >   		goto out_unlock;
> > >   	}
> > >   
> > > -#define retry_required(__tries, __vm) \
> > > -	(IS_ENABLED(CONFIG_DRM_XE_USERPTR_INVAL_INJECT) ? \
> > > -	(!(__tries)++ || __xe_vm_userptr_needs_repin(__vm)) : \
> > > -	__xe_vm_userptr_needs_repin(__vm))
> > > -
> > >   	xe_svm_notifier_lock(vm);
> > > -	if (retry_required(tries, vm)) {
> > > +	if (xe_vm_rebind_worker_inject_retry(vm) ||
> > > +	    __xe_vm_userptr_needs_repin(vm)) {
> > >   		xe_svm_notifier_unlock(vm);
> > >   		err = -EAGAIN;
> > >   		goto out_unlock;
> > >   	}
> > >   
> > > -#undef retry_required
> > > -
> > >   	xe_vm_bump_lru(vm, &exec);
> > >   
> > >   	/* Point of no return. */
> > > @@ -642,20 +659,20 @@ static void preempt_rebind_work_func(struct
> > > work_struct *w)
> > >   		trace_xe_vm_rebind_worker_retry(vm);
> > >   
> > >   		/*
> > > -		 * We can't block in workers on a VF which
> > > supports
> > > migration
> > > -		 * given this can block the VF post-migration
> > > workers from
> > > -		 * getting scheduled.
> > > +		 * Drop all locks and requeue rather than
> > > retrying
> > > in-line.
> > > +		 * This avoids blocking other work items, such
> > > as VF
> > > +		 * post-migration workers, and lets other users
> > > of
> > > the vm
> > > +		 * lock make progress between attempts.
> > >   		 */
> > > -		if (IS_SRIOV_VF(vm->xe) &&
> > > -		    xe_sriov_vf_migration_supported(vm->xe)) {
> > > -			up_write(&vm->lock);
> > > -			xe_vm_queue_rebind_worker(vm);
> > > -			return;
> > > -		}
> > > -
> > > -		goto retry;
> > > +		up_write(&vm->lock);
> > > +		free_preempt_fences(&preempt_fences);
> > > +		xe_vm_queue_rebind_worker(vm);
> > > +		trace_xe_vm_rebind_worker_exit(vm);
> > > +		return;
> > >   	}
> > >   
> > > +	xe_vm_rebind_worker_reset_inject(vm);
> > > +
> > >   	if (err) {
> > >   		drm_warn(&vm->xe->drm, "VM worker error: %d\n",
> > > err);
> > >   		xe_vm_kill(vm, true);
> > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h
> > > b/drivers/gpu/drm/xe/xe_vm_types.h
> > > index fb9305124679..756cf2dbee8a 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > @@ -350,6 +350,14 @@ struct xe_vm {
> > >   		 * kicked on resume.
> > >   		 */
> > >   		struct list_head pm_activate_link;
> > > +#if IS_ENABLED(CONFIG_DRM_XE_USERPTR_INVAL_INJECT)
> > > +		/**
> > > +		 * @inval_inject_tries: Number of rebind
> > > attempts in
> > > the
> > > +		 * current rebind sequence. Used to force a
> > > retry
> > > for testing.
> > > +		 * Protected by @lock.
> > > +		 */
> > > +		unsigned int inval_inject_tries;
> > > +#endif
> > >   	} preempt;
> > >   
> > >   	/** @exec_queues: Manages list of exec queues attached
> > > to
> > > this VM, protected by lock. */

  reply	other threads:[~2026-11-19 17:27 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 11:14 [PATCH 0/5] Non-fatal repin retries and NUMA-local rebind worker Matthew Auld
2026-11-19 11:14 ` [PATCH 1/5] drm/xe/vm: restore userptr invalidations on VMA op unwind Matthew Auld
2026-11-19 12:48   ` Thomas Hellström
2026-11-19 16:35   ` Himal prasad Ghimiray
2026-11-19 11:14 ` [PATCH 2/5] drm/xe: Requeue the rebind worker instead of retrying in-line on -EAGAIN Matthew Auld
2026-11-19 12:51   ` Thomas Hellström
2026-11-19 16:29     ` Matthew Auld
2026-11-19 17:26       ` Thomas Hellström [this message]
2026-11-19 17:38         ` Matthew Auld
2026-11-19 16:36   ` Himal prasad Ghimiray
2026-11-19 11:14 ` [PATCH 3/5] drm/xe: Add a userptr repin timeout sysfs attribute Matthew Auld
2026-11-19 16:38   ` Himal prasad Ghimiray
2026-11-19 11:14 ` [PATCH 4/5] drm/xe/userptr: Make the repin retry loop non-fatal Matthew Auld
2026-11-19 13:03   ` Thomas Hellström
2026-11-19 17:10   ` Himal prasad Ghimiray
2026-11-19 11:14 ` [PATCH 5/5] drm/xe: run compute rebind worker on device NUMA node Matthew Auld
2026-11-19 13:11   ` Thomas Hellström
2026-11-19 13:38     ` Matthew Auld
2026-11-19 13:16 ` ✓ CI.KUnit: success for Non-fatal repin retries and NUMA-local rebind worker Patchwork
2026-11-19 13:54 ` ✓ Xe.CI.BAT: " Patchwork

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=90e8b4228e4233d89c5295fec195f804a8a487de.camel@linux.intel.com \
    --to=thomas.hellstrom@linux.intel.com \
    --cc=himal.prasad.ghimiray@intel.com \
    --cc=intel-xe@lists.freedesktop.org \
    --cc=matthew.auld@intel.com \
    --cc=matthew.brost@intel.com \
    --cc=stuart.summers@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox