Re: [Intel-xe] [PATCH 3/3] drm/xe: Use Xe ordered workqueue for rebind worker

["Thomas Hellström" <thomas.hellstrom@linux.intel.com>]

On Fri, 2023-06-09 at 11:26 -0700, Matthew Brost wrote:
> A mix of the system unbound wq and Xe ordered wq was used for the
> rebind, only use the Xe ordered wq. This will ensure only 1 rebind is
> occuring at a time providing a somewhat clunky work around for short
> comings in TTM wrt to memory contention. Once the TTM memory
> contention
> is resolved we should be able to use a dedicated non-ordered
> workqueue.
> 
> Also add helper to queue rebind worker to avoid using wrong workqueue
> going forward.

Should probably add a mutex as well to protect against competing !LR
execbufs. But let's do a follow-up if needed.

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> 
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_preempt_fence.c | 2 +-
>  drivers/gpu/drm/xe/xe_vm.c            | 3 +--
>  drivers/gpu/drm/xe/xe_vm.h            | 6 ++++++
>  3 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_preempt_fence.c
> b/drivers/gpu/drm/xe/xe_preempt_fence.c
> index 78ad8c209873..219eefeb90ff 100644
> --- a/drivers/gpu/drm/xe/xe_preempt_fence.c
> +++ b/drivers/gpu/drm/xe/xe_preempt_fence.c
> @@ -25,7 +25,7 @@ static void preempt_fence_work_func(struct
> work_struct *w)
>         dma_fence_signal(&pfence->base);
>         dma_fence_end_signalling(cookie);
>  
> -       queue_work(system_unbound_wq, &e->vm->preempt.rebind_work);
> +       xe_vm_queue_rebind_worker(e->vm);
>  
>         xe_engine_put(e);
>  }
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 1966a583b761..208e2f0bf41d 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -3103,8 +3103,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev,
> void *data, struct drm_file *file)
>  
>                         /* Rebinds may have been blocked, give worker
> a kick */
>                         if (xe_vm_in_compute_mode(vm))
> -                               queue_work(vm->xe->ordered_wq,
> -                                          &vm->preempt.rebind_work);
> +                               xe_vm_queue_rebind_worker(vm);
>                 }
>  
>                 goto put_engine;
> diff --git a/drivers/gpu/drm/xe/xe_vm.h b/drivers/gpu/drm/xe/xe_vm.h
> index 858207a20017..6086c455a51d 100644
> --- a/drivers/gpu/drm/xe/xe_vm.h
> +++ b/drivers/gpu/drm/xe/xe_vm.h
> @@ -134,6 +134,12 @@ int xe_vma_userptr_pin_pages(struct xe_vma
> *vma);
>  
>  int xe_vma_userptr_check_repin(struct xe_vma *vma);
>  
> +static inline void xe_vm_queue_rebind_worker(struct xe_vm *vm)
> +{
> +       XE_WARN_ON(!xe_vm_in_compute_mode(vm));
> +       queue_work(vm->xe->ordered_wq, &vm->preempt.rebind_work);
> +}
> +
>  /*
>   * XE_ONSTACK_TV is used to size the tv_onstack array that is input
>   * to xe_vm_lock_dma_resv() and xe_vm_unlock_dma_resv().