Re: [Intel-xe] [PATCH v4 1/7] drm/xe: hold mem_access.ref for CT fast-path

[Matthew Brost <matthew.brost@intel.com>]

On Thu, Jul 06, 2023 at 09:29:09AM +0100, Matthew Auld wrote:
> On 06/07/2023 04:51, Matthew Brost wrote:
> > On Wed, Jul 05, 2023 at 05:06:04PM +0100, Matthew Auld wrote:
> > > Just checking xe_device_mem_access_ongoing() is not enough, we also need
> > > to hold the reference otherwise the ref can transition from 1 -> 0 as we
> > > enter g2h_read(), leading to warnings. While we can't do a full rpm sync
> > > in the IRQ, we can keep the device awake if the ref is non-zero.
> > > Introduce a new helper for this and set it to work in for the CT
> > > fast-path.
> > > 
> > > Signed-off-by: Matthew Auld <matthew.auld@intel.com>
> > > Cc: Matthew Brost <matthew.brost@intel.com>
> > > Cc: José Roberto de Souza <jose.souza@intel.com>
> > > ---
> > >   drivers/gpu/drm/xe/xe_device.c | 5 +++++
> > >   drivers/gpu/drm/xe/xe_device.h | 1 +
> > >   drivers/gpu/drm/xe/xe_guc_ct.c | 5 ++++-
> > >   3 files changed, 10 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> > > index 07ae208af809..94b0089b0dee 100644
> > > --- a/drivers/gpu/drm/xe/xe_device.c
> > > +++ b/drivers/gpu/drm/xe/xe_device.c
> > > @@ -412,6 +412,11 @@ u32 xe_device_ccs_bytes(struct xe_device *xe, u64 size)
> > >   		DIV_ROUND_UP(size, NUM_BYTES_PER_CCS_BYTE) : 0;
> > >   }
> > > +bool xe_device_mem_access_get_if_ongoing(struct xe_device *xe)
> > > +{
> > > +	return atomic_inc_not_zero(&xe->mem_access.ref);
> > > +}
> > > +
> > >   void xe_device_mem_access_get(struct xe_device *xe)
> > >   {
> > >   	bool resumed = xe_pm_runtime_resume_if_suspended(xe);
> > > diff --git a/drivers/gpu/drm/xe/xe_device.h b/drivers/gpu/drm/xe/xe_device.h
> > > index 779f71d066e6..8e01bbadb149 100644
> > > --- a/drivers/gpu/drm/xe/xe_device.h
> > > +++ b/drivers/gpu/drm/xe/xe_device.h
> > > @@ -138,6 +138,7 @@ static inline struct xe_force_wake * gt_to_fw(struct xe_gt *gt)
> > >   }
> > >   void xe_device_mem_access_get(struct xe_device *xe);
> > > +bool xe_device_mem_access_get_if_ongoing(struct xe_device *xe);
> > >   void xe_device_mem_access_put(struct xe_device *xe);
> > >   static inline bool xe_device_mem_access_ongoing(struct xe_device *xe)
> > > diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
> > > index 22bc9ce846db..b7aecc480098 100644
> > > --- a/drivers/gpu/drm/xe/xe_guc_ct.c
> > > +++ b/drivers/gpu/drm/xe/xe_guc_ct.c
> > > @@ -1038,7 +1038,8 @@ void xe_guc_ct_fast_path(struct xe_guc_ct *ct)
> > >   	struct xe_device *xe = ct_to_xe(ct);
> > >   	int len;
> > > -	if (!xe_device_in_fault_mode(xe) || !xe_device_mem_access_ongoing(xe))
> > > +	if (!xe_device_in_fault_mode(xe) ||
> > > +	    !xe_device_mem_access_get_if_ongoing(xe))
> > >   		return;
> > >   	spin_lock(&ct->fast_lock);
> > > @@ -1048,6 +1049,8 @@ void xe_guc_ct_fast_path(struct xe_guc_ct *ct)
> > >   			g2h_fast_path(ct, ct->fast_msg, len);
> > >   	} while (len > 0);
> > >   	spin_unlock(&ct->fast_lock);
> > > +
> > > +	xe_device_mem_access_put(xe);
> > 
> > Can't this sleep if would go from 1->0, i.e. can't xe_pm_runtime_put sleep?
> 
> Thanks for the review. The rpm put() in xe_device_mem_access_put() always
> uses RPM_ASYNC underneath, and that is always safe to use from atomic
> context. The kernel-doc for __pm_runtime_suspend() says:
> 
> "This routine may be called in atomic context if the RPM_ASYNC flag is set"
> 
> It only really queues the work to run our rpm suspend callback, and never
> runs it directly if using RPM_ASYNC.
> 

Thanks the explaination, with that:

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> > 
> > Matt
> > 
> > >   }
> > >   /* Returns less than zero on error, 0 on done, 1 on more available */
> > > -- 
> > > 2.41.0
> > >