Re: [Intel-xe] [PATCH v4 6/7] drm/xe/tlb: also update seqno_recv during reset

[Matthew Auld <matthew.auld@intel.com>]

On 06/07/2023 05:05, Matthew Brost wrote:
> On Wed, Jul 05, 2023 at 05:06:09PM +0100, Matthew Auld wrote:
>> We might have various kworkers waiting for TLB flushes to complete which
>> are not tracked with an explicit TLB fence, however at this stage that
>> will never happen since the CT is already disabled, so make sure we
>> signal them here under the assumption that we have completed a full GT
>> reset.
>>
>> Signed-off-by: Matthew Auld <matthew.auld@intel.com>
>> Cc: Matthew Brost <matthew.brost@intel.com>
>> Cc: José Roberto de Souza <jose.souza@intel.com>
>> ---
>>   drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c | 18 +++++++++++++++++-
>>   1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c
>> index b38da572d268..51789ec9ad57 100644
>> --- a/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c
>> +++ b/drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c
>> @@ -89,10 +89,26 @@ invalidation_fence_signal(struct xe_gt_tlb_invalidation_fence *fence)
>>    void xe_gt_tlb_invalidation_reset(struct xe_gt *gt)
>>   {
>>   	struct xe_gt_tlb_invalidation_fence *fence, *next;
>> +	struct xe_guc *guc = &gt->uc.guc;
>>   
>> +	/*
>> +	 * CT channel is already disabled at this point. No new TLB requests can
>> +	 * appear.
>> +	 */
>> +
>> +	mutex_lock(&gt->uc.guc.ct.lock);
>>   	cancel_delayed_work(&gt->tlb_invalidation.fence_tdr);
>> +	/*
>> +	 * We might have various kworkers waiting for TLB flushes to complete
>> +	 * which are not tracked with an explicit TLB fence, however at this
>> +	 * stage that will never happen since the CT is already disabled, so
>> +	 * make sure we signal them here under the assumption that we have
>> +	 * completed a full GT reset.
>> +	 */
>> +	gt->tlb_invalidation.seqno_recv = gt->tlb_invalidation.seqno;
>> +	smp_wmb();
> 
> The smp_wmb() probably isn't needed, this my mistake and have this wrong
> in a places in the code. Barriers are not my strong point though so
> maybe double check on this.

I think we usually need some kind of barrier on the reader side also, so 
here this would likely be at the start of tlb_invalidation_seqno_past() 
or so, which is called by wait_event_timeout().

But reading[1] under the section "SLEEP AND WAKE-UP FUNCTIONS" it looks 
like wait_event_timeout() and wake_up() will already have the correct 
barriers for us wrt writing seqno_recv before the wake_up() and reading 
it from wait_event_timeout(). I can try to type a patch.

[1] https://www.kernel.org/doc/Documentation/memory-barriers.txt

> 
> Otherwise LGTM.
> 
> With that:
> Reviewed-by: Matthew Brost <matthew.brost@intel.com>

Thanks.

> 
>> +	wake_up_all(&guc->ct.wq);
>>   
>> -	mutex_lock(&gt->uc.guc.ct.lock);
>>   	list_for_each_entry_safe(fence, next,
>>   				 &gt->tlb_invalidation.pending_fences, link)
>>   		invalidation_fence_signal(fence);
>> -- 
>> 2.41.0
>>