Re: [PATCH 1/4] drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups

[Matthew Brost <matthew.brost@intel.com>]

On Mon, Sep 22, 2025 at 10:47:07AM -0700, Matthew Brost wrote:
> On Thu, Sep 18, 2025 at 01:00:04PM +0530, Satyanarayana K V P wrote:
> > The migrate VM builds the CCS metadata save/restore batch buffer (BB) in
> > advance and retains it so the GuC can submit it directly when saving a
> > VM’s state.
> > 
> > When a VM migrates between VFs, the GGTT base can change. Any GGTT-based
> > addresses embedded in the BB would then have to be parsed and patched.
> > 
> > Use PPGTT addresses in the BB (including for TLB invalidation) so the BB
> > remains GGTT-agnostic and requires no address fixups during migration.
> > 
> > Signed-off-by: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
> > Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
> > Cc: Matthew Brost <matthew.brost@intel.com>
> 
> Longterm we need a bit of cleanup in xe_migrate.c to really make it
> clear how the kernel PTEs are being used + kernel doc but as far as I
> can tell this change should work with the current code base.
> 
> With that:
> Reviewed-by: Matthew Brost <matthew.brost@intel.com>
> 

Also, feel free to send this one out by itself and merge it ahead of the
rest of the series.

Matt

> > ---
> >  drivers/gpu/drm/xe/xe_migrate.c | 28 ++++++++++++++++++++--------
> >  1 file changed, 20 insertions(+), 8 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c
> > index 6fad5d469629..d375ebc9323d 100644
> > --- a/drivers/gpu/drm/xe/xe_migrate.c
> > +++ b/drivers/gpu/drm/xe/xe_migrate.c
> > @@ -971,15 +971,27 @@ struct xe_lrc *xe_migrate_lrc(struct xe_migrate *migrate)
> >  	return migrate->q->lrc[0];
> >  }
> >  
> > -static int emit_flush_invalidate(struct xe_exec_queue *q, u32 *dw, int i,
> > -				 u32 flags)
> > +static u64 migrate_vm_ppgtt_addr_tlb_inval(void)
> >  {
> > -	struct xe_lrc *lrc = xe_exec_queue_lrc(q);
> > +	/*
> > +	 * The migrate VM is self-referential so it can modify its own PTEs (see
> > +	 * pte_update_size() or emit_pte() functions). We reserve NUM_KERNEL_PDE
> > +	 * entries for kernel operations (copies, clears, CCS migrate), and
> > +	 * suballocate the rest to user operations (binds/unbinds). With
> > +	 * NUM_KERNEL_PDE = 15, NUM_KERNEL_PDE - 1 is already used for PTE updates,
> > +	 * so assign NUM_KERNEL_PDE - 2 for TLB invalidation.
> > +	 */
> > +	return (NUM_KERNEL_PDE - 2) * XE_PAGE_SIZE;
> > +}
> > +
> > +static int emit_flush_invalidate(u32 *dw, int i, u32 flags)
> > +{
> > +	u64 addr = migrate_vm_ppgtt_addr_tlb_inval();
> > +
> >  	dw[i++] = MI_FLUSH_DW | MI_INVALIDATE_TLB | MI_FLUSH_DW_OP_STOREDW |
> >  		  MI_FLUSH_IMM_DW | flags;
> > -	dw[i++] = lower_32_bits(xe_lrc_start_seqno_ggtt_addr(lrc)) |
> > -		  MI_FLUSH_DW_USE_GTT;
> > -	dw[i++] = upper_32_bits(xe_lrc_start_seqno_ggtt_addr(lrc));
> > +	dw[i++] = lower_32_bits(addr);
> > +	dw[i++] = upper_32_bits(addr);
> >  	dw[i++] = MI_NOOP;
> >  	dw[i++] = MI_NOOP;
> >  
> > @@ -1092,11 +1104,11 @@ int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q,
> >  
> >  		emit_pte(m, bb, ccs_pt, false, false, &ccs_it, ccs_size, src);
> >  
> > -		bb->len = emit_flush_invalidate(q, bb->cs, bb->len, flush_flags);
> > +		bb->len = emit_flush_invalidate(bb->cs, bb->len, flush_flags);
> >  		flush_flags = xe_migrate_ccs_copy(m, bb, src_L0_ofs, src_is_pltt,
> >  						  src_L0_ofs, dst_is_pltt,
> >  						  src_L0, ccs_ofs, true);
> > -		bb->len = emit_flush_invalidate(q, bb->cs, bb->len, flush_flags);
> > +		bb->len = emit_flush_invalidate(bb->cs, bb->len, flush_flags);
> >  
> >  		size -= src_L0;
> >  	}
> > -- 
> > 2.51.0
> >