Re: [PATCH v2 18/32] drm/xe/vm: Add attributes struct as member of vma

[Matthew Brost <matthew.brost@intel.com>]

On Tue, May 20, 2025 at 02:57:45PM +0530, Ghimiray, Himal Prasad wrote:
> 
> 
> On 15-05-2025 00:06, Matthew Brost wrote:
> > On Mon, Apr 07, 2025 at 03:47:05PM +0530, Himal Prasad Ghimiray wrote:
> > > The attribute of xe_vma will determine the migration policy and the
> > > encoding of the page table entries (PTEs) for that vma.
> > > This attribute helps manage how memory pages are moved and how their
> > > addresses are translated. It will be used by madvise to set the
> > > behavior of the vma.
> > > 
> > > Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
> > > ---
> > >   drivers/gpu/drm/xe/xe_vm.c       |  6 ++++++
> > >   drivers/gpu/drm/xe/xe_vm_types.h | 20 ++++++++++++++++++++
> > >   2 files changed, 26 insertions(+)
> > > 
> > > diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> > > index 27a8dbe709c2..1ff9e477e061 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm.c
> > > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > > @@ -2470,6 +2470,12 @@ static struct xe_vma *new_vma(struct xe_vm *vm, struct drm_gpuva_op_map *op,
> > >   		vma = ERR_PTR(err);
> > >   	}
> > > +	/*TODO: assign devmem_fd of local vram once multi device
> > > +	 * support is added.
> > > +	 */
> > > +	vma->attr.preferred_loc.devmem_fd = 1;
> > 
> > Assigning a value of '1' is a bit odd... I'd prefer using a define or
> > something similar to indicate the intended behavior. I noticed a few
> > other assignments to '1' in the final result—same comment applies to
> > those.
> 
> Sure
> 
> > 
> > > +	vma->attr.atomic_access = DRM_XE_VMA_ATOMIC_UNDEFINED;
> > > +
> > >   	return vma;
> > >   }
> > > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
> > > index d3c1209348e9..5f5feffecb82 100644
> > > --- a/drivers/gpu/drm/xe/xe_vm_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> > > @@ -77,6 +77,19 @@ struct xe_userptr {
> > >   #endif
> > >   };
> > > +/**
> > > + * struct xe_vma_mem_attr - memory attributes associated with vma
> > > + */
> > > +struct xe_vma_mem_attr {
> > > +	/** @preferred_loc: perferred memory_location*/
> > > +	struct {
> > > +		u32 migration_policy; /* represents migration policies */
> > > +		u32 devmem_fd; /* devmem_fd used for determining pagemap_fd requested by user */
> > > +	} preferred_loc;
> > 
> > I'm a little unclear on how these variables work.
> > 
> > In the uAPI for migration_policy, I see MIGRATE_ALL_PAGES and
> > MIGRATE_ONLY_SYSTEM_PAGES (these should probably be normalized with a
> > DRM_XE_* prefix, by the way), but it's unclear to me what exactly these
> > mean or how they're used based on the final result—could you clarify?
> 
> With multi-device support the idea was to have flexibility to move only
> system pages to preferred location or also move pages from other vram
> location to preferred location.
> 

Ok, I think having bits set aside for this makes sense.

> > 
> > Likewise, I'm confused about the devmem_fd usage. It can either be
> > assigned a devmem_fd from the uAPI, but in some cases, it's interpreted
> > as a region. I assume this is anticipating multi-GPU support, but again,
> > the plan isn't clear to me. Could you explain?
> 
> The devmem_fd is intended to be used to determine the struct drm_pagemap *,
> which in turn will be used to identify the tile associated with VRAM for
> allocation and binding. The changes that introduce the
> devmem_fd->drm_pagemap->tile [1] linkage will be part of the upcoming
> multi-GPU support.
> 
> To ensure that the current changes are easily scalable and can be extended
> for multi-GPU support, I am defining devmem_fd in the UAPI and using it in
> the KMD as a region placeholder until multi-GPU support is integrated.
> 

Hmm, will this break the uAPI though? e.g. How does the UMD choose
between region and FD at the uAPI level? If the answer is once multi-GPU
lands it is always a FD rather than region then we really need to land
some of the multi-GPU patches at same time as madvise - at least the
ones which export memory regions as FDs.

Let's loop in Thomas on the multi-GPU assumptions too to ensure
correctness.

Matt 

> [1] https://patchwork.freedesktop.org/patch/642773/?series=146227&rev=1
> 
> 
> > 
> > In general I agree with the idea of xe_vma_mem_attr though.
> > 
> > Matt
> > 
> > > +	/** @atomic_access: The atomic access type for the vma */
> > > +	u32 atomic_access;
> > > +};
> > > +
> > >   struct xe_vma {
> > >   	/** @gpuva: Base GPUVA object */
> > >   	struct drm_gpuva gpuva;
> > > @@ -128,6 +141,13 @@ struct xe_vma {
> > >   	 * Needs to be signalled before UNMAP can be processed.
> > >   	 */
> > >   	struct xe_user_fence *ufence;
> > > +
> > > +	/**
> > > +	 * @attr: The attributes of vma which determines the migration policy
> > > +	 * and encoding of the PTEs for this vma.
> > > +	 */
> > > +	struct xe_vma_mem_attr attr;
> > > +
> > >   };
> > >   /**
> > > -- 
> > > 2.34.1
> > > 
>