Re: [PATCH v4 3/8] drm/xe/madvise: Implement purgeable buffer object support

["Thomas Hellström" <thomas.hellstrom@linux.intel.com>]

On Tue, 2026-01-20 at 08:58 -0800, Matthew Brost wrote:
> On Tue, Jan 20, 2026 at 11:38:49AM +0530, Arvind Yadav wrote:
> > This allows userspace applications to provide memory usage hints to
> > the kernel for better memory management under pressure:
> > 
> > Add the core implementation for purgeable buffer objects, enabling
> > memory
> > reclamation of user-designated DONTNEED buffers during eviction.
> > 
> > This patch implements the purge operation and state machine
> > transitions:
> > 
> > Purgeable States (from xe_madv_purgeable_state):
> >  - WILLNEED (0): BO should be retained, actively used
> >  - DONTNEED (1): BO eligible for purging, not currently needed
> >  - PURGED (2): BO backing store reclaimed, permanently invalid
> > 
> > Design Rationale:
> >   - Async TLB invalidation via trigger_rebind (no blocking
> > xe_vm_invalidate_vma)
> >   - i915 compatibility: retained field, "once purged always purged"
> > semantics
> >   - Shared BO protection prevents multi-process memory corruption
> >   - Scratch PTE reuse avoids new infrastructure, safe for fault
> > mode
> > 
> > v2:
> >   - Use xe_bo_trigger_rebind() for async TLB invalidation (Thomas
> > Hellström)
> >   - Add NULL rebind with scratch PTEs for fault mode (Thomas
> > Hellström)
> >   - Implement i915-compatible retained field logic (Thomas
> > Hellström)
> >   - Skip BO validation for purged BOs in page fault handler (crash
> > fix)
> >   - Add scratch VM check in page fault path (non-scratch VMs fail
> > fault)
> >   - Force clear_pt for non-scratch VMs to avoid phys addr 0 mapping
> > (review fix)
> >   - Add !is_purged check to resource cursor setup to prevent stale
> > access
> > 
> > v3:
> >   - Rebase as xe_gt_pagefault.c is gone upstream and replaced
> >     with xe_pagefault.c (Matthew Brost)
> >   - Xe specific warn on (Matthew Brost)
> >   - Call helpers for madv_purgeable access(Matthew Brost)
> >   - Remove bo NULL check(Matthew Brost)
> >   - Use xe_bo_assert_held instead of dma assert(Matthew Brost)
> >   - Move the xe_bo_is_purged check under the dma-resv lock( by
> > Matt)
> >   - Drop is_purged from xe_pt_stage_bind_entry and just set is_null
> > to true
> >     for purged BO rename s/is_null/is_null_or_purged (by Matt)
> >   - UAPI rule should not be changed.(Matthew Brost)
> >   - Make 'retained' a userptr (Matthew Brost)
> > 
> > v4:
> >   - @madv_purgeable atomic_t → u32 change across all relevant
> > patches. (Matt)
> > 
> > Cc: Matthew Brost <matthew.brost@intel.com>
> > Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
> > Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
> > ---
> >  drivers/gpu/drm/xe/xe_bo.c         | 61 +++++++++++++++++----
> >  drivers/gpu/drm/xe/xe_pagefault.c  | 12 ++++
> >  drivers/gpu/drm/xe/xe_pt.c         | 38 +++++++++++--
> >  drivers/gpu/drm/xe/xe_vm.c         | 11 +++-
> >  drivers/gpu/drm/xe/xe_vm_madvise.c | 88
> > ++++++++++++++++++++++++++++++
> >  5 files changed, 191 insertions(+), 19 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_bo.c
> > b/drivers/gpu/drm/xe/xe_bo.c
> > index 408c74216fdf..d0a6d340b255 100644
> > --- a/drivers/gpu/drm/xe/xe_bo.c
> > +++ b/drivers/gpu/drm/xe/xe_bo.c
> > @@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo,
> >  	return 0;
> >  }
> >  
> > +/**
> > + * xe_ttm_bo_purge() - Purge buffer object backing store
> > + * @ttm_bo: The TTM buffer object to purge
> > + * @ctx: TTM operation context
> > + *
> > + * This function purges the backing store of a BO marked as
> > DONTNEED and
> > + * triggers rebind to invalidate stale GPU mappings. For fault-
> > mode VMs,
> > + * this zaps the PTEs. The next GPU access will trigger a page
> > fault and
> > + * perform NULL rebind (scratch pages or clear PTEs based on VM
> > config).
> > + */
> > +static void xe_ttm_bo_purge(struct ttm_buffer_object *ttm_bo,
> > struct ttm_operation_ctx *ctx)
> > +{
> > +	struct xe_device *xe = ttm_to_xe_device(ttm_bo->bdev);
> > +	struct xe_bo *bo = ttm_to_xe_bo(ttm_bo);
> > +
> 
> xe_bo_assert_held(bo);
> 
> > +	if (ttm_bo->ttm) {
> > +		struct ttm_placement place = {};
> > +		int ret = ttm_bo_validate(ttm_bo, &place, ctx);
> > +
> > +		drm_WARN_ON(&xe->drm, ret);
> 
> I think since 'xe' in available here, you should use xe_assert in
> place
> of drm_WARN_ON.
> 
> > +		if (!ret) {
> > +			if (xe_bo_madv_is_dontneed(bo)) {
> > +				bo->madv_purgeable =
> > XE_MADV_PURGEABLE_PURGED;
> 
> Helper to set madv_purgeable state /w lockdep assert?
> 
> Also perhaps assert valid state transitions in the helper (e.g., you
> cannot tranistion out of XE_MADV_PURGEABLE_PURGED.
> 
> > +
> > +				/*
> > +				 * Trigger rebind to invalidate
> > stale GPU mappings.
> > +				 * - Non-fault mode: Marks VMAs
> > for rebind
> > +				 * - Fault mode: Zaps PTEs (sets
> > to 0), next access triggers fault
> > +				 *   and NULL rebind with
> > scratch/clear PTEs per VM config
> > +				 */
> > +				ret = xe_bo_trigger_rebind(xe, bo,
> > ctx);
> > +				XE_WARN_ON(ret);
> 
> I think xe_bo_trigger_rebind is allowed to fail if ctx->no_wait_gpu
> is
> set. In both the faulting fast path and certain parts of the shrinker
> we
> set this. So I think any error returned from xe_bo_trigger_rebind
> needs
> to propagte up the call stack.

If possible, I think we should call xe_bo_move_notify(), which will in
turn call xe_bo_trigger_rebind() rather than call
xe_bo_trigger_rebind(), since xe_bo_move_notify() is intended to unbind
/ unmap everything needed before a bo move / purge. In this case
xe_bo_trigger_rebind() may be sufficient, but perhaps not in the
future.

> 
> > +			}
> > +		}
> > +	}
> > +}
> > +
> >  static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool
> > evict,
> >  		      struct ttm_operation_ctx *ctx,
> >  		      struct ttm_resource *new_mem,
> > @@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object
> > *ttm_bo, bool evict,
> >  				  ttm && ttm_tt_is_populated(ttm))
> > ? true : false;
> >  	int ret = 0;
> >  
> > +	/*
> > +	 * Purge only non-shared BOs explicitly marked DONTNEED by
> > userspace.
> > +	 * The move_notify callback will handle invalidation
> > asynchronously.
> > +	 */
> > +	if (evict && xe_bo_madv_is_dontneed(bo)) {
> > +		xe_ttm_bo_purge(ttm_bo, ctx);
> 
> With above, we need to send errors from xe_ttm_bo_purge up the call
> stack.
> 
> > +		return 0;
> > +	}
> > +
> >  	/* Bo creation path, moving to system or TT. */
> >  	if ((!old_mem && ttm) && !handle_system_ccs) {
> >  		if (new_mem->mem_type == XE_PL_TT)
> > @@ -1604,18 +1650,6 @@ static void
> > xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo)
> >  	}
> >  }
> >  
> > -static void xe_ttm_bo_purge(struct ttm_buffer_object *ttm_bo,
> > struct ttm_operation_ctx *ctx)
> > -{
> > -	struct xe_device *xe = ttm_to_xe_device(ttm_bo->bdev);
> > -
> > -	if (ttm_bo->ttm) {
> > -		struct ttm_placement place = {};
> > -		int ret = ttm_bo_validate(ttm_bo, &place, ctx);
> > -
> > -		drm_WARN_ON(&xe->drm, ret);
> > -	}
> > -}
> > -
> >  static void xe_ttm_bo_swap_notify(struct ttm_buffer_object
> > *ttm_bo)
> >  {
> >  	struct ttm_operation_ctx ctx = {
> > @@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct
> > xe_device *xe, struct xe_bo *bo,
> >  #endif
> >  	INIT_LIST_HEAD(&bo->vram_userfault_link);
> >  
> > +	/* Initialize purge advisory state */
> > +	bo->madv_purgeable = XE_MADV_PURGEABLE_WILLNEED;
> > +
> >  	drm_gem_private_object_init(&xe->drm, &bo->ttm.base,
> > size);
> >  
> >  	if (resv) {
> > diff --git a/drivers/gpu/drm/xe/xe_pagefault.c
> > b/drivers/gpu/drm/xe/xe_pagefault.c
> > index 6bee53d6ffc3..e3ace179e9cf 100644
> > --- a/drivers/gpu/drm/xe/xe_pagefault.c
> > +++ b/drivers/gpu/drm/xe/xe_pagefault.c
> > @@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec
> > *exec, struct xe_vma *vma,
> >  	if (!bo)
> >  		return 0;
> >  
> > +	/*
> > +	 * Check if BO is purged (under dma-resv lock).
> > +	 * For purged BOs:
> > +	 * - Scratch VMs: Skip validation, rebind will use scratch
> > PTEs
> > +	 * - Non-scratch VMs: FAIL the page fault (no scratch page
> > available)
> > +	 */
> > +	if (unlikely(xe_bo_is_purged(bo))) {
> > +		if (!xe_vm_has_scratch(vm))
> > +			return -EACCES;
> > +		return 0;
> > +	}
> > +
> >  	return need_vram_move ? xe_bo_migrate(bo, vram->placement,
> > NULL, exec) :
> >  		xe_bo_validate(bo, vm, true, exec);
> >  }
> > diff --git a/drivers/gpu/drm/xe/xe_pt.c
> > b/drivers/gpu/drm/xe/xe_pt.c
> > index 6703a7049227..c8c66300e25b 100644
> > --- a/drivers/gpu/drm/xe/xe_pt.c
> > +++ b/drivers/gpu/drm/xe/xe_pt.c
> > @@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> > pgoff_t offset,
> >  	/* Is this a leaf entry ?*/
> >  	if (level == 0 || xe_pt_hugepte_possible(addr, next,
> > level, xe_walk)) {
> >  		struct xe_res_cursor *curs = xe_walk->curs;
> > -		bool is_null = xe_vma_is_null(xe_walk->vma);
> > -		bool is_vram = is_null ? false :
> > xe_res_is_vram(curs);
> > +		struct xe_bo *bo = xe_vma_bo(xe_walk->vma);
> > +		bool is_null_or_purged = xe_vma_is_null(xe_walk-
> > >vma) ||
> > +					 (bo &&
> > xe_bo_is_purged(bo));
> > +		bool is_vram = is_null_or_purged ? false :
> > xe_res_is_vram(curs);
> >  
> >  		XE_WARN_ON(xe_walk->va_curs_start != addr);
> >  
> >  		if (xe_walk->clear_pt) {
> >  			pte = 0;
> >  		} else {
> > -			pte = vm->pt_ops->pte_encode_vma(is_null ?
> > 0 :
> > +			/*
> > +			 * For purged BOs, treat like null VMAs -
> > pass address 0.
> > +			 * The pte_encode_vma will set XE_PTE_NULL
> > flag for scratch mapping.
> > +			 */
> > +			pte = vm->pt_ops-
> > >pte_encode_vma(is_null_or_purged ? 0 :
> >  							
> > xe_res_dma(curs) +
> >  							 xe_walk-
> > >dma_offset,
> >  							 xe_walk-
> > >vma,
> >  							
> > pat_index, level);
> > -			if (!is_null)
> > +			if (!is_null_or_purged)
> >  				pte |= is_vram ? xe_walk-
> > >default_vram_pte :
> >  					xe_walk-
> > >default_system_pte;
> >  
> > @@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,
> > pgoff_t offset,
> >  		if (unlikely(ret))
> >  			return ret;
> >  
> > -		if (!is_null && !xe_walk->clear_pt)
> > +		if (!is_null_or_purged && !xe_walk->clear_pt)
> >  			xe_res_next(curs, next - addr);
> >  		xe_walk->va_curs_start = next;
> >  		xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K <<
> > level);
> > @@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, struct
> > xe_vma *vma,
> >  	};
> >  	struct xe_pt *pt = vm->pt_root[tile->id];
> >  	int ret;
> > +	bool is_purged = false;
> > +
> > +	/*
> > +	 * Check if BO is purged:
> > +	 * - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe
> > zero reads
> > +	 * - Non-scratch VMs: Clear PTEs to zero (non-present) to
> > avoid mapping to phys addr 0
> > +	 *
> > +	 * For non-scratch VMs, we force clear_pt=true so leaf
> > PTEs become completely
> > +	 * zero instead of creating a PRESENT mapping to physical
> > address 0.
> > +	 */
> > +	if (bo && xe_bo_is_purged(bo)) {
> > +		is_purged = true;
> > +
> > +		/*
> > +		 * For non-scratch VMs, a NULL rebind should use
> > zero PTEs
> > +		 * (non-present), not a present PTE to phys 0.
> > +		 */
> > +		if (!xe_vm_has_scratch(vm))
> > +			xe_walk.clear_pt = true;
> > +	}
> >  
> >  	if (range) {
> >  		/* Move this entire thing to xe_svm.c? */
> > @@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct
> > xe_vma *vma,
> >  	if (!range)
> >  		xe_bo_assert_held(bo);
> >  
> > -	if (!xe_vma_is_null(vma) && !range) {
> > +	if (!xe_vma_is_null(vma) && !range && !is_purged) {
> >  		if (xe_vma_is_userptr(vma))
> >  			xe_res_first_dma(to_userptr_vma(vma)-
> > >userptr.pages.dma_addr, 0,
> >  					 xe_vma_size(vma), &curs);
> > diff --git a/drivers/gpu/drm/xe/xe_vm.c
> > b/drivers/gpu/drm/xe/xe_vm.c
> > index 694f592a0f01..c3a5fe76ff96 100644
> > --- a/drivers/gpu/drm/xe/xe_vm.c
> > +++ b/drivers/gpu/drm/xe/xe_vm.c
> > @@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo
> > *bo, u64 bo_offset,
> >  static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma,
> >  			       u16 pat_index, u32 pt_level)
> >  {
> > +	struct xe_bo *bo = xe_vma_bo(vma);
> > +	struct xe_vm *vm = xe_vma_vm(vma);
> > +
> >  	pte |= XE_PAGE_PRESENT;
> >  
> >  	if (likely(!xe_vma_read_only(vma)))
> > @@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte,
> > struct xe_vma *vma,
> >  	pte |= pte_encode_pat_index(pat_index, pt_level);
> >  	pte |= pte_encode_ps(pt_level);
> >  
> > -	if (unlikely(xe_vma_is_null(vma)))
> > +	/*
> > +	 * NULL PTEs redirect to scratch page (return zeros on
> > read).
> > +	 * Set for: 1) explicit null VMAs, 2) purged BOs on
> > scratch VMs.
> > +	 * Never set NULL flag without scratch page - causes
> > undefined behavior.
> > +	 */
> > +	if (unlikely(xe_vma_is_null(vma) ||
> > +		     (bo && xe_bo_is_purged(bo) &&
> > xe_vm_has_scratch(vm))))
> >  		pte |= XE_PTE_NULL;
> >  
> >  	return pte;
> > diff --git a/drivers/gpu/drm/xe/xe_vm_madvise.c
> > b/drivers/gpu/drm/xe/xe_vm_madvise.c
> > index add9a6ca2390..dfeab9e24a09 100644
> > --- a/drivers/gpu/drm/xe/xe_vm_madvise.c
> > +++ b/drivers/gpu/drm/xe/xe_vm_madvise.c
> > @@ -179,6 +179,56 @@ static void madvise_pat_index(struct xe_device
> > *xe, struct xe_vm *vm,
> >  	}
> >  }
> >  
> > +/*:
> > + * Handle purgeable buffer object advice for
> > DONTNEED/WILLNEED/PURGED.
> > + * Returns true if any BO was purged, false otherwise.
> > + * Caller must copy retained value to userspace after releasing
> > locks.
> > + */
> > +static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe,
> > struct xe_vm *vm,
> > +				       struct xe_vma **vmas, int
> > num_vmas,
> > +				       struct drm_xe_madvise *op)
> 
> Shouldn't this check be a vfunc in madvise_funcs?
> 
> Also I think you can hook into xe_madvise_details for the return
> value /
> final copy to user.
> 
> > +{
> > +	bool has_purged_bo = false;
> > +	int i;
> > +
> > +	xe_assert(vm->xe, op->type ==
> > DRM_XE_VMA_ATTR_PURGEABLE_STATE);
> > +
> > +	for (i = 0; i < num_vmas; i++) {
> > +		struct xe_bo *bo = xe_vma_bo(vmas[i]);
> > +
> > +		if (!bo)
> > +			continue;
> > +
> > +		/* BO must be locked before modifying madv state
> > */
> > +		xe_bo_assert_held(bo);
> > +
> > +		/*
> > +		 * Once purged, always purged. Cannot transition
> > back to WILLNEED.
> > +		 * This matches i915 semantics where purged BOs
> > are permanently invalid.
> > +		 */
> > +		if (xe_bo_is_purged(bo)) {
> > +			has_purged_bo = true;
> > +			continue;
> > +		}
> > +
> > +		switch (op->purge_state_val.val) {
> > +		case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED:
> > +			bo->madv_purgeable =
> > XE_MADV_PURGEABLE_WILLNEED;
> > +			break;
> > +		case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED:
> > +			bo->madv_purgeable =
> > XE_MADV_PURGEABLE_DONTNEED;
> 
> Use above suggested helper to set this state?
> 
> > +			break;
> > +		default:
> > +			drm_warn(&vm->xe->drm, "Invalid madvice
> > value = %d\n",
> > +				 op->purge_state_val.val);
> > +			return false;
> > +		}
> > +	}
> > +
> > +	/* Return whether any BO was purged; caller will copy to
> > user after unlocking */
> > +	return has_purged_bo;
> > +}
> > +
> >  typedef void (*madvise_func)(struct xe_device *xe, struct xe_vm
> > *vm,
> >  			     struct xe_vma **vmas, int num_vmas,
> >  			     struct drm_xe_madvise *op,
> > @@ -306,6 +356,16 @@ static bool madvise_args_are_sane(struct
> > xe_device *xe, const struct drm_xe_madv
> >  			return false;
> >  		break;
> >  	}
> > +	case DRM_XE_VMA_ATTR_PURGEABLE_STATE:
> > +	{
> > +		u32 val = args->purge_state_val.val;
> > +
> > +		if (XE_IOCTL_DBG(xe, !(val ==
> > DRM_XE_VMA_PURGEABLE_STATE_WILLNEED ||
> > +				       val ==
> > DRM_XE_VMA_PURGEABLE_STATE_DONTNEED)))
> > +			return false;
> > +
> > +		break;
> > +	}
> >  	default:
> >  		if (XE_IOCTL_DBG(xe, 1))
> >  			return false;
> > @@ -465,6 +525,34 @@ int xe_vm_madvise_ioctl(struct drm_device
> > *dev, void *data, struct drm_file *fil
> >  					goto err_fini;
> >  			}
> >  		}
> > +		if (args->type == DRM_XE_VMA_ATTR_PURGEABLE_STATE)
> > {
> > +			bool has_purged_bo;
> > +
> > +			has_purged_bo =
> > xe_vm_madvise_purgeable_bo(xe, vm, madvise_range.vmas,
> > +								  
> > madvise_range.num_vmas, args);
> > +
> 
> Again use the existing vfuncs here.
> 
> > +			/* Release BO locks */
> > +			drm_exec_fini(&exec);
> > +			kfree(madvise_range.vmas);
> > +			up_write(&vm->lock);
> > +
> > +			/*
> > +			 * Set retained flag to indicate if
> > backing store still exists.
> > +			 * Matches i915: retained = 1 if not
> > purged, 0 if purged.
> > +			 * Must copy_to_user AFTER releasing ALL
> > locks to avoid circular dependency.
> > +			 */
> > +			if (args->purge_state_val.retained) {
> > +				u32 retained = !has_purged_bo;
> > +
> > +				if
> > (copy_to_user(u64_to_user_ptr(args->purge_state_val.retained),
> > +						 &retained,
> > sizeof(retained)))
> 
> I don't think remained needs to be a u64 - maybe a u16? Will comment
> on
> uAPI too.
> 
> > +					drm_warn(&vm->xe->drm,
> > "Failed to copy retained value to user\n");
> 
> See above, use xe_madvise_details_fini for the final copy to user.

Can we use put_user() rather than copy_to_user() ?

Also, should the IOCTL return a failure in this case?

Another option is ofc to assert that retained is set to false on IOCTL
call, so that if put_user() fails, UMD will not try to reuse a bo whose
retained state is unclear.


Thanks,
Thomas


> 
> Matt
> 
> > +			}
> > +
> > +			/* Final cleanup for early return */
> > +			xe_vm_put(vm);
> > +			return 0;
> > +		}
> >  	}
> >  
> >  	if (madvise_range.has_svm_userptr_vmas) {
> > -- 
> > 2.43.0
> >