Re: [PATCH] drm/xe: Fix unnecessary host-side population of ttm_tt on non-TT resources

[Matthew Brost <matthew.brost@intel.com>]

On Wed, Aug 19, 2026 at 11:19:01PM -0400, Nathan Bourgeois wrote:

Thanks the patch.

> When a buffer object (BO) has a created but unpopulated ttm_tt and is
> exported, the default behavior populates the ttm_tt, even if the manager
> does not require a TT. This unnecessary host-side population reduces
> host memory available to the user.
> 
> This occurs when a BO is created with XE_BO_FLAG_DEFER_BACKING and
> migration places the BO into a resource whose manager does not use the
> TT backing. The retained ttm_tt is not authoritative for storage, yet
> the default ttm_bo_setup_export() treats the existence of the ttm_tt as
> requiring population.
> 

Thanks for explaining this, I see the problem.

> The issue was reproduced with vLLM 0.27.1 loading
> 0xSer0/DeepSeek-V4-Flash-180B (d3c704b) on a system with 128 GB of RAM
> and six Intel Arc Pro B70 GPUs providing 192 GB of VRAM. Loading the
> weights (100.61 GB) triggered host-memory exhaustion and OOM events on
> the baseline kernel.
> 
> On Ubuntu 24.04 with 7.0-12-generic, the vLLM service cgroup grew by
> 115.84 GB on the instrumented baseline and by 15.34 GB with this change,
> a reduction of 100.50 GB. Diagnostic instrumentation counting cumulative
> TT pages across all six GPUs recorded 109.22 GB on baseline and 8.72 GB
> with this change. The reduction is 0.11 GB less than the model weight
> size.
> 
> The change creates an Xe-local export setup helper which skips
> ttm_bo_populate() when the BO has a current resource manager and if that
> manager has use_tt == false. If the manager does not use TT, then
> population is skipped, while an absent manager conservatively calls
> ttm_bo_populate(). Behavior for a manager which requires a TT backing
> remains unchanged. Keep this policy local to Xe so other TTM drivers
> retain their existing export behavior.
> 

Reading the commit which added ttm_bo_setup_export, to me this looks
like a bug in ttm_bo_setup_export() or in general TTM.

git format-patch -1 50243079865ae

'This only applies currently to TTM_PL_SYSTEM objects, because
GTT objects get populated on first validate, and VRAM doesn't
use TT.'

I'd probably move this fix into ttm_bo_setup_export() and post it for
discussion.

It may also be the case that the ttm_tt preallocated with
XE_BO_FLAG_DEFER_BACKING should be dropped once the BO moves to VRAM, as
that dangling ttm_tt could cause other issues. I'll need to take a
closer look at that angle.

> The regression test creates both the control case and the VRAM migration
> case. The control case creates the deferred BO in system RAM and leaves
> it there, which requires a TT and population when xe_gem_prime_export() is
> called. The VRAM migration case creates the deferred BO in system RAM
> and then migrates it to VRAM, which does not require a TT and thus, when
> xe_gem_prime_export() is called, no population occurs.
>

This part looks good as different patch from what I'm assuming will be a
TTM fix.

Matt

> Fixes: 91494dee1091 ("xe: populate buffers before exporting them.")
> Cc: stable@vger.kernel.org
> Assisted-by: Codex:gpt-5.6-sol
> Signed-off-by: Nathan Bourgeois <iridescentrosesfall@gmail.com>
> ---
> Newer Testing:
> - Built and booted b4f95affc66ef76342c1f6bf3849f6c8ade6b9d6 plus this patch on EPYC 7352 with six Intel Arc Pro B70.
> - KUnit xe_live_test: xe_dma_buf_kunit: pass:6 fail:0 skip:0 total:6 PASS
> - vLLM 0.27.1 loading DeepSeek-V4-Flash-0731 (full model compared to REAP) on six Intel Arc Pro B70 GPUs: No OOM error, 17.82 GB delta.
>  drivers/gpu/drm/xe/tests/xe_dma_buf.c | 105 ++++++++++++++++++++++++++
>  drivers/gpu/drm/xe/xe_dma_buf.c       |  28 ++++++-
>  2 files changed, 132 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/xe/tests/xe_dma_buf.c b/drivers/gpu/drm/xe/tests/xe_dma_buf.c
> index 0be8440b3976..bff803ab7a57 100644
> --- a/drivers/gpu/drm/xe/tests/xe_dma_buf.c
> +++ b/drivers/gpu/drm/xe/tests/xe_dma_buf.c
> @@ -260,12 +260,117 @@ static const struct dma_buf_test_params test_params[] = {
>  	{}
>  };
>  
> +static void xe_test_dmabuf_export_deferred(struct xe_device *xe, u32 bo_flags,
> +					   u32 mem_type, bool expect_populated)
> +{
> +	struct drm_exec *exec = XE_VALIDATION_OPT_OUT;
> +	struct kunit *test = kunit_get_current_test();
> +	struct ttm_resource_manager *man;
> +	struct dma_buf *dmabuf;
> +	struct xe_bo *bo;
> +	size_t size = PAGE_SIZE;
> +	int err;
> +
> +	/* No VRAM on device? */
> +	if (!ttm_manager_type(&xe->ttm, mem_type))
> +		return;
> +
> +	if (mem_type == XE_PL_VRAM0 &&
> +	    xe->info.vram_flags & XE_VRAM_FLAGS_NEED64K)
> +		size = SZ_64K;
> +
> +	/*
> +	 * DEFER_BACKING places the BO in SYSTEM and creates a ttm_tt which
> +	 * is unpopulated. In the case of VRAM, migrating leaves the ttm_tt
> +	 * retained and unpopulated while VRAM becomes the real backing.
> +	 */
> +	bo = xe_bo_create_user(xe, NULL, size, DRM_XE_GEM_CPU_CACHING_WC,
> +			       bo_flags | XE_BO_FLAG_DEFER_BACKING, NULL);
> +	if (IS_ERR(bo)) {
> +		KUNIT_FAIL(test, "BO creation failed: %pe\n", bo);
> +		return;
> +	}
> +
> +	err = xe_bo_lock(bo, false);
> +	if (err) {
> +		KUNIT_FAIL(test, "BO lock failed: %d\n", err);
> +		goto out_put_bo;
> +	}
> +
> +	if (bo->ttm.resource->mem_type != mem_type)
> +		err = xe_bo_migrate(bo, mem_type, NULL, exec);
> +	if (err) {
> +		KUNIT_FAIL(test, "BO migration to %u failed: %d\n", mem_type,
> +			   err);
> +		goto out_unlock;
> +	}
> +
> +	man = ttm_manager_type(bo->ttm.bdev, bo->ttm.resource->mem_type);
> +	if (!man || !bo->ttm.ttm) {
> +		KUNIT_FAIL(test, "Expected a retained unpopulated TT\n");
> +		goto out_unlock;
> +	}
> +
> +	/* Precondition: ttm_tt starts unpopulated after migration */
> +	KUNIT_EXPECT_EQ(test, man->use_tt, expect_populated);
> +	KUNIT_EXPECT_FALSE(test, ttm_tt_is_populated(bo->ttm.ttm));
> +
> +	xe_bo_unlock(bo);
> +
> +	dmabuf = xe_gem_prime_export(&bo->ttm.base, 0);
> +	if (IS_ERR(dmabuf)) {
> +		KUNIT_FAIL(test, "dma-buf export failed: %pe\n", dmabuf);
> +		goto out_put_bo;
> +	}
> +
> +	err = xe_bo_lock(bo, false);
> +	if (err) {
> +		KUNIT_FAIL(test, "post-export BO lock failed: %d\n", err);
> +		goto out_put_dmabuf;
> +	}
> +
> +	/* Postcondition: if VRAM, ttm_tt remains unpopulated, if SYSTEM ttm_tt is populated */
> +	KUNIT_EXPECT_EQ(test, bo->ttm.resource->mem_type, mem_type);
> +	KUNIT_EXPECT_NOT_NULL(test, bo->ttm.ttm);
> +	if (bo->ttm.ttm)
> +		KUNIT_EXPECT_EQ(test, ttm_tt_is_populated(bo->ttm.ttm),
> +				expect_populated);
> +
> +	xe_bo_unlock(bo);
> +	dma_buf_put(dmabuf);
> +	drm_gem_object_put(&bo->ttm.base);
> +	return;
> +
> +out_unlock:
> +	xe_bo_unlock(bo);
> +	goto out_put_bo;
> +out_put_dmabuf:
> +	dma_buf_put(dmabuf);
> +out_put_bo:
> +	drm_gem_object_put(&bo->ttm.base);
> +}
> +
>  static int dma_buf_run_device(struct xe_device *xe)
>  {
>  	const struct dma_buf_test_params *params;
>  	struct kunit *test = kunit_get_current_test();
>  
>  	guard(xe_pm_runtime)(xe);
> +
> +	/*
> +	 * A retained TT must not be populated when VRAM is the backing
> +	 * resource.
> +	 */
> +	xe_test_dmabuf_export_deferred(xe, XE_BO_FLAG_VRAM0, XE_PL_VRAM0,
> +				       false);
> +
> +	/*
> +	 * Control case: deferred SYSTEM backing must still be populated
> +	 * before export.
> +	 */
> +	xe_test_dmabuf_export_deferred(xe, XE_BO_FLAG_SYSTEM, XE_PL_SYSTEM,
> +				       true);
> +
>  	for (params = test_params; params->mem_mask; ++params) {
>  		struct dma_buf_test_params p = *params;
>  
> diff --git a/drivers/gpu/drm/xe/xe_dma_buf.c b/drivers/gpu/drm/xe/xe_dma_buf.c
> index bf0728838ead..0c4e4e2e1a81 100644
> --- a/drivers/gpu/drm/xe/xe_dma_buf.c
> +++ b/drivers/gpu/drm/xe/xe_dma_buf.c
> @@ -219,6 +219,32 @@ static const struct dma_buf_ops xe_dmabuf_ops = {
>  	.vunmap = drm_gem_dmabuf_vunmap,
>  };
>  
> +static int xe_dma_bo_setup_export(struct ttm_buffer_object *tbo,
> +				  struct ttm_operation_ctx *ctx)
> +{
> +	struct ttm_resource_manager *man = NULL;
> +	int ret;
> +
> +	ret = ttm_bo_reserve(tbo, false, false, NULL);
> +	if (ret)
> +		return ret;
> +
> +	if (tbo->resource)
> +		man = ttm_manager_type(tbo->bdev, tbo->resource->mem_type);
> +
> +	/*
> +	 * Do not populate BO-sized system pages when backed by a non-TT resource.
> +	 * This is Xe-specific; the generic ttm_bo_setup_export() always populates.
> +	 */
> +	if (man && !man->use_tt)
> +		ret = 0;
> +	else
> +		ret = ttm_bo_populate(tbo, ctx);
> +
> +	ttm_bo_unreserve(tbo);
> +	return ret;
> +}
> +
>  struct dma_buf *xe_gem_prime_export(struct drm_gem_object *obj, int flags)
>  {
>  	struct xe_bo *bo = gem_to_xe_bo(obj);
> @@ -257,7 +283,7 @@ struct dma_buf *xe_gem_prime_export(struct drm_gem_object *obj, int flags)
>  	xe_bo_willneed_get_locked(bo);
>  	xe_bo_unlock(bo);
>  
> -	ret = ttm_bo_setup_export(&bo->ttm, &ctx);
> +	ret = xe_dma_bo_setup_export(&bo->ttm, &ctx);
>  	if (ret)
>  		goto out_put;
>  
> 
> base-commit: b4f95affc66ef76342c1f6bf3849f6c8ade6b9d6
> -- 
> 2.55.0
>