Re: [PATCH v3 12/33] drm/xe: Add debugfs stats for DMA-mapped pages per order

[Matthew Brost <matthew.brost@intel.com>]

On Tue, Nov 03, 2026 at 08:06:28PM -0800, Niranjana Vishwanathapura wrote:
> On Fri, Jul 10, 2026 at 07:55:58PM -0700, Matthew Brost wrote:
> > From: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > 
> > Expose per-page-order DMA mapping counts for the system memory that the
> > xe driver maps for GPU access, split into two categories: TTM buffer
> > objects, and the GPU SVM / userptr ranges (which share a single
> > drm_gpusvm instance per VM).
> > 
> > The stats are visible at:
> > 
> >  <debugfs>/dri/<N>/dma_mapped_pages
> > 
> > and are broken out into two rows (bo, svm/userptr), one column per page
> > order, mirroring the layout of the TTM pool page_pool stat.
> > 
> > For TTM BOs the per-order counts are derived from the DMA scatter-gather
> > table: each mapped segment is physically contiguous, so its page count is
> > split into the largest power-of-two runs (a segment of N pages contributes
> > a 2^k chunk for the largest k with 2^k <= N, repeated for the remainder).
> > Driving the accounting off the sg table rather than the pages[] array keeps
> > it correct even when pages[] entries are cleared while still DMA-mapped,
> > as happens on the defrag page-borrowing path.
> > 
> > For the SVM and userptr ranges the accounting is driven by the
> > drm_gpusvm @dma_map_account callback, which fires once per dma_addr[]
> > entry at the exact point the entry is DMA-mapped and unmapped. This
> > makes the accounting symmetric by construction and avoids driver-side
> > walks that could drift across migration, partial unmaps and the iova vs
> > non-iova paths. Only DRM_INTERCONNECT_SYSTEM entries are real DMA maps
> > and counted; device interconnect (VRAM, P2P) entries are skipped, so the
> > counter reflects only system pages actually mapped through the DMA layer.
> > 
> > The callback is shared between the full SVM instance (fault mode) and the
> > core drm_gpusvm_pages-only instance used for userptr, so userptr
> > mappings are accounted in both the CONFIG_DRM_XE_GPUSVM=y and =n (but
> > CONFIG_DRM_GPUSVM=y) configurations, into the same svm/userptr counter.
> > 
> > Since the counts are only ever consumed through debugfs, gate the counter
> > storage and all accounting on CONFIG_DEBUG_FS so that builds without
> > debugfs carry no extra atomic traffic or device state; the drm_gpusvm
> > callback is simply left unregistered in that case.
> > 
> > On device teardown assert that every counter has returned to zero, to
> > catch any unbalanced accounting (a leaked DMA mapping) during testing.
> > 
> > Assisted-by: GitHub_Copilot:claude-opus-4.8
> > Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> > 
> 
> Patches 11 and 12 can be spinned out as a separate patch series
> (for fast tracking).
> 

Yep, spliting these out now.

> Also, dma map/unmap of dmabuf imported BOs are skipped here (rightly so).
> But perhaps add a comments in commit message that either self-import
> returns the same BO and BOs of other device gets counted on exporter
> side when it gets first migrated to TT first.
>

Sure.

> > ---
> > 
> > v3:
> > - Fix drm_gpusvm_init() argument order, clamp DMA-page order to
> >   NR_PAGE_ORDERS - 1, and drain DMA-page accounting after
> >   xe_bo_dev_fini() (Sashiko)
> > ---
> > drivers/gpu/drm/xe/xe_bo.c           | 46 ++++++++++++++++++++
> > drivers/gpu/drm/xe/xe_debugfs.c      | 26 ++++++++++++
> > drivers/gpu/drm/xe/xe_device.c       | 30 +++++++++++++
> > drivers/gpu/drm/xe/xe_device_types.h | 17 ++++++++
> > drivers/gpu/drm/xe/xe_svm.c          | 44 ++++++++++++++++++-
> > drivers/gpu/drm/xe/xe_svm.h          |  4 +-
> > drivers/gpu/drm/xe/xe_userptr.c      | 63 ++++++++++++++++++++++++++++
> > drivers/gpu/drm/xe/xe_userptr.h      |  1 +
> > 8 files changed, 228 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c
> > index 85e6d9a0f575..60e0e568aa31 100644
> > --- a/drivers/gpu/drm/xe/xe_bo.c
> > +++ b/drivers/gpu/drm/xe/xe_bo.c
> > @@ -367,6 +367,50 @@ struct xe_ttm_tt {
> > 	bool purgeable;
> > };
> > 
> > +/*
> > + * xe_tt_account_dma_pages - account DMA-mapped tt pages per order
> > + * @xe: the xe device
> > + * @xe_tt: the xe_ttm_tt whose sg table to walk
> > + * @sign: +1 to add, -1 to subtract
> > + *
> > + * Walk the tt's DMA scatter-gather table and split each (physically
> > + * contiguous) segment into the largest power-of-two page runs: a segment of N
> > + * pages contributes a 2^k chunk for the largest k with 2^k <= N, repeated for
> > + * the remainder. Accumulate the count of 2^k pages into
> > + * xe->mem.dma_mapped_pages[k]. Driving this off the sg table rather than
> > + * tt->pages keeps the accounting correct even when pages[] entries are cleared
> > + * while still DMA-mapped (e.g. the defrag page-borrowing path).
> > + */
> > +#if IS_ENABLED(CONFIG_DEBUG_FS)
> > +static void xe_tt_account_dma_pages(struct xe_device *xe,
> > +				    struct xe_ttm_tt *xe_tt, int sign)
> > +{
> > +	struct scatterlist *sg;
> > +	unsigned int i;
> > +
> > +	for_each_sgtable_sg(xe_tt->sg, sg, i) {
> > +		unsigned long nr_pages = sg->length >> PAGE_SHIFT;
> > +
> > +		while (nr_pages) {
> > +			unsigned int order = min_t(unsigned int,
> > +						   __fls(nr_pages),
> > +						   MAX_PAGE_ORDER);
> > +			unsigned long chunk = 1UL << order;
> > +
> > +			atomic_long_add((long)chunk * sign,
> > +					&xe->mem.dma_mapped_pages[order]);
> > +
> 
> NIT: As there is an atomic operation involved, instead of doing it under
> CONFIG_DEBUG_FS, should we be doing it under a separate xe specific
> debug config?
> 

I think GT stats are just gated by CONFIG_DEBUG_FS, I say this is
roughly idea. If someone complains about perf we could change this
per-cpu counter like GT stats or I guess add different Kconfig gate?

> > +			nr_pages -= chunk;
> > +		}
> > +	}
> > +}
> > +#else
> > +static void xe_tt_account_dma_pages(struct xe_device *xe,
> > +				    struct xe_ttm_tt *xe_tt, int sign)
> > +{
> > +}
> > +#endif
> > +
> > static int xe_tt_map_sg(struct xe_device *xe, struct ttm_tt *tt)
> > {
> > 	struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
> > @@ -396,6 +440,7 @@ static int xe_tt_map_sg(struct xe_device *xe, struct ttm_tt *tt)
> > 		return ret;
> > 	}
> > 
> > +	xe_tt_account_dma_pages(xe, xe_tt, 1);
> > 	return 0;
> > }
> > 
> > @@ -404,6 +449,7 @@ static void xe_tt_unmap_sg(struct xe_device *xe, struct ttm_tt *tt)
> > 	struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
> > 
> > 	if (xe_tt->sg) {
> > +		xe_tt_account_dma_pages(xe, xe_tt, -1);
> > 		dma_unmap_sgtable(xe->drm.dev, xe_tt->sg,
> > 				  DMA_BIDIRECTIONAL, 0);
> > 		sg_free_table(xe_tt->sg);
> 
> There is another dma_unamp_sgtable() in xe_bo_dma_unmap_pinned().
> I guess we should account for that unmap path also.
> I am seeing sashiko reported this as well.

I see dma_buf_unmap_attachment and call to xe_tt_unmap_sg but I'm on
drm-tip now, so at in what I'll post shortly this I don't think this is
an issue.

> 
> > diff --git a/drivers/gpu/drm/xe/xe_debugfs.c b/drivers/gpu/drm/xe/xe_debugfs.c
> > index 8c391c7b017a..97b47c5c4213 100644
> > --- a/drivers/gpu/drm/xe/xe_debugfs.c
> > +++ b/drivers/gpu/drm/xe/xe_debugfs.c
> > @@ -6,6 +6,7 @@
> > #include "xe_debugfs.h"
> > 
> > #include <linux/debugfs.h>
> > +#include <linux/mmzone.h>
> > #include <linux/fault-inject.h>
> > #include <linux/string_helpers.h>
> > 
> > @@ -231,6 +232,28 @@ static int dgfx_pcie_link_residencies_show(struct seq_file *m, void *data)
> > 	return 0;
> > }
> > 
> > +static int dma_mapped_pages_show(struct seq_file *m, void *data)
> > +{
> > +	struct xe_device *xe = m->private;
> > +	unsigned int i;
> > +
> > +	seq_printf(m, "%-11s ", "");
> > +	for (i = 0; i < NR_PAGE_ORDERS; i++)
> > +		seq_printf(m, " ---%2u---", i);
> > +	seq_printf(m, "\n%-11s:", "bo");
> > +	for (i = 0; i < NR_PAGE_ORDERS; i++)
> > +		seq_printf(m, " %8lu",
> > +			   atomic_long_read(&xe->mem.dma_mapped_pages[i]));
> > +	seq_printf(m, "\n%-11s:", "svm/userptr");
> > +	for (i = 0; i < NR_PAGE_ORDERS; i++)
> > +		seq_printf(m, " %8lu",
> > +			   atomic_long_read(&xe->mem.dma_mapped_pages_svm[i]));
> > +	seq_puts(m, "\n");
> > +
> 
> NIT: As order-10 also prints higher order (1GB) pages, may be print it as '>=10'?
> 
> NIT: This is a rough snapshot as array elements could be changing as we print.
> I guess that is fine.
> 
> > +	return 0;
> > +}
> > +DEFINE_SHOW_ATTRIBUTE(dma_mapped_pages);
> > +
> > static const struct drm_info_list debugfs_list[] = {
> > 	{"info", info, 0},
> > 	{ .name = "sriov_info", .show = sriov_info, },
> > @@ -632,6 +655,9 @@ void xe_debugfs_register(struct xe_device *xe)
> > 	if (man)
> > 		ttm_resource_manager_create_debugfs(man, root, "stolen_mm");
> > 
> > +	debugfs_create_file("dma_mapped_pages", 0444, root, xe,
> > +			    &dma_mapped_pages_fops);
> > +
> > 	for_each_tile(tile, xe, tile_id)
> > 		xe_tile_debugfs_register(tile);
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> > index ad7f3e61d457..97cc2d185e45 100644
> > --- a/drivers/gpu/drm/xe/xe_device.c
> > +++ b/drivers/gpu/drm/xe/xe_device.c
> > @@ -455,12 +455,42 @@ bool xe_device_is_admin_only(const struct xe_device *xe)
> > }
> > #endif
> > 
> > +#if IS_ENABLED(CONFIG_DEBUG_FS)
> > +static void xe_device_assert_dma_pages_zero(struct xe_device *xe)
> > +{
> > +	unsigned int i;
> > +
> > +	/*
> > +	 * All BOs, userptr VMAs and SVM ranges must have been torn down by the
> > +	 * time the device is destroyed, so every DMA-mapped-pages counter must
> > +	 * have returned to zero. A non-zero value indicates unbalanced
> > +	 * accounting, i.e. a missing unmap-side decrement.
> > +	 */
> > +	for (i = 0; i < NR_PAGE_ORDERS; i++) {
> > +		xe_assert(xe, !atomic_long_read(&xe->mem.dma_mapped_pages[i]));
> > +		xe_assert(xe, !atomic_long_read(&xe->mem.dma_mapped_pages_svm[i]));
> 
> This also probably good to do it under a separate xe DEBUG config instead of
> CONFIG_DEBUG_FS.
> 

I can make this change but IMO this is needed. I would think we'd want
these stats on all production systems for debug, i.e., CONFIG_DEBUG_FS
is likely the right Kconfig.

Matt

> Niranjana
> 
> > +	}
> > +}
> > +#else
> > +static void xe_device_assert_dma_pages_zero(struct xe_device *xe)
> > +{
> > +}
> > +#endif
> > +
> > static void xe_device_destroy(struct drm_device *dev, void *dummy)
> > {
> > 	struct xe_device *xe = to_xe_device(dev);
> > 
> > 	xe_bo_dev_fini(&xe->bo_device);
> > 
> > +	/*
> > +	 * Assert the DMA-mapped-pages accounting only after xe_bo_dev_fini()
> > +	 * has flushed the async BO-free worker: BOs pending async free still
> > +	 * hold their DMA mappings (and counts) until the worker runs, so
> > +	 * asserting before the flush could trip a false positive.
> > +	 */
> > +	xe_device_assert_dma_pages_zero(xe);
> > +
> > 	if (xe->preempt_fence_wq)
> > 		destroy_workqueue(xe->preempt_fence_wq);
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> > index 022e08205897..95e7cea91cbf 100644
> > --- a/drivers/gpu/drm/xe/xe_device_types.h
> > +++ b/drivers/gpu/drm/xe/xe_device_types.h
> > @@ -6,6 +6,7 @@
> > #ifndef _XE_DEVICE_TYPES_H_
> > #define _XE_DEVICE_TYPES_H_
> > 
> > +#include <linux/mmzone.h>
> > #include <linux/pci.h>
> > 
> > #include <drm/drm_device.h>
> > @@ -279,6 +280,22 @@ struct xe_device {
> > 		struct xe_shrinker *shrinker;
> > 		/** @mem.stolen_mgr: stolen memory manager. */
> > 		struct xe_ttm_stolen_mgr *stolen_mgr;
> > +#if IS_ENABLED(CONFIG_DEBUG_FS)
> > +		/**
> > +		 * @mem.dma_mapped_pages: number of DMA-mapped pages per page
> > +		 * order currently live for this device, for TTM BOs. Only
> > +		 * accounted when CONFIG_DEBUG_FS is enabled, since it is solely
> > +		 * exposed through debugfs.
> > +		 */
> > +		atomic_long_t dma_mapped_pages[NR_PAGE_ORDERS];
> > +		/**
> > +		 * @mem.dma_mapped_pages_svm: number of DMA-mapped pages per
> > +		 * page order currently live for this device, for SVM ranges
> > +		 * and userptr VMAs (both share a single drm_gpusvm instance).
> > +		 * Only accounted when CONFIG_DEBUG_FS is enabled.
> > +		 */
> > +		atomic_long_t dma_mapped_pages_svm[NR_PAGE_ORDERS];
> > +#endif
> > 	} mem;
> > 
> > 	/** @sriov: device level virtualization data */
> > diff --git a/drivers/gpu/drm/xe/xe_svm.c b/drivers/gpu/drm/xe/xe_svm.c
> > index b228a737cfd6..ebe48cb76ad9 100644
> > --- a/drivers/gpu/drm/xe/xe_svm.c
> > +++ b/drivers/gpu/drm/xe/xe_svm.c
> > @@ -827,10 +827,50 @@ static int xe_svm_get_pagemaps(struct xe_vm *vm)
> > }
> > #endif
> > 
> > +#if IS_ENABLED(CONFIG_DEBUG_FS)
> > +/*
> > + * xe_svm_dma_map_account - drm_gpusvm DMA-mapping accounting callback
> > + * @gpusvm: The GPU SVM the mapping belongs to
> > + * @addr: The address descriptor of the chunk being (un)mapped
> > + * @sign: +1 when @addr was DMA-mapped, -1 when it is being unmapped
> > + *
> > + * Maintain per-order counts of the system-memory pages that the full SVM
> > + * instance (SVM ranges, and userptr VMAs in fault mode) has DMA-mapped for GPU
> > + * access. Only DRM_INTERCONNECT_SYSTEM entries are real DMA maps and counted;
> > + * device interconnect (VRAM, P2P) entries are skipped. The counts are solely
> > + * exposed through debugfs, so the callback is only registered when
> > + * CONFIG_DEBUG_FS is enabled.
> > + */
> > +static void xe_svm_dma_map_account(struct drm_gpusvm *gpusvm,
> > +				   const struct drm_pagemap_addr *addr,
> > +				   int sign)
> > +{
> > +	struct xe_device *xe = gpusvm_to_vm(gpusvm)->xe;
> > +	unsigned int order = addr->order;
> > +
> > +	if (addr->proto != DRM_INTERCONNECT_SYSTEM)
> > +		return;
> > +
> > +	/*
> > +	 * The CPU-PTE map order reported by hmm can exceed MAX_PAGE_ORDER
> > +	 * (e.g. an order-18 1GB mapping), so clamp the array index to avoid an
> > +	 * out-of-bounds access. Clamping is symmetric across the +1/-1 calls,
> > +	 * so the per-order counters stay balanced.
> > +	 */
> > +	order = min(order, (unsigned int)NR_PAGE_ORDERS - 1);
> > +
> > +	atomic_long_add((long)(1UL << addr->order) * sign,
> > +			&xe->mem.dma_mapped_pages_svm[order]);
> > +}
> > +#endif
> > +
> > static const struct drm_gpusvm_ops gpusvm_ops = {
> > 	.range_alloc = xe_svm_range_alloc,
> > 	.range_free = xe_svm_range_free,
> > 	.invalidate = xe_svm_invalidate,
> > +#if IS_ENABLED(CONFIG_DEBUG_FS)
> > +	.dma_map_account = xe_svm_dma_map_account,
> > +#endif
> > };
> > 
> > static const unsigned long fault_chunk_sizes[] = {
> > @@ -925,8 +965,8 @@ int xe_svm_init(struct xe_vm *vm)
> > 		}
> > 	} else {
> > 		err = drm_gpusvm_init(&vm->svm.gpusvm, "Xe SVM (simple)",
> > -				      NULL, 0, 0, 0, NULL,
> > -				      NULL, 0);
> > +				      NULL, 0, 0, 0,
> > +				      xe_userptr_gpusvm_ops_get(), NULL, 0);
> > 	}
> > 
> > 	return err;
> > diff --git a/drivers/gpu/drm/xe/xe_svm.h b/drivers/gpu/drm/xe/xe_svm.h
> > index a921556d3466..8e347137a7a6 100644
> > --- a/drivers/gpu/drm/xe/xe_svm.h
> > +++ b/drivers/gpu/drm/xe/xe_svm.h
> > @@ -194,6 +194,7 @@ struct drm_pagemap *xe_drm_pagemap_from_fd(int fd, u32 region_instance);
> > 
> > #else
> > #include <linux/interval_tree.h>
> > +#include "xe_userptr.h"
> > #include "xe_vm.h"
> > 
> > struct drm_pagemap_addr;
> > @@ -236,7 +237,8 @@ int xe_svm_init(struct xe_vm *vm)
> > {
> > #if IS_ENABLED(CONFIG_DRM_GPUSVM)
> > 	return drm_gpusvm_init(&vm->svm.gpusvm, "Xe SVM (simple)",
> > -			       NULL, 0, 0, 0, NULL, NULL, 0);
> > +			       NULL, 0, 0, 0,
> > +			       xe_userptr_gpusvm_ops_get(), NULL, 0);
> > #else
> > 	return 0;
> > #endif
> > diff --git a/drivers/gpu/drm/xe/xe_userptr.c b/drivers/gpu/drm/xe/xe_userptr.c
> > index 8b2d461ea0b2..6b9e1530ead4 100644
> > --- a/drivers/gpu/drm/xe/xe_userptr.c
> > +++ b/drivers/gpu/drm/xe/xe_userptr.c
> > @@ -8,9 +8,72 @@
> > 
> > #include <linux/mm.h>
> > 
> > +#include <drm/drm_pagemap.h>
> > +
> > #include "xe_tlb_inval.h"
> > #include "xe_trace_bo.h"
> > 
> > +#if IS_ENABLED(CONFIG_DEBUG_FS)
> > +/*
> > + * xe_userptr_dma_map_account - drm_gpusvm DMA-mapping accounting callback
> > + * @gpusvm: The GPU SVM the mapping belongs to
> > + * @addr: The address descriptor of the chunk being (un)mapped
> > + * @sign: +1 when @addr was DMA-mapped, -1 when it is being unmapped
> > + *
> > + * Account, per page order, the system-memory pages DMA-mapped for GPU access
> > + * through the drm_gpusvm_pages instance shared by userptr (and, in fault mode,
> > + * SVM). Only DRM_INTERCONNECT_SYSTEM entries are real DMA maps and counted;
> > + * device interconnect (VRAM, P2P) entries are skipped. The counts are solely
> > + * exposed through debugfs, so the callback is only registered when
> > + * CONFIG_DEBUG_FS is enabled.
> > + */
> > +static void xe_userptr_dma_map_account(struct drm_gpusvm *gpusvm,
> > +				       const struct drm_pagemap_addr *addr,
> > +				       int sign)
> > +{
> > +	struct xe_vm *vm = container_of(gpusvm, struct xe_vm, svm.gpusvm);
> > +	unsigned int order = addr->order;
> > +
> > +	if (addr->proto != DRM_INTERCONNECT_SYSTEM)
> > +		return;
> > +
> > +	/*
> > +	 * The CPU-PTE map order reported by hmm can exceed MAX_PAGE_ORDER
> > +	 * (e.g. an order-18 1GB mapping), so clamp the array index to avoid an
> > +	 * out-of-bounds access. Clamping is symmetric across the +1/-1 calls,
> > +	 * so the per-order counters stay balanced.
> > +	 */
> > +	order = min(order, (unsigned int)NR_PAGE_ORDERS - 1);
> > +
> > +	atomic_long_add((long)(1UL << addr->order) * sign,
> > +			&vm->xe->mem.dma_mapped_pages_svm[order]);
> > +}
> > +
> > +static const struct drm_gpusvm_ops xe_userptr_gpusvm_ops = {
> > +	.dma_map_account = xe_userptr_dma_map_account,
> > +};
> > +#endif
> > +
> > +/**
> > + * xe_userptr_gpusvm_ops_get() - Accounting ops for the simple gpusvm instance
> > + *
> > + * The core drm_gpusvm_pages-only instance used for userptr (and, in fault
> > + * mode, shared with SVM) is initialised with these ops so that DMA-mapped
> > + * system pages are accounted and exposed through debugfs. Returns NULL when
> > + * CONFIG_DEBUG_FS is disabled, in which case no accounting is kept and the
> > + * instance is initialised without ops.
> > + *
> > + * Return: Pointer to the restricted &drm_gpusvm_ops, or NULL.
> > + */
> > +const struct drm_gpusvm_ops *xe_userptr_gpusvm_ops_get(void)
> > +{
> > +#if IS_ENABLED(CONFIG_DEBUG_FS)
> > +	return &xe_userptr_gpusvm_ops;
> > +#else
> > +	return NULL;
> > +#endif
> > +}
> > +
> > static void xe_userptr_assert_in_notifier(struct xe_vm *vm)
> > {
> > 	lockdep_assert(lockdep_is_held_type(&vm->svm.gpusvm.notifier_lock, 0) ||
> > diff --git a/drivers/gpu/drm/xe/xe_userptr.h b/drivers/gpu/drm/xe/xe_userptr.h
> > index 2a3cd1b5efbb..1e9601be5713 100644
> > --- a/drivers/gpu/drm/xe/xe_userptr.h
> > +++ b/drivers/gpu/drm/xe/xe_userptr.h
> > @@ -108,6 +108,7 @@ int __xe_vm_userptr_needs_repin(struct xe_vm *vm);
> > int xe_vm_userptr_check_repin(struct xe_vm *vm);
> > int xe_vma_userptr_pin_pages(struct xe_userptr_vma *uvma);
> > int xe_vma_userptr_check_repin(struct xe_userptr_vma *uvma);
> > +const struct drm_gpusvm_ops *xe_userptr_gpusvm_ops_get(void);
> > #else
> > static inline void xe_userptr_remove(struct xe_userptr_vma *uvma) {}
> > 
> > -- 
> > 2.34.1
> >