From: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>
To: intel-xe@lists.freedesktop.org
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>,
Rodrigo Vivi <rodrigo.vivi@intel.com>,
Matthew Brost <matthew.brost@intel.com>,
Matthew Auld <matthew.auld@intel.com>,
Andi Shyti <andi.shyti@kernel.org>
Subject: Re: [PATCH v3] drm/xe: Add a per-client debugfs file reporting bo NUMA node distribution
Date: Wed, 18 Nov 2026 16:01:22 +0100 [thread overview]
Message-ID: <eb25eaec041bad67aa7d7241d79f22d61b50b07a.camel@linux.intel.com> (raw)
In-Reply-To: <20261118134203.646500-1-thomas.hellstrom@linux.intel.com>
Subject above should ofc be v4.
/Thomas
On Wed, 2026-11-18 at 14:42 +0100, Thomas Hellström wrote:
> When debugging NUMA placement of bo system memory, there is currently
> no way to tell on which nodes the pages backing a bo reside. The
> /proc/<pid>/numa_maps interface doesn't help since bo CPU mappings
> are
> VM_PFNMAP, and clients may have thousands of bos, so a dump of all
> bos
> is undesirable.
>
> Add a "bo_numa" file to the drm core per-client debugfs directory,
> client-<id>/. The file implements a request-response protocol per
> open
> file: Writing a GEM handle valid for the client takes a snapshot,
> under
> the bo's dma_resv lock, of the bo size, placement, and the number of
> pages on each NUMA node. A bo whose pages are swapped out, backed up
> or purged is reported as swapped/purged, and one whose pages are not
> populated as not populated. Subsequent reads return that snapshot.
> Once the
> snapshot has been completely read it is discarded, and a new handle
> may be written. Until then writes fail with -EBUSY, so that a pending
> snapshot can't be replaced before it is consumed. This allows
> querying
> many bos using a single open file.
>
> The write-request / read-response protocol per open file follows the
> simple_transaction helpers in fs/libfs.c, used by e.g. nfsd and
> selinuxfs, and amdgpu already populates the per-client debugfs
> directory with its vm_pagetable_info file.
>
> For sg bos, typically imported dma-bufs, the backing pages are owned
> by the exporter and can't be inspected, so only size and placement
> are
> reported.
>
> Example:
> exec 3<>/sys/kernel/debug/dri/client-<id>/bo_numa
> echo <handle1> >&3
> cat <&3
> echo <handle2> >&3
> cat <&3
> exec 3>&-
>
> The client id can be obtained from the drm-client-id fdinfo key.
>
> v2:
> - Use drm_coredump_printer_is_full() (Andi Shyti)
> - Drop the Assisted-by tag and add a Signed-off-by (Andi Shyti)
>
> v3:
> - Report swapped or purged bos as swapped/purged (Sashiko)
>
> v4:
> - Also report bos purged via xe_ttm_bo_purge() as swapped/purged
> (Sashiko)
>
> Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Matthew Brost <matthew.brost@intel.com>
> Cc: Matthew Auld <matthew.auld@intel.com>
> Cc: Andi Shyti <andi.shyti@kernel.org>
> Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
> # v3
> ---
> drivers/gpu/drm/xe/xe_bo.c | 113 +++++++++++++++++++++++
> drivers/gpu/drm/xe/xe_bo.h | 6 ++
> drivers/gpu/drm/xe/xe_debugfs.c | 157
> ++++++++++++++++++++++++++++++++
> drivers/gpu/drm/xe/xe_debugfs.h | 3 +
> drivers/gpu/drm/xe/xe_device.c | 2 +
> 5 files changed, 281 insertions(+)
>
> diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c
> index 42a6aa1a2e4a..353ac1e9a9d6 100644
> --- a/drivers/gpu/drm/xe/xe_bo.c
> +++ b/drivers/gpu/drm/xe/xe_bo.c
> @@ -6,12 +6,14 @@
> #include "xe_bo.h"
>
> #include <linux/dma-buf.h>
> +#include <linux/nodemask.h>
> #include <linux/nospec.h>
>
> #include <drm/drm_drv.h>
> #include <drm/drm_dumb_buffers.h>
> #include <drm/drm_gem_ttm_helper.h>
> #include <drm/drm_managed.h>
> +#include <drm/drm_print.h>
> #include <drm/ttm/ttm_backup.h>
> #include <drm/ttm/ttm_device.h>
> #include <drm/ttm/ttm_placement.h>
> @@ -4248,6 +4250,117 @@ void
> xe_bo_runtime_pm_release_mmap_offset(struct xe_bo *bo)
> list_del_init(&bo->vram_userfault_link);
> }
>
> +#if IS_ENABLED(CONFIG_DEBUG_FS)
> +static void xe_bo_count_numa_pages(struct ttm_tt *tt, unsigned long
> *nodes)
> +{
> + pgoff_t i;
> +
> + for (i = 0; i < tt->num_pages; ++i)
> + nodes[page_to_nid(tt->pages[i])]++;
> +}
> +
> +static void xe_bo_print_numa_info_locked(struct xe_bo *bo, unsigned
> long *nodes,
> + struct drm_printer *p)
> +{
> + struct ttm_resource *res = bo->ttm.resource;
> + struct ttm_tt *tt = bo->ttm.ttm;
> + unsigned long total = 0;
> + const char *name;
> + int nid;
> +
> + xe_bo_assert_held(bo);
> +
> + name = res ? xe_mem_type_to_name[res->mem_type] : NULL;
> + drm_printf(p, "size: %zu\n", xe_bo_size(bo));
> + if (name)
> + drm_printf(p, "placement: %s\n", name);
> + else if (res)
> + drm_printf(p, "placement: mem_type %u\n", res-
> >mem_type);
> + else
> + drm_puts(p, "placement: none\n");
> +
> + /* The pages of sg bos are owned by the exporter and not
> visible. */
> + if (bo->ttm.type == ttm_bo_type_sg) {
> + drm_puts(p, "pages: unknown (sg bo)\n");
> + return;
> + }
> +
> + /*
> + * Purging via xe_ttm_bo_purge() guts the bo without setting
> any
> + * swapped or backed-up flag on the tt.
> + */
> + if (xe_bo_is_purged(bo) || (tt && ttm_tt_is_swapped(tt))) {
> + drm_puts(p, "pages: swapped/purged\n");
> + return;
> + }
> +
> + if (tt && !ttm_tt_is_populated(tt)) {
> + drm_puts(p, "pages: not populated\n");
> + return;
> + }
> +
> + if (tt)
> + xe_bo_count_numa_pages(tt, nodes);
> +
> + for_each_node(nid) {
> + if (!nodes[nid] && !node_online(nid))
> + continue;
> + drm_printf(p, "node%d: %lu\n", nid, nodes[nid]);
> + total += nodes[nid];
> + }
> +
> + drm_printf(p, "total: %lu\n", total);
> +}
> +
> +/**
> + * xe_bo_print_numa_info() - Print the NUMA node distribution of a
> bo
> + * @file: The drm_file of the client owning @handle.
> + * @handle: The GEM handle of the bo.
> + * @p: The drm_printer to print to.
> + *
> + * Print the size and placement of the bo and, for bos with
> + * driver-allocated system pages, the number of pages residing on
> each
> + * NUMA node. If the bo's pages are currently swapped out, backed up
> or
> + * purged by the shrinker, the bo is reported as swapped/purged, and
> if
> + * they are not populated, as not populated. For sg bos, typically
> + * imported dma-bufs, the backing pages are owned by the exporter
> and are
> + * reported as unknown. The information is gathered under the bo's
> + * dma_resv lock and thus represents a consistent snapshot.
> + *
> + * Return: 0 on success, -ENOENT if @handle doesn't refer to a bo,
> + * negative error code on other failures.
> + */
> +int xe_bo_print_numa_info(struct drm_file *file, u32 handle,
> + struct drm_printer *p)
> +{
> + struct drm_gem_object *obj;
> + unsigned long *nodes;
> + int ret;
> +
> + nodes = kcalloc(nr_node_ids, sizeof(*nodes),
> GFP_KERNEL_ACCOUNT);
> + if (!nodes)
> + return -ENOMEM;
> +
> + obj = drm_gem_object_lookup(file, handle);
> + if (!obj) {
> + ret = -ENOENT;
> + goto out_free;
> + }
> +
> + ret = dma_resv_lock_interruptible(obj->resv, NULL);
> + if (!ret) {
> + xe_bo_print_numa_info_locked(gem_to_xe_bo(obj),
> nodes, p);
> + dma_resv_unlock(obj->resv);
> + }
> +
> + drm_gem_object_put(obj);
> +out_free:
> + kfree(nodes);
> +
> + return ret;
> +}
> +#endif
> +
> #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
> #include "tests/xe_bo.c"
> #endif
> diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h
> index 77b3a02e9b8e..13197e4e0358 100644
> --- a/drivers/gpu/drm/xe/xe_bo.h
> +++ b/drivers/gpu/drm/xe/xe_bo.h
> @@ -112,6 +112,7 @@ enum xe_madv_purgeable_state {
> XE_MADV_PURGEABLE_PURGED,
> };
>
> +struct drm_printer;
> struct sg_table;
>
> struct xe_bo *xe_bo_alloc(void);
> @@ -458,6 +459,11 @@ int xe_bo_dumb_create(struct drm_file
> *file_priv,
> struct drm_device *dev,
> struct drm_mode_create_dumb *args);
>
> +#if IS_ENABLED(CONFIG_DEBUG_FS)
> +int xe_bo_print_numa_info(struct drm_file *file, u32 handle,
> + struct drm_printer *p);
> +#endif
> +
> bool xe_bo_needs_ccs_pages(struct xe_bo *bo);
>
> int xe_bo_decompress(struct xe_bo *bo);
> diff --git a/drivers/gpu/drm/xe/xe_debugfs.c
> b/drivers/gpu/drm/xe/xe_debugfs.c
> index e59d4964d811..1a74a7106462 100644
> --- a/drivers/gpu/drm/xe/xe_debugfs.c
> +++ b/drivers/gpu/drm/xe/xe_debugfs.c
> @@ -11,6 +11,8 @@
> #include <linux/string_helpers.h>
>
> #include <drm/drm_debugfs.h>
> +#include <drm/drm_file.h>
> +#include <drm/drm_print.h>
>
> #include "regs/xe_pmt.h"
> #include "xe_bo.h"
> @@ -785,3 +787,158 @@ void xe_debugfs_register(struct xe_device *xe)
> else if (IS_SRIOV_VF(xe))
> xe_sriov_vf_debugfs_register(xe, root);
> }
> +
> +/*
> + * Per-client "bo_numa" file: Report the NUMA node distribution of
> the
> + * system pages backing a single bo.
> + *
> + * The file implements a request-response protocol: write() takes a
> GEM
> + * handle, valid for the client, and takes a snapshot of the bo's
> page
> + * distribution under the bo's dma_resv lock. Subsequent read()s
> return
> + * that snapshot. Once the snapshot has been completely read, it is
> + * discarded and a new handle may be written. Until then, writes
> return
> + * -EBUSY, so that a pending snapshot is never replaced before it is
> + * consumed.
> + *
> + * See xe_bo_print_numa_info() for the snapshot contents.
> + */
> +struct bo_numa_snapshot {
> + /* Serializes reads and writes on the open file. */
> + struct mutex lock;
> + /* The pending formatted snapshot, or NULL if none. */
> + char *buf;
> + /* Length of the formatted snapshot. */
> + size_t len;
> +};
> +
> +static int bo_numa_open(struct inode *inode, struct file *f)
> +{
> + struct bo_numa_snapshot *snap;
> +
> + snap = kzalloc_obj(*snap, GFP_KERNEL_ACCOUNT);
> + if (!snap)
> + return -ENOMEM;
> +
> + mutex_init(&snap->lock);
> + f->private_data = snap;
> +
> + return nonseekable_open(inode, f);
> +}
> +
> +static int bo_numa_release(struct inode *inode, struct file *f)
> +{
> + struct bo_numa_snapshot *snap = f->private_data;
> +
> + kvfree(snap->buf);
> + mutex_destroy(&snap->lock);
> + kfree(snap);
> +
> + return 0;
> +}
> +
> +static ssize_t bo_numa_read(struct file *f, char __user *ubuf,
> size_t size,
> + loff_t *pos)
> +{
> + struct bo_numa_snapshot *snap = f->private_data;
> + ssize_t ret = 0;
> +
> + mutex_lock(&snap->lock);
> + if (snap->buf) {
> + ret = simple_read_from_buffer(ubuf, size, pos, snap-
> >buf,
> + snap->len);
> + if (*pos >= snap->len) {
> + kvfree(snap->buf);
> + snap->buf = NULL;
> + }
> + }
> + mutex_unlock(&snap->lock);
> +
> + return ret;
> +}
> +
> +static ssize_t bo_numa_write(struct file *f, const char __user
> *ubuf,
> + size_t size, loff_t *pos)
> +{
> + struct drm_file *file = file_inode(f)->i_private;
> + struct bo_numa_snapshot *snap = f->private_data;
> + struct drm_print_iterator iter;
> + struct drm_printer p;
> + size_t bufsize = PAGE_SIZE / 2;
> + char *buf = NULL;
> + u32 handle;
> + int ret;
> +
> + ret = kstrtou32_from_user(ubuf, size, 0, &handle);
> + if (ret)
> + return ret;
> +
> + ret = mutex_lock_interruptible(&snap->lock);
> + if (ret)
> + return ret;
> +
> + if (snap->buf) {
> + ret = -EBUSY;
> + goto out_unlock;
> + }
> +
> + /*
> + * Retry with a larger buffer until the snapshot fits. The
> coredump
> + * printer is used as a generic bounded buffer printer, and
> reports
> + * when its buffer is full.
> + */
> + do {
> + kvfree(buf);
> + bufsize *= 2;
> + buf = kvzalloc(bufsize, GFP_KERNEL_ACCOUNT);
> + if (!buf) {
> + ret = -ENOMEM;
> + goto out_unlock;
> + }
> +
> + iter = (struct drm_print_iterator) {
> + .data = buf,
> + .remain = bufsize,
> + };
> + p = drm_coredump_printer(&iter);
> + ret = xe_bo_print_numa_info(file, handle, &p);
> + if (ret)
> + goto out_free;
> + } while (drm_coredump_printer_is_full(&p));
> +
> + snap->buf = buf;
> + snap->len = bufsize - iter.remain;
> + *pos = 0;
> + buf = NULL;
> +
> +out_free:
> + kvfree(buf);
> +out_unlock:
> + mutex_unlock(&snap->lock);
> +
> + return ret ?: size;
> +}
> +
> +static const struct file_operations bo_numa_fops = {
> + .owner = THIS_MODULE,
> + .open = bo_numa_open,
> + .release = bo_numa_release,
> + .read = bo_numa_read,
> + .write = bo_numa_write,
> +};
> +
> +/**
> + * xe_debugfs_client_add() - Add xe-specific per-client debugfs
> files
> + * @file: The drm_file of the client.
> + *
> + * Populate the drm core per-client debugfs directory with xe-
> specific
> + * files. The files are removed by the drm core together with the
> + * directory when @file is released.
> + */
> +void xe_debugfs_client_add(struct drm_file *file)
> +{
> + if (IS_ERR_OR_NULL(file->debugfs_client))
> + return;
> +
> + debugfs_create_file("bo_numa", 0600, file->debugfs_client,
> file,
> + &bo_numa_fops);
> +}
> diff --git a/drivers/gpu/drm/xe/xe_debugfs.h
> b/drivers/gpu/drm/xe/xe_debugfs.h
> index 88d91c78036b..90effba271d9 100644
> --- a/drivers/gpu/drm/xe/xe_debugfs.h
> +++ b/drivers/gpu/drm/xe/xe_debugfs.h
> @@ -8,6 +8,7 @@
>
> #include <linux/types.h>
>
> +struct drm_file;
> struct xe_device;
>
> #ifdef CONFIG_DEBUG_FS
> @@ -16,12 +17,14 @@ bool xe_fault_csc_hw_error(void);
> bool xe_fault_wedge_cold_reset(void);
> bool xe_fault_mempage_offline(void);
> void xe_debugfs_register(struct xe_device *xe);
> +void xe_debugfs_client_add(struct drm_file *file);
> #else
> static inline bool xe_fault_gt_reset(void) { return false; }
> static inline bool xe_fault_csc_hw_error(void) { return false; }
> static inline bool xe_fault_wedge_cold_reset(void) { return false; }
> static inline bool xe_fault_mempage_offline(void) { return false; }
> static inline void xe_debugfs_register(struct xe_device *xe) { }
> +static inline void xe_debugfs_client_add(struct drm_file *file) { }
> #endif
>
> #endif
> diff --git a/drivers/gpu/drm/xe/xe_device.c
> b/drivers/gpu/drm/xe/xe_device.c
> index 616e02066ae6..f5a581d39c96 100644
> --- a/drivers/gpu/drm/xe/xe_device.c
> +++ b/drivers/gpu/drm/xe/xe_device.c
> @@ -127,6 +127,8 @@ static int xe_file_open(struct drm_device *dev,
> struct drm_file *file)
> put_task_struct(task);
> }
>
> + xe_debugfs_client_add(file);
> +
> return 0;
> }
>
next prev parent reply other threads:[~2026-11-18 15:02 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 13:42 [PATCH v3] drm/xe: Add a per-client debugfs file reporting bo NUMA node distribution Thomas Hellström
2026-11-18 15:01 ` Thomas Hellström [this message]
-- strict thread matches above, loose matches on Subject: below --
2026-11-04 15:13 Thomas Hellström
2026-11-04 15:36 ` sashiko-bot
2026-11-12 13:58 ` Himal prasad Ghimiray
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=eb25eaec041bad67aa7d7241d79f22d61b50b07a.camel@linux.intel.com \
--to=thomas.hellstrom@linux.intel.com \
--cc=andi.shyti@kernel.org \
--cc=himal.prasad.ghimiray@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=matthew.auld@intel.com \
--cc=matthew.brost@intel.com \
--cc=rodrigo.vivi@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox