Re: [PATCH V16 10/12] drm/xe: Add sysfs interface for bad gpu vram pages

[Rodrigo Vivi <rodrigo.vivi@intel.com>]

On Mon, Aug 17, 2026 at 02:58:31PM +0000, Upadhyay, Tejas wrote:
> 
> 
> > -----Original Message-----
> > From: Wajdeczko, Michal <michal.wajdeczko@intel.com>
> > Sent: 17 August 2026 16:57
> > To: Upadhyay, Tejas <tejas.upadhyay@intel.com>; intel-
> > xe@lists.freedesktop.org; Vivi, Rodrigo <rodrigo.vivi@intel.com>; Thomas
> > Hellström <thomas.hellstrom@linux.intel.com>
> > Cc: Ghimiray, Himal Prasad <himal.prasad.ghimiray@intel.com>
> > Subject: Re: [PATCH V16 10/12] drm/xe: Add sysfs interface for bad gpu vram
> > pages
> > 
> > 
> > 
> > On 8/17/2026 8:51 AM, Tejas Upadhyay wrote:
> > > Include a sysfs interface designed to expose information about bad
> > > VRAM pages — those identified as having hardware faults (e.g., ECC
> > > errors). This interface allows userspace tools and administrators to
> > > monitor the health of the GPU's local memory and track the status of
> > > page retirement. Details on bad gpu vram pages can be found under
> > > /sys/bus/pci/devices/<bdf>/vram_bad_pages.
> > 
> > since those new files are xe driver specific, shouldn't we refer to them using
> > 
> > 	/sys/bus/pci/drivers/xe/<bdf>/vram...
> > 
> > >
> > > The format is: pfn : gpu_page_size : flags
> > 
> > kernel documentation [1] says
> > 
> > 	"Mixing types, expressing multiple lines of data, and doing
> > 	fancy formatting of data is heavily frowned upon"
> > 
> > [1] https://docs.kernel.org/filesystems/sysfs.html#attributes
> > 
> > so to follow the guidelines maybe we expose the separate files:
> > 
> > /sys/bus/pci/drivers/xe/<bdf>/vram_page_size		u64
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_count	u64
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_reserved	u64[]
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_pending	u64[]
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_failed	u64[]
> > 
> > or
> > 
> > /sys/bus/pci/drivers/xe/<bdf>
> > |
> > +-- vram/
> >     +-- page_size	u64
> >     +-- bad_pages/
> >         +-- count	u64
> >         +-- reserved	u64[]
> >         +-- pending	u64[]
> >         +-- failed	u64[]
> > 
> > then
> > 
> > /sys/bus/pci/drivers/xe/<bdf>/vram_page_size:0x1000
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_count:5
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_reserved:0x000000000000
> > 0000
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_pending:0x0000000001234
> > 000
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_pending:0x0000000001235
> > 000
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_pending:0x0000000001236
> > 000
> > /sys/bus/pci/drivers/xe/<bdf>/vram_bad_pages_pending:0x0000000001237
> > 000
> 
> Thanks for comment, this is documented format by design doc. Sysman also depending on this format. So I don’t see this can be done without design being changed for everyone.

Internal design docs don't superseed upstream documentation.
It is the other way around.

But also, the files will be there one way or another. Both paths
are valid, so I don't believe that change in here force changes
in the userspace. Although, yes consistency is good...

That said, I don't have a strong feeling for one way or the other.

Since we are adding to the device level anyway, I believe it should
be okay. But Michal, do you know any doc or any precedence that kind
of force us to go the other way?

> 
> Tejas
> > 
> > >
> > > flags:
> > >   R: reserved, this gpu page is reserved.
> > >   P: pending for reserve, this gpu page is marked as bad, will be
> > >      reserved in next window of page_reserve.
> > >   F: unable to reserve, this gpu page can't be reserved due to some
> > >      reasons.
> > >
> > > For example, cat /sys/bus/pci/devices/<bdf>/vram_bad_pages:
> > >   max_pages : 10000
> > >   0x0000000000000000 : 0x0000000000001000 : R
> > >   0x0000000000001234 : 0x0000000000001000 : P
> > >
> > > The sysfs binary attribute is created under the PCI device kobject
> > > when the platform supports it and the configfs bad_page_reservation
> > > policy is enabled. Uses RCU-protected list traversal so reads never
> > > block normal VRAM allocation operations.
> > >
> > > Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
> > > ---
> > >  drivers/gpu/drm/xe/xe_device_sysfs.c       |   7 +
> > >  drivers/gpu/drm/xe/xe_ttm_vram_mgr.c       | 141
> > +++++++++++++++++++++
> > >  drivers/gpu/drm/xe/xe_ttm_vram_mgr.h       |   1 +
> > >  drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h |   2 +
> > >  4 files changed, 151 insertions(+)
> > >
> > > diff --git a/drivers/gpu/drm/xe/xe_device_sysfs.c
> > > b/drivers/gpu/drm/xe/xe_device_sysfs.c
> > > index a73e0e957cb0..47c5be4180fe 100644
> > > --- a/drivers/gpu/drm/xe/xe_device_sysfs.c
> > > +++ b/drivers/gpu/drm/xe/xe_device_sysfs.c
> > > @@ -8,12 +8,14 @@
> > >  #include <linux/pci.h>
> > >  #include <linux/sysfs.h>
> > >
> > > +#include "xe_configfs.h"
> > >  #include "xe_device.h"
> > >  #include "xe_device_sysfs.h"
> > >  #include "xe_mmio.h"
> > >  #include "xe_pcode_api.h"
> > >  #include "xe_pcode.h"
> > >  #include "xe_pm.h"
> > > +#include "xe_ttm_vram_mgr.h"
> > >
> > >  /**
> > >   * DOC: Xe device sysfs
> > > @@ -267,6 +269,7 @@ static const struct attribute_group
> > > auto_link_downgrade_attr_group = {  int xe_device_sysfs_init(struct
> > > xe_device *xe)  {
> > >  	struct device *dev = xe->drm.dev;
> > > +	bool policy;
> > >  	int ret;
> > >
> > >  	if (xe->d3cold.capable) {
> > > @@ -285,5 +288,9 @@ int xe_device_sysfs_init(struct xe_device *xe)
> > >  			return ret;
> > >  	}
> > >
> > > +	policy = xe_configfs_get_bad_page_reservation(to_pci_dev(dev));
> > > +	if (xe->info.platform == XE_CRESCENTISLAND && policy)
> > > +		xe_ttm_vram_sysfs_init(xe);
> > > +
> > >  	return 0;
> > >  }
> > > diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
> > > b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
> > > index a48196ce303f..73cd06c22d37 100644
> > > --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
> > > +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
> > > @@ -820,3 +820,144 @@ int xe_ttm_vram_handle_addr_fault(struct
> > xe_device *xe, u64 addr)
> > >  	return xe_ttm_vram_reserve_page_at_addr(xe, addr - vr->dpa_base,
> > > vram_mgr, mm);  }  EXPORT_SYMBOL(xe_ttm_vram_handle_addr_fault);
> > > +
> > > +static size_t serialize_bad_pages(struct xe_ttm_vram_mgr *mgr, char
> > > +*buf, size_t max_len) {
> > > +	struct xe_ttm_vram_offline_resource *pos;
> > > +	struct gpu_buddy_block *block;
> > > +	size_t s = 0;
> > > +	int printed;
> > > +	int count = 0;
> > > +
> > > +	rcu_read_lock();
> > > +
> > > +	printed = scnprintf(buf + s, max_len - s, "max_pages: %d\n", mgr-
> > >max_pages);
> > > +	s += printed;
> > > +
> > > +	list_for_each_entry_rcu(pos, &mgr->offlined_pages, offlined_link) {
> > > +		if (count >= 10000 || s >= max_len)
> > > +			break;
> > > +
> > > +		block = list_first_entry_or_null(&pos->blocks, struct
> > gpu_buddy_block, link);
> > > +		if (!block)
> > > +			continue;
> > > +
> > > +		printed = scnprintf(buf + s, max_len - s, "0x%016llx : 0x%016llx
> > : %c\n",
> > > +				    gpu_buddy_block_offset(block) >>
> > PAGE_SHIFT,
> > > +				    gpu_buddy_block_size(&mgr->mm, block),
> > 'R');
> > > +		s += printed;
> > > +		count++;
> > > +	}
> > > +	list_for_each_entry_rcu(pos, &mgr->queued_pages, queued_link) {
> > > +		u64 pfn, blk_size;
> > > +
> > > +		if (count >= 10000 || s >= max_len)
> > > +			break;
> > > +
> > > +		block = list_first_entry_or_null(&pos->blocks, struct
> > gpu_buddy_block, link);
> > > +		if (block) {
> > > +			pfn = gpu_buddy_block_offset(block) >> PAGE_SHIFT;
> > > +			blk_size = gpu_buddy_block_size(&mgr->mm, block);
> > > +		} else {
> > > +			pfn = pos->addr >> PAGE_SHIFT;
> > > +			blk_size = PAGE_SIZE;
> > > +		}
> > > +
> > > +		printed = scnprintf(buf + s, max_len - s, "0x%016llx : 0x%016llx
> > : %c\n",
> > > +				    pfn, blk_size, pos->status ? 'F' : 'P');
> > > +		s += printed;
> > > +		count++;
> > > +	}
> > > +
> > > +	rcu_read_unlock();
> > > +	return s;
> > > +}
> > > +
> > > +static ssize_t vram_bad_pages_bin_read(struct file *filp, struct kobject
> > *kobj,
> > > +				       const struct bin_attribute *attr, char *buf,
> > > +				       loff_t off, size_t count)
> > > +{
> > > +	struct device *dev = kobj_to_dev(kobj);
> > > +	struct pci_dev *pdev = to_pci_dev(dev);
> > > +	struct ttm_resource_manager *man;
> > > +	struct xe_ttm_vram_mgr *mgr;
> > > +	size_t allocation_size;
> > > +	struct xe_device *xe;
> > > +	size_t full_data_len;
> > > +	int active_entries;
> > > +	char *temp_buf;
> > > +
> > > +	xe = pdev_to_xe_device(pdev);
> > > +	man = ttm_manager_type(&xe->ttm, XE_PL_VRAM0);
> > > +	if (!man)
> > > +		return -ENODEV;
> > > +	mgr = to_xe_ttm_vram_mgr(man);
> > > +
> > > +	active_entries = READ_ONCE(mgr->n_offlined_pages) +
> > > +READ_ONCE(mgr->n_queued_pages);
> > > +
> > > +	if (active_entries > 10000)
> > > +		active_entries = 10000;
> > > +
> > > +	allocation_size = 64 + (active_entries * 48);
> > > +
> > > +	temp_buf = kvmalloc(allocation_size, GFP_KERNEL);
> > > +	if (!temp_buf)
> > > +		return -ENOMEM;
> > > +
> > > +	/* serialize_bad_pages uses rcu_read_lock internally */
> > > +	full_data_len = serialize_bad_pages(mgr, temp_buf, allocation_size);
> > > +
> > > +	if (off >= full_data_len) {
> > > +		kvfree(temp_buf);
> > > +		return 0;
> > > +	}
> > > +
> > > +	if (off + count > full_data_len)
> > > +		count = full_data_len - off;
> > > +
> > > +	memcpy(buf, temp_buf + off, count);
> > > +
> > > +	kvfree(temp_buf);
> > > +	return count;
> > > +}
> > > +
> > > +static const struct bin_attribute bin_attr_vram_bad_pages = {
> > > +	.attr = { .name = "vram_bad_pages", .mode = 0444 },
> > > +	.read = vram_bad_pages_bin_read,
> > > +	.size = 0,
> > > +};
> > > +
> > > +static void xe_ttm_vram_sysfs_fini(void *arg) {
> > > +	struct xe_device *xe = arg;
> > > +	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
> > > +
> > > +	sysfs_remove_bin_file(&pdev->dev.kobj,
> > &bin_attr_vram_bad_pages); }
> > > +
> > > +/**
> > > + * xe_ttm_vram_sysfs_init - Initialize vram bad pages sysfs binary
> > > +file
> > > + * @xe: Xe Device object
> > > + *
> > > + * Creates a binary sysfs file under the PCI device for reading
> > > + * offlined and queued VRAM pages. Supports large entry counts
> > > + * via offset/count pagination.
> > > + *
> > > + * Returns: 0 on success, negative error code on error.
> > > + */
> > > +int xe_ttm_vram_sysfs_init(struct xe_device *xe) {
> > > +	struct pci_dev *pdev = to_pci_dev(xe->drm.dev);
> > > +	int err;
> > > +
> > > +	err = sysfs_create_bin_file(&pdev->dev.kobj,
> > &bin_attr_vram_bad_pages);
> > > +	if (err) {
> > > +		dev_err(&pdev->dev,
> > > +			"Failed to create vram_bad_pages sysfs: %d\n",
> > > +			err);
> > > +		return err;
> > > +	}
> > > +
> > > +	return devm_add_action_or_reset(&pdev->dev,
> > xe_ttm_vram_sysfs_fini,
> > > +xe); } EXPORT_SYMBOL(xe_ttm_vram_sysfs_init);
> > > diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.h
> > > b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.h
> > > index d5392beff30c..eb55b0f74ef3 100644
> > > --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.h
> > > +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.h
> > > @@ -32,6 +32,7 @@ void xe_ttm_vram_get_used(struct
> > ttm_resource_manager *man,
> > >  			  u64 *used, u64 *used_visible);
> > >
> > >  int xe_ttm_vram_handle_addr_fault(struct xe_device *xe, u64 addr);
> > > +int xe_ttm_vram_sysfs_init(struct xe_device *xe);
> > >  static inline struct xe_ttm_vram_mgr_resource *
> > > to_xe_ttm_vram_mgr_resource(struct ttm_resource *res)  { diff --git
> > > a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
> > > b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
> > > index dc97b0ad0e51..efcf3e1d4e80 100644
> > > --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
> > > @@ -37,6 +37,8 @@ struct xe_ttm_vram_mgr {
> > >  	struct mutex lock;
> > >  	/** @mem_type: The TTM memory type */
> > >  	u32 mem_type;
> > > +	/** @max_pages: max pages that can be in offline queue retrieved
> > from FW */
> > > +	u16 max_pages;
> > >  };
> > >
> > >  /**
>