* [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
2026-07-02 19:25 [PATCH 0/4] Introduce nvgrace-egm driver for Extended GPU Memory Ankit Agrawal
@ 2026-07-02 19:25 ` Ankit Agrawal
2026-07-02 22:22 ` Armin Wolf
2026-07-02 19:25 ` [PATCH 2/4] platform/nvidia: Implement mmap and memory scrubbing for EGM chardev Ankit Agrawal
` (2 subsequent siblings)
3 siblings, 1 reply; 7+ messages in thread
From: Ankit Agrawal @ 2026-07-02 19:25 UTC (permalink / raw)
To: gregkh, jai.luthra, ilpo.jarvinen
Cc: W_Armin, alex, jgg, vsethi, mhonap, mochs, ankita, clg,
linux-kernel
The Extended GPU Memory (EGM) feature enables the GPU to access system
memory allocations within and across nodes through a high-bandwidth path
on Grace-based systems. The GPU can utilize system memory on the same
socket, a different socket or a different node in a multi-node setup.
When EGM mode is enabled through SBIOS, host system memory is partitioned
into two parts: a Hypervisor region for the host OS and a
Hypervisor-Invisible (HI) region for the VM. Only the hypervisor region
is part of the host EFI map and visible to the host OS on bootup. The
HI partition is interchangeably called the EGM region. Its base SPA and
size are exposed through ACPI DSDT properties.
Whilst the EGM region is accessible on the host, it is not
added/hot-plugged to the kernel. The HI region is assigned to a VM
by mapping the QEMU VMA to the SPA using remap_pfn_range(). So
it looks like the following:
|---- Sysmem ----| VM view
| |
|IPA <-> SPA map |
| |
|--- HI / EGM ---|-- Host Mem --| Host view
Introduce the nvgrace-egm platform driver module and implement full EGM
region discovery, char device exposure and sysfs metadata.
Build infrastructure:
- Kconfig, Makefile entries, MAINTAINERS entry.
- An 'egm' device class and a range of char device minor numbers.
MAX_EGM_NODES is set to 4 to matches the largest 4-socket Grace
configuration.
Discovery:
- Walk all PCI devices at module init time. For each one advertising
"nvidia,egm-pxm", read the EGM region properties (base SPA, size and
PXM domain) from the ACPI DSD table via the GPU device handle, using
"nvidia,egm-base-pa" and "nvidia,egm-size" properties.
- Maintain a module-level list (egm_chardevs) of discovered regions,
de-duplicating by PXM so each socket gets exactly one EGM device
regardless of how many GPUs it has (2 GPUs per EGM device in GB200).
Char device creation:
- Each EGM region is exposed as /dev/egmX, where X is the PXM number.
Embed struct device and struct cdev inside nvgrace_egm_dev so the
device lifecycle follows the standard kernel device model.
- Implement setup_egm_chardev() and del_egm_chardev() for setup and
teardown. The devnode callback sets device permissions to 0600.
- Stub file operations (open, release and mmap) are registered.
GPU tracking and sysfs:
- Track which PCI devices are associated with each EGM region in a
per-device GPU list (struct gpu_node).
- Expose sysfs symlinks from each EGM device to its associated GPU PCI
devices to allow the userspace to determine GPU-to-socket affinity and
replicate the host EGM topology in the VM. Helps admin understand
GPU association.
- Expose the EGM region size via an egm_size sysfs attribute so QEMU
can determine the memory-backend-file size. On a 2-socket Grace
Blackwell setup this appears as:
Socket0: /sys/devices/virtual/egm/egm4/egm_size
Socket1: /sys/devices/virtual/egm/egm5/egm_size
Introduce nvgrace_egm_create_pci_egm_devs() and its counterpart
nvgrace_egm_destroy_pci_devs() that are called from module init/exit.
Suggested-by: Alex Williamson <alex@shazbot.org>
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Suggested-by: Aniket Agashe <aniketa@nvidia.com>
Suggested-by: Matthew R. Ochs <mochs@nvidia.com>
Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
---
MAINTAINERS | 5 +
drivers/platform/Kconfig | 1 +
drivers/platform/Makefile | 1 +
drivers/platform/nvidia/Kconfig | 10 +
drivers/platform/nvidia/Makefile | 3 +
drivers/platform/nvidia/egm.c | 459 +++++++++++++++++++++++++++++++
6 files changed, 479 insertions(+)
create mode 100644 drivers/platform/nvidia/Kconfig
create mode 100644 drivers/platform/nvidia/Makefile
create mode 100644 drivers/platform/nvidia/egm.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 15011f5752a9..2edf903a7746 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -19344,6 +19344,11 @@ L: linux-acpi@vger.kernel.org
S: Maintained
F: drivers/acpi/apei/ghes-nvidia.c
+NVIDIA GRACE EGM PLATFORM DRIVER
+M: Ankit Agrawal <ankita@nvidia.com>
+S: Supported
+F: drivers/platform/nvidia/
+
NVIDIA VRS RTC DRIVER
M: Shubhi Garg <shgarg@nvidia.com>
L: linux-tegra@vger.kernel.org
diff --git a/drivers/platform/Kconfig b/drivers/platform/Kconfig
index 312788f249c9..21f2dc901301 100644
--- a/drivers/platform/Kconfig
+++ b/drivers/platform/Kconfig
@@ -22,3 +22,4 @@ source "drivers/platform/arm64/Kconfig"
source "drivers/platform/raspberrypi/Kconfig"
source "drivers/platform/wmi/Kconfig"
+source "drivers/platform/nvidia/Kconfig"
diff --git a/drivers/platform/Makefile b/drivers/platform/Makefile
index fa322e7f8716..96de8fa26ed5 100644
--- a/drivers/platform/Makefile
+++ b/drivers/platform/Makefile
@@ -15,3 +15,4 @@ obj-$(CONFIG_SURFACE_PLATFORMS) += surface/
obj-$(CONFIG_ARM64_PLATFORM_DEVICES) += arm64/
obj-$(CONFIG_BCM2835_VCHIQ) += raspberrypi/
obj-$(CONFIG_ACPI_WMI) += wmi/
+obj-$(CONFIG_NVGRACE_EGM) += nvidia/
diff --git a/drivers/platform/nvidia/Kconfig b/drivers/platform/nvidia/Kconfig
new file mode 100644
index 000000000000..8554a58b1141
--- /dev/null
+++ b/drivers/platform/nvidia/Kconfig
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NVGRACE_EGM
+ tristate "EGM driver for NVIDIA Grace Hopper and Blackwell Superchip"
+ depends on ARM64 || (COMPILE_TEST && 64BIT)
+ help
+ Extended GPU Memory (EGM) support for the GPU in the NVIDIA Grace
+ based chips required to avail the CPU memory as additional
+ cross-node/cross-socket memory for GPU using KVM/qemu.
+
+ If you don't know what to do here, say N.
diff --git a/drivers/platform/nvidia/Makefile b/drivers/platform/nvidia/Makefile
new file mode 100644
index 000000000000..a89f820908fb
--- /dev/null
+++ b/drivers/platform/nvidia/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_NVGRACE_EGM) += nvgrace-egm.o
+nvgrace-egm-y := egm.o
diff --git a/drivers/platform/nvidia/egm.c b/drivers/platform/nvidia/egm.c
new file mode 100644
index 000000000000..a340c4fcbc7a
--- /dev/null
+++ b/drivers/platform/nvidia/egm.c
@@ -0,0 +1,459 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#include <linux/vfio_pci_core.h>
+
+#define NVGRACE_EGM_DEV_NAME "egm"
+#define MAX_EGM_NODES 4
+
+static dev_t dev;
+static struct class *class;
+
+struct gpu_node {
+ struct list_head list;
+ struct pci_dev *pdev;
+};
+
+struct nvgrace_egm_dev {
+ struct device device;
+ struct cdev cdev;
+ phys_addr_t egmphys;
+ size_t egmlength;
+ u64 egmpxm;
+ struct list_head gpus;
+};
+
+struct nvgrace_egm_dev_entry {
+ struct list_head list;
+ struct nvgrace_egm_dev *egm_dev;
+};
+
+/*
+ * Track egm device lists. Note that there is one device per socket.
+ * All the GPUs belonging to the same sockets are associated with
+ * the EGM device for that socket.
+ */
+static LIST_HEAD(egm_chardevs);
+
+static int nvgrace_egm_open(struct inode *inode, struct file *file)
+{
+ return 0;
+}
+
+static int nvgrace_egm_release(struct inode *inode, struct file *file)
+{
+ return 0;
+}
+
+static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ /*
+ * Mapping the EGM region into userspace is implemented by a later
+ * patch. Until then refuse the mmap.
+ */
+ return -EOPNOTSUPP;
+}
+
+static const struct file_operations file_ops = {
+ .owner = THIS_MODULE,
+ .open = nvgrace_egm_open,
+ .release = nvgrace_egm_release,
+ .mmap = nvgrace_egm_mmap,
+};
+
+static int nvgrace_egm_create_gpu_links(struct nvgrace_egm_dev *egm_dev,
+ struct pci_dev *pdev)
+{
+ int ret;
+
+ ret = sysfs_create_link(&egm_dev->device.kobj,
+ &pdev->dev.kobj,
+ dev_name(&pdev->dev));
+
+ if (ret && ret != -EEXIST)
+ return ret;
+
+ return 0;
+}
+
+static void remove_egm_symlinks(struct nvgrace_egm_dev *egm_dev,
+ struct pci_dev *pdev)
+{
+ sysfs_remove_link(&egm_dev->device.kobj, dev_name(&pdev->dev));
+}
+
+static int add_gpu(struct nvgrace_egm_dev *egm_dev, struct pci_dev *pdev)
+{
+ struct gpu_node *node;
+
+ node = kzalloc_obj(*node, GFP_KERNEL);
+ if (!node)
+ return -ENOMEM;
+
+ /*
+ * The pdev is stored for the lifetime of the EGM device but it is
+ * obtained from a for_each_pci_dev() walk that drops its reference as
+ * it advances. Take a reference here so the stored pointer stays valid
+ * even if the GPU is later unbound; remove_gpus() drops it.
+ */
+ node->pdev = pci_dev_get(pdev);
+
+ list_add_tail(&node->list, &egm_dev->gpus);
+
+ return nvgrace_egm_create_gpu_links(egm_dev, pdev);
+}
+
+static void remove_gpus(struct nvgrace_egm_dev *egm_dev)
+{
+ struct gpu_node *node, *tmp;
+
+ list_for_each_entry_safe(node, tmp, &egm_dev->gpus, list) {
+ remove_egm_symlinks(egm_dev, node->pdev);
+ list_del(&node->list);
+ pci_dev_put(node->pdev);
+ kfree(node);
+ }
+}
+
+static void egm_chardev_release(struct device *dev)
+{
+ struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device);
+
+ remove_gpus(egm_chardev);
+ kfree(egm_chardev);
+}
+
+static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,
+ u64 egmpxm)
+{
+ struct nvgrace_egm_dev *egm_chardev;
+ unsigned int baseminor = MINOR(dev);
+ int ret;
+
+ /*
+ * Only MAX_EGM_NODES minors from baseminor were reserved. Reject a PXM
+ * outside that window.
+ */
+ if (egmpxm < baseminor || egmpxm - baseminor >= MAX_EGM_NODES) {
+ pr_err("nvgrace-egm: EGM proximity domain %llu outside reserved minor window [%u, %u)\n",
+ egmpxm, baseminor, baseminor + MAX_EGM_NODES);
+ goto create_err;
+ }
+
+ egm_chardev = kzalloc_obj(*egm_chardev, GFP_KERNEL);
+ if (!egm_chardev)
+ goto create_err;
+
+ device_initialize(&egm_chardev->device);
+
+ /*
+ * Use the proximity domain number as the device minor number.
+ * So the EGM corresponding to node X would be /dev/egmX.
+ */
+ egm_chardev->egmphys = egmphys;
+ egm_chardev->egmlength = egmlength;
+ egm_chardev->egmpxm = egmpxm;
+ INIT_LIST_HEAD(&egm_chardev->gpus);
+
+ egm_chardev->device.devt = MKDEV(MAJOR(dev), egm_chardev->egmpxm);
+ egm_chardev->device.class = class;
+ egm_chardev->device.release = egm_chardev_release;
+ cdev_init(&egm_chardev->cdev, &file_ops);
+ egm_chardev->cdev.owner = THIS_MODULE;
+
+ ret = dev_set_name(&egm_chardev->device, "egm%llu", egm_chardev->egmpxm);
+ if (ret)
+ goto error_exit;
+
+ ret = cdev_device_add(&egm_chardev->cdev, &egm_chardev->device);
+ if (ret)
+ goto error_exit;
+
+ return egm_chardev;
+
+error_exit:
+ put_device(&egm_chardev->device);
+create_err:
+ return NULL;
+}
+
+static void del_egm_chardev(struct nvgrace_egm_dev *egm_chardev)
+{
+ cdev_device_del(&egm_chardev->cdev, &egm_chardev->device);
+ put_device(&egm_chardev->device);
+}
+
+static char *egm_devnode(const struct device *device, umode_t *mode)
+{
+ if (mode)
+ *mode = 0600;
+
+ return NULL;
+}
+
+static ssize_t egm_size_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct nvgrace_egm_dev *egm_dev =
+ container_of(dev, struct nvgrace_egm_dev, device);
+
+ return sysfs_emit(buf, "0x%zx\n", egm_dev->egmlength);
+}
+
+static DEVICE_ATTR_RO(egm_size);
+
+static struct attribute *attrs[] = {
+ &dev_attr_egm_size.attr,
+ NULL,
+};
+
+static const struct attribute_group attr_group = {
+ .attrs = attrs,
+};
+
+static const struct attribute_group *attr_groups[] = {
+ &attr_group,
+ NULL
+};
+
+/*
+ * Determine if the EGM feature is enabled. If disabled, there
+ * will be no EGM properties populated in the ACPI tables and this
+ * fetch would fail.
+ */
+static int has_egm_property(struct pci_dev *pdev, u64 *pegmpxm)
+{
+ return device_property_read_u64(&pdev->dev, "nvidia,egm-pxm",
+ pegmpxm);
+}
+
+static int fetch_egm_property(struct pci_dev *pdev, u64 *pegmphys,
+ u64 *pegmlength)
+{
+ int ret;
+
+ /*
+ * The memory information is present in the system ACPI tables as DSD
+ * properties nvidia,egm-base-pa and nvidia,egm-size.
+ */
+ ret = device_property_read_u64(&pdev->dev, "nvidia,egm-size",
+ pegmlength);
+ if (ret)
+ return ret;
+
+ ret = device_property_read_u64(&pdev->dev, "nvidia,egm-base-pa",
+ pegmphys);
+
+ return ret;
+}
+
+static struct nvgrace_egm_dev *get_egm_dev(u64 egmpxm)
+{
+ struct nvgrace_egm_dev_entry *egm_entry;
+
+ list_for_each_entry(egm_entry, &egm_chardevs, list) {
+ /*
+ * A system could have multiple GPUs associated with an
+ * EGM region and will have the same set of EGM region
+ * information. Return the existing EGM device if already
+ * registered through a different GPU on the same socket.
+ */
+ if (egm_entry->egm_dev->egmpxm == egmpxm)
+ return egm_entry->egm_dev;
+ }
+
+ return NULL;
+}
+
+static void nvgrace_egm_destroy_pci_devs(void)
+{
+ struct nvgrace_egm_dev_entry *entry, *tmp;
+
+ list_for_each_entry_safe(entry, tmp, &egm_chardevs, list) {
+ list_del(&entry->list);
+ del_egm_chardev(entry->egm_dev);
+ kfree(entry);
+ }
+}
+
+/*
+ * Walk all PCI devices and for each one that has a companion ACPI object
+ * advertising EGM properties, create an auxiliary device for that EGM range.
+ * Multiple GPUs on the same socket share a single EGM region (same proximity
+ * domain). So de-duplicate by skipping a pxm that is already registered.
+ */
+static int nvgrace_egm_create_pci_egm_devs(void)
+{
+ struct nvgrace_egm_dev_entry *egm_entry;
+ struct nvgrace_egm_dev *egm_dev;
+ struct pci_dev *pdev = NULL;
+ int ret;
+
+ for_each_pci_dev(pdev) {
+ u64 egmphys, egmlength, egmpxm;
+
+ if (has_egm_property(pdev, &egmpxm))
+ continue;
+
+ ret = fetch_egm_property(pdev, &egmphys, &egmlength);
+ if (ret)
+ continue;
+
+ egm_dev = get_egm_dev(egmpxm);
+ if (egm_dev) {
+ ret = add_gpu(egm_dev, pdev);
+ if (ret)
+ goto free_dev;
+ continue;
+ }
+
+ egm_entry = kzalloc_obj(*egm_entry, GFP_KERNEL);
+ if (!egm_entry) {
+ pci_dev_put(pdev);
+ return -ENOMEM;
+ }
+
+ egm_dev = setup_egm_chardev(egmphys, egmlength, egmpxm);
+ if (!egm_dev) {
+ kfree(egm_entry);
+ pci_dev_put(pdev);
+ return -EINVAL;
+ }
+
+ egm_entry->egm_dev = egm_dev;
+
+ ret = add_gpu(egm_entry->egm_dev, pdev);
+ if (ret) {
+ del_egm_chardev(egm_dev);
+ kfree(egm_entry);
+ goto free_dev;
+ }
+
+ list_add_tail(&egm_entry->list, &egm_chardevs);
+ }
+
+ return 0;
+
+free_dev:
+ /*
+ * Reached only via goto from inside the loop where the pdev still
+ * holds the reference from the last pci_get_device(). So drop it here.
+ */
+ pci_dev_put(pdev);
+ nvgrace_egm_destroy_pci_devs();
+ return ret;
+}
+
+/*
+ * The char device minor is the EGM proximity domain number which is sparse
+ * and need not start at 0. Find the lowest proximity domain among the EGM
+ * devices so the minor range can be reserved starting there. Returns -ENODEV
+ * if the system has no EGM devices.
+ *
+ * EGM PXMs are assumed contiguous within [lowest, lowest + MAX_EGM_NODES),
+ * which holds for the supported Grace configurations.
+ */
+static int nvgrace_egm_lowest_pxm(u64 *pmin_pxm)
+{
+ struct pci_dev *pdev = NULL;
+ bool found = false;
+
+ for_each_pci_dev(pdev) {
+ u64 egmphys, egmlength, egmpxm;
+
+ if (has_egm_property(pdev, &egmpxm))
+ continue;
+
+ if (fetch_egm_property(pdev, &egmphys, &egmlength))
+ continue;
+
+ if (!found || egmpxm < *pmin_pxm) {
+ *pmin_pxm = egmpxm;
+ found = true;
+ }
+ }
+
+ if (!found)
+ return -ENODEV;
+
+ /*
+ * The PXM is used verbatim as the minor (only MINORBITS wide). Validate
+ * the window would not overflow MINORMASK and be truncated.
+ */
+ if (*pmin_pxm + (MAX_EGM_NODES - 1) > MINORMASK) {
+ pr_err("nvgrace-egm: lowest EGM proximity domain %llu\n"
+ "exceeds the minor range\n", *pmin_pxm);
+ return -ERANGE;
+ }
+
+ return 0;
+}
+
+static int __init nvgrace_egm_init(void)
+{
+ u64 min_pxm;
+ int ret;
+
+ /*
+ * Each EGM region is exposed as /dev/egmX with the proximity domain X
+ * as its minor number. Reserve MAX_EGM_NODES minors starting at
+ * the lowest proximity domain so that every egmX device falls within
+ * the reserved range. With no EGM devices present, load without
+ * reserving any resources.
+ */
+ ret = nvgrace_egm_lowest_pxm(&min_pxm);
+ if (ret == -ENODEV)
+ return 0;
+ else if (ret)
+ return ret;
+
+ ret = alloc_chrdev_region(&dev, min_pxm, MAX_EGM_NODES,
+ NVGRACE_EGM_DEV_NAME);
+ if (ret < 0)
+ return ret;
+
+ class = class_create(NVGRACE_EGM_DEV_NAME);
+ if (IS_ERR(class)) {
+ unregister_chrdev_region(dev, MAX_EGM_NODES);
+ return PTR_ERR(class);
+ }
+
+ class->devnode = egm_devnode;
+ class->dev_groups = attr_groups;
+
+ ret = nvgrace_egm_create_pci_egm_devs();
+ if (ret)
+ goto cleanup_pci_devs;
+
+ return 0;
+
+cleanup_pci_devs:
+ nvgrace_egm_destroy_pci_devs();
+ class_destroy(class);
+ unregister_chrdev_region(dev, MAX_EGM_NODES);
+
+ return ret;
+}
+
+static void __exit nvgrace_egm_cleanup(void)
+{
+ /*
+ * When no EGM devices were present, the module loaded inertly and
+ * reserved nothing. @class stays NULL and there is nothing to undo.
+ */
+ if (!class)
+ return;
+
+ nvgrace_egm_destroy_pci_devs();
+ class_destroy(class);
+ unregister_chrdev_region(dev, MAX_EGM_NODES);
+}
+
+module_init(nvgrace_egm_init);
+module_exit(nvgrace_egm_cleanup);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Ankit Agrawal <ankita@nvidia.com>");
+MODULE_DESCRIPTION("NVGRACE EGM - Module to support Extended GPU Memory on NVIDIA Grace Based systems");
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
2026-07-02 19:25 ` [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions Ankit Agrawal
@ 2026-07-02 22:22 ` Armin Wolf
2026-07-03 13:59 ` Jason Gunthorpe
0 siblings, 1 reply; 7+ messages in thread
From: Armin Wolf @ 2026-07-02 22:22 UTC (permalink / raw)
To: Ankit Agrawal, gregkh, jai.luthra, ilpo.jarvinen
Cc: alex, jgg, vsethi, mhonap, mochs, clg, linux-kernel
Am 02.07.26 um 21:25 schrieb Ankit Agrawal:
> The Extended GPU Memory (EGM) feature enables the GPU to access system
> memory allocations within and across nodes through a high-bandwidth path
> on Grace-based systems. The GPU can utilize system memory on the same
> socket, a different socket or a different node in a multi-node setup.
>
> When EGM mode is enabled through SBIOS, host system memory is partitioned
> into two parts: a Hypervisor region for the host OS and a
> Hypervisor-Invisible (HI) region for the VM. Only the hypervisor region
> is part of the host EFI map and visible to the host OS on bootup. The
> HI partition is interchangeably called the EGM region. Its base SPA and
> size are exposed through ACPI DSDT properties.
>
> Whilst the EGM region is accessible on the host, it is not
> added/hot-plugged to the kernel. The HI region is assigned to a VM
> by mapping the QEMU VMA to the SPA using remap_pfn_range(). So
> it looks like the following:
>
> |---- Sysmem ----| VM view
> | |
> |IPA <-> SPA map |
> | |
> |--- HI / EGM ---|-- Host Mem --| Host view
>
> Introduce the nvgrace-egm platform driver module and implement full EGM
> region discovery, char device exposure and sysfs metadata.
>
> Build infrastructure:
> - Kconfig, Makefile entries, MAINTAINERS entry.
> - An 'egm' device class and a range of char device minor numbers.
> MAX_EGM_NODES is set to 4 to matches the largest 4-socket Grace
> configuration.
>
> Discovery:
> - Walk all PCI devices at module init time. For each one advertising
> "nvidia,egm-pxm", read the EGM region properties (base SPA, size and
> PXM domain) from the ACPI DSD table via the GPU device handle, using
> "nvidia,egm-base-pa" and "nvidia,egm-size" properties.
> - Maintain a module-level list (egm_chardevs) of discovered regions,
> de-duplicating by PXM so each socket gets exactly one EGM device
> regardless of how many GPUs it has (2 GPUs per EGM device in GB200).
I assume that the PCI device itself is being handled by the GPU driver, right?
In this case i suggest that you do the discovery inside the GPU driver during probing,
otherwise you might run intro problems once PCI hotplug comes into play.
Thanks,
Armin Wolf
>
> Char device creation:
> - Each EGM region is exposed as /dev/egmX, where X is the PXM number.
> Embed struct device and struct cdev inside nvgrace_egm_dev so the
> device lifecycle follows the standard kernel device model.
> - Implement setup_egm_chardev() and del_egm_chardev() for setup and
> teardown. The devnode callback sets device permissions to 0600.
> - Stub file operations (open, release and mmap) are registered.
>
> GPU tracking and sysfs:
> - Track which PCI devices are associated with each EGM region in a
> per-device GPU list (struct gpu_node).
> - Expose sysfs symlinks from each EGM device to its associated GPU PCI
> devices to allow the userspace to determine GPU-to-socket affinity and
> replicate the host EGM topology in the VM. Helps admin understand
> GPU association.
> - Expose the EGM region size via an egm_size sysfs attribute so QEMU
> can determine the memory-backend-file size. On a 2-socket Grace
> Blackwell setup this appears as:
> Socket0: /sys/devices/virtual/egm/egm4/egm_size
> Socket1: /sys/devices/virtual/egm/egm5/egm_size
>
> Introduce nvgrace_egm_create_pci_egm_devs() and its counterpart
> nvgrace_egm_destroy_pci_devs() that are called from module init/exit.
>
> Suggested-by: Alex Williamson <alex@shazbot.org>
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Suggested-by: Aniket Agashe <aniketa@nvidia.com>
> Suggested-by: Matthew R. Ochs <mochs@nvidia.com>
> Assisted-by: Claude:claude-opus-4.8
> Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
> ---
> MAINTAINERS | 5 +
> drivers/platform/Kconfig | 1 +
> drivers/platform/Makefile | 1 +
> drivers/platform/nvidia/Kconfig | 10 +
> drivers/platform/nvidia/Makefile | 3 +
> drivers/platform/nvidia/egm.c | 459 +++++++++++++++++++++++++++++++
> 6 files changed, 479 insertions(+)
> create mode 100644 drivers/platform/nvidia/Kconfig
> create mode 100644 drivers/platform/nvidia/Makefile
> create mode 100644 drivers/platform/nvidia/egm.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 15011f5752a9..2edf903a7746 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -19344,6 +19344,11 @@ L: linux-acpi@vger.kernel.org
> S: Maintained
> F: drivers/acpi/apei/ghes-nvidia.c
>
> +NVIDIA GRACE EGM PLATFORM DRIVER
> +M: Ankit Agrawal <ankita@nvidia.com>
> +S: Supported
> +F: drivers/platform/nvidia/
> +
> NVIDIA VRS RTC DRIVER
> M: Shubhi Garg <shgarg@nvidia.com>
> L: linux-tegra@vger.kernel.org
> diff --git a/drivers/platform/Kconfig b/drivers/platform/Kconfig
> index 312788f249c9..21f2dc901301 100644
> --- a/drivers/platform/Kconfig
> +++ b/drivers/platform/Kconfig
> @@ -22,3 +22,4 @@ source "drivers/platform/arm64/Kconfig"
> source "drivers/platform/raspberrypi/Kconfig"
>
> source "drivers/platform/wmi/Kconfig"
> +source "drivers/platform/nvidia/Kconfig"
> diff --git a/drivers/platform/Makefile b/drivers/platform/Makefile
> index fa322e7f8716..96de8fa26ed5 100644
> --- a/drivers/platform/Makefile
> +++ b/drivers/platform/Makefile
> @@ -15,3 +15,4 @@ obj-$(CONFIG_SURFACE_PLATFORMS) += surface/
> obj-$(CONFIG_ARM64_PLATFORM_DEVICES) += arm64/
> obj-$(CONFIG_BCM2835_VCHIQ) += raspberrypi/
> obj-$(CONFIG_ACPI_WMI) += wmi/
> +obj-$(CONFIG_NVGRACE_EGM) += nvidia/
> diff --git a/drivers/platform/nvidia/Kconfig b/drivers/platform/nvidia/Kconfig
> new file mode 100644
> index 000000000000..8554a58b1141
> --- /dev/null
> +++ b/drivers/platform/nvidia/Kconfig
> @@ -0,0 +1,10 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config NVGRACE_EGM
> + tristate "EGM driver for NVIDIA Grace Hopper and Blackwell Superchip"
> + depends on ARM64 || (COMPILE_TEST && 64BIT)
> + help
> + Extended GPU Memory (EGM) support for the GPU in the NVIDIA Grace
> + based chips required to avail the CPU memory as additional
> + cross-node/cross-socket memory for GPU using KVM/qemu.
> +
> + If you don't know what to do here, say N.
> diff --git a/drivers/platform/nvidia/Makefile b/drivers/platform/nvidia/Makefile
> new file mode 100644
> index 000000000000..a89f820908fb
> --- /dev/null
> +++ b/drivers/platform/nvidia/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +obj-$(CONFIG_NVGRACE_EGM) += nvgrace-egm.o
> +nvgrace-egm-y := egm.o
> diff --git a/drivers/platform/nvidia/egm.c b/drivers/platform/nvidia/egm.c
> new file mode 100644
> index 000000000000..a340c4fcbc7a
> --- /dev/null
> +++ b/drivers/platform/nvidia/egm.c
> @@ -0,0 +1,459 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + */
> +
> +#include <linux/vfio_pci_core.h>
> +
> +#define NVGRACE_EGM_DEV_NAME "egm"
> +#define MAX_EGM_NODES 4
> +
> +static dev_t dev;
> +static struct class *class;
> +
> +struct gpu_node {
> + struct list_head list;
> + struct pci_dev *pdev;
> +};
> +
> +struct nvgrace_egm_dev {
> + struct device device;
> + struct cdev cdev;
> + phys_addr_t egmphys;
> + size_t egmlength;
> + u64 egmpxm;
> + struct list_head gpus;
> +};
> +
> +struct nvgrace_egm_dev_entry {
> + struct list_head list;
> + struct nvgrace_egm_dev *egm_dev;
> +};
> +
> +/*
> + * Track egm device lists. Note that there is one device per socket.
> + * All the GPUs belonging to the same sockets are associated with
> + * the EGM device for that socket.
> + */
> +static LIST_HEAD(egm_chardevs);
> +
> +static int nvgrace_egm_open(struct inode *inode, struct file *file)
> +{
> + return 0;
> +}
> +
> +static int nvgrace_egm_release(struct inode *inode, struct file *file)
> +{
> + return 0;
> +}
> +
> +static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma)
> +{
> + /*
> + * Mapping the EGM region into userspace is implemented by a later
> + * patch. Until then refuse the mmap.
> + */
> + return -EOPNOTSUPP;
> +}
> +
> +static const struct file_operations file_ops = {
> + .owner = THIS_MODULE,
> + .open = nvgrace_egm_open,
> + .release = nvgrace_egm_release,
> + .mmap = nvgrace_egm_mmap,
> +};
> +
> +static int nvgrace_egm_create_gpu_links(struct nvgrace_egm_dev *egm_dev,
> + struct pci_dev *pdev)
> +{
> + int ret;
> +
> + ret = sysfs_create_link(&egm_dev->device.kobj,
> + &pdev->dev.kobj,
> + dev_name(&pdev->dev));
> +
> + if (ret && ret != -EEXIST)
> + return ret;
> +
> + return 0;
> +}
> +
> +static void remove_egm_symlinks(struct nvgrace_egm_dev *egm_dev,
> + struct pci_dev *pdev)
> +{
> + sysfs_remove_link(&egm_dev->device.kobj, dev_name(&pdev->dev));
> +}
> +
> +static int add_gpu(struct nvgrace_egm_dev *egm_dev, struct pci_dev *pdev)
> +{
> + struct gpu_node *node;
> +
> + node = kzalloc_obj(*node, GFP_KERNEL);
> + if (!node)
> + return -ENOMEM;
> +
> + /*
> + * The pdev is stored for the lifetime of the EGM device but it is
> + * obtained from a for_each_pci_dev() walk that drops its reference as
> + * it advances. Take a reference here so the stored pointer stays valid
> + * even if the GPU is later unbound; remove_gpus() drops it.
> + */
> + node->pdev = pci_dev_get(pdev);
> +
> + list_add_tail(&node->list, &egm_dev->gpus);
> +
> + return nvgrace_egm_create_gpu_links(egm_dev, pdev);
> +}
> +
> +static void remove_gpus(struct nvgrace_egm_dev *egm_dev)
> +{
> + struct gpu_node *node, *tmp;
> +
> + list_for_each_entry_safe(node, tmp, &egm_dev->gpus, list) {
> + remove_egm_symlinks(egm_dev, node->pdev);
> + list_del(&node->list);
> + pci_dev_put(node->pdev);
> + kfree(node);
> + }
> +}
> +
> +static void egm_chardev_release(struct device *dev)
> +{
> + struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device);
> +
> + remove_gpus(egm_chardev);
> + kfree(egm_chardev);
> +}
> +
> +static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,
> + u64 egmpxm)
> +{
> + struct nvgrace_egm_dev *egm_chardev;
> + unsigned int baseminor = MINOR(dev);
> + int ret;
> +
> + /*
> + * Only MAX_EGM_NODES minors from baseminor were reserved. Reject a PXM
> + * outside that window.
> + */
> + if (egmpxm < baseminor || egmpxm - baseminor >= MAX_EGM_NODES) {
> + pr_err("nvgrace-egm: EGM proximity domain %llu outside reserved minor window [%u, %u)\n",
> + egmpxm, baseminor, baseminor + MAX_EGM_NODES);
> + goto create_err;
> + }
> +
> + egm_chardev = kzalloc_obj(*egm_chardev, GFP_KERNEL);
> + if (!egm_chardev)
> + goto create_err;
> +
> + device_initialize(&egm_chardev->device);
> +
> + /*
> + * Use the proximity domain number as the device minor number.
> + * So the EGM corresponding to node X would be /dev/egmX.
> + */
> + egm_chardev->egmphys = egmphys;
> + egm_chardev->egmlength = egmlength;
> + egm_chardev->egmpxm = egmpxm;
> + INIT_LIST_HEAD(&egm_chardev->gpus);
> +
> + egm_chardev->device.devt = MKDEV(MAJOR(dev), egm_chardev->egmpxm);
> + egm_chardev->device.class = class;
> + egm_chardev->device.release = egm_chardev_release;
> + cdev_init(&egm_chardev->cdev, &file_ops);
> + egm_chardev->cdev.owner = THIS_MODULE;
> +
> + ret = dev_set_name(&egm_chardev->device, "egm%llu", egm_chardev->egmpxm);
> + if (ret)
> + goto error_exit;
> +
> + ret = cdev_device_add(&egm_chardev->cdev, &egm_chardev->device);
> + if (ret)
> + goto error_exit;
> +
> + return egm_chardev;
> +
> +error_exit:
> + put_device(&egm_chardev->device);
> +create_err:
> + return NULL;
> +}
> +
> +static void del_egm_chardev(struct nvgrace_egm_dev *egm_chardev)
> +{
> + cdev_device_del(&egm_chardev->cdev, &egm_chardev->device);
> + put_device(&egm_chardev->device);
> +}
> +
> +static char *egm_devnode(const struct device *device, umode_t *mode)
> +{
> + if (mode)
> + *mode = 0600;
> +
> + return NULL;
> +}
> +
> +static ssize_t egm_size_show(struct device *dev, struct device_attribute *attr,
> + char *buf)
> +{
> + struct nvgrace_egm_dev *egm_dev =
> + container_of(dev, struct nvgrace_egm_dev, device);
> +
> + return sysfs_emit(buf, "0x%zx\n", egm_dev->egmlength);
> +}
> +
> +static DEVICE_ATTR_RO(egm_size);
> +
> +static struct attribute *attrs[] = {
> + &dev_attr_egm_size.attr,
> + NULL,
> +};
> +
> +static const struct attribute_group attr_group = {
> + .attrs = attrs,
> +};
> +
> +static const struct attribute_group *attr_groups[] = {
> + &attr_group,
> + NULL
> +};
> +
> +/*
> + * Determine if the EGM feature is enabled. If disabled, there
> + * will be no EGM properties populated in the ACPI tables and this
> + * fetch would fail.
> + */
> +static int has_egm_property(struct pci_dev *pdev, u64 *pegmpxm)
> +{
> + return device_property_read_u64(&pdev->dev, "nvidia,egm-pxm",
> + pegmpxm);
> +}
> +
> +static int fetch_egm_property(struct pci_dev *pdev, u64 *pegmphys,
> + u64 *pegmlength)
> +{
> + int ret;
> +
> + /*
> + * The memory information is present in the system ACPI tables as DSD
> + * properties nvidia,egm-base-pa and nvidia,egm-size.
> + */
> + ret = device_property_read_u64(&pdev->dev, "nvidia,egm-size",
> + pegmlength);
> + if (ret)
> + return ret;
> +
> + ret = device_property_read_u64(&pdev->dev, "nvidia,egm-base-pa",
> + pegmphys);
> +
> + return ret;
> +}
> +
> +static struct nvgrace_egm_dev *get_egm_dev(u64 egmpxm)
> +{
> + struct nvgrace_egm_dev_entry *egm_entry;
> +
> + list_for_each_entry(egm_entry, &egm_chardevs, list) {
> + /*
> + * A system could have multiple GPUs associated with an
> + * EGM region and will have the same set of EGM region
> + * information. Return the existing EGM device if already
> + * registered through a different GPU on the same socket.
> + */
> + if (egm_entry->egm_dev->egmpxm == egmpxm)
> + return egm_entry->egm_dev;
> + }
> +
> + return NULL;
> +}
> +
> +static void nvgrace_egm_destroy_pci_devs(void)
> +{
> + struct nvgrace_egm_dev_entry *entry, *tmp;
> +
> + list_for_each_entry_safe(entry, tmp, &egm_chardevs, list) {
> + list_del(&entry->list);
> + del_egm_chardev(entry->egm_dev);
> + kfree(entry);
> + }
> +}
> +
> +/*
> + * Walk all PCI devices and for each one that has a companion ACPI object
> + * advertising EGM properties, create an auxiliary device for that EGM range.
> + * Multiple GPUs on the same socket share a single EGM region (same proximity
> + * domain). So de-duplicate by skipping a pxm that is already registered.
> + */
> +static int nvgrace_egm_create_pci_egm_devs(void)
> +{
> + struct nvgrace_egm_dev_entry *egm_entry;
> + struct nvgrace_egm_dev *egm_dev;
> + struct pci_dev *pdev = NULL;
> + int ret;
> +
> + for_each_pci_dev(pdev) {
> + u64 egmphys, egmlength, egmpxm;
> +
> + if (has_egm_property(pdev, &egmpxm))
> + continue;
> +
> + ret = fetch_egm_property(pdev, &egmphys, &egmlength);
> + if (ret)
> + continue;
> +
> + egm_dev = get_egm_dev(egmpxm);
> + if (egm_dev) {
> + ret = add_gpu(egm_dev, pdev);
> + if (ret)
> + goto free_dev;
> + continue;
> + }
> +
> + egm_entry = kzalloc_obj(*egm_entry, GFP_KERNEL);
> + if (!egm_entry) {
> + pci_dev_put(pdev);
> + return -ENOMEM;
> + }
> +
> + egm_dev = setup_egm_chardev(egmphys, egmlength, egmpxm);
> + if (!egm_dev) {
> + kfree(egm_entry);
> + pci_dev_put(pdev);
> + return -EINVAL;
> + }
> +
> + egm_entry->egm_dev = egm_dev;
> +
> + ret = add_gpu(egm_entry->egm_dev, pdev);
> + if (ret) {
> + del_egm_chardev(egm_dev);
> + kfree(egm_entry);
> + goto free_dev;
> + }
> +
> + list_add_tail(&egm_entry->list, &egm_chardevs);
> + }
> +
> + return 0;
> +
> +free_dev:
> + /*
> + * Reached only via goto from inside the loop where the pdev still
> + * holds the reference from the last pci_get_device(). So drop it here.
> + */
> + pci_dev_put(pdev);
> + nvgrace_egm_destroy_pci_devs();
> + return ret;
> +}
> +
> +/*
> + * The char device minor is the EGM proximity domain number which is sparse
> + * and need not start at 0. Find the lowest proximity domain among the EGM
> + * devices so the minor range can be reserved starting there. Returns -ENODEV
> + * if the system has no EGM devices.
> + *
> + * EGM PXMs are assumed contiguous within [lowest, lowest + MAX_EGM_NODES),
> + * which holds for the supported Grace configurations.
> + */
> +static int nvgrace_egm_lowest_pxm(u64 *pmin_pxm)
> +{
> + struct pci_dev *pdev = NULL;
> + bool found = false;
> +
> + for_each_pci_dev(pdev) {
> + u64 egmphys, egmlength, egmpxm;
> +
> + if (has_egm_property(pdev, &egmpxm))
> + continue;
> +
> + if (fetch_egm_property(pdev, &egmphys, &egmlength))
> + continue;
> +
> + if (!found || egmpxm < *pmin_pxm) {
> + *pmin_pxm = egmpxm;
> + found = true;
> + }
> + }
> +
> + if (!found)
> + return -ENODEV;
> +
> + /*
> + * The PXM is used verbatim as the minor (only MINORBITS wide). Validate
> + * the window would not overflow MINORMASK and be truncated.
> + */
> + if (*pmin_pxm + (MAX_EGM_NODES - 1) > MINORMASK) {
> + pr_err("nvgrace-egm: lowest EGM proximity domain %llu\n"
> + "exceeds the minor range\n", *pmin_pxm);
> + return -ERANGE;
> + }
> +
> + return 0;
> +}
> +
> +static int __init nvgrace_egm_init(void)
> +{
> + u64 min_pxm;
> + int ret;
> +
> + /*
> + * Each EGM region is exposed as /dev/egmX with the proximity domain X
> + * as its minor number. Reserve MAX_EGM_NODES minors starting at
> + * the lowest proximity domain so that every egmX device falls within
> + * the reserved range. With no EGM devices present, load without
> + * reserving any resources.
> + */
> + ret = nvgrace_egm_lowest_pxm(&min_pxm);
> + if (ret == -ENODEV)
> + return 0;
> + else if (ret)
> + return ret;
> +
> + ret = alloc_chrdev_region(&dev, min_pxm, MAX_EGM_NODES,
> + NVGRACE_EGM_DEV_NAME);
> + if (ret < 0)
> + return ret;
> +
> + class = class_create(NVGRACE_EGM_DEV_NAME);
> + if (IS_ERR(class)) {
> + unregister_chrdev_region(dev, MAX_EGM_NODES);
> + return PTR_ERR(class);
> + }
> +
> + class->devnode = egm_devnode;
> + class->dev_groups = attr_groups;
> +
> + ret = nvgrace_egm_create_pci_egm_devs();
> + if (ret)
> + goto cleanup_pci_devs;
> +
> + return 0;
> +
> +cleanup_pci_devs:
> + nvgrace_egm_destroy_pci_devs();
> + class_destroy(class);
> + unregister_chrdev_region(dev, MAX_EGM_NODES);
> +
> + return ret;
> +}
> +
> +static void __exit nvgrace_egm_cleanup(void)
> +{
> + /*
> + * When no EGM devices were present, the module loaded inertly and
> + * reserved nothing. @class stays NULL and there is nothing to undo.
> + */
> + if (!class)
> + return;
> +
> + nvgrace_egm_destroy_pci_devs();
> + class_destroy(class);
> + unregister_chrdev_region(dev, MAX_EGM_NODES);
> +}
> +
> +module_init(nvgrace_egm_init);
> +module_exit(nvgrace_egm_cleanup);
> +
> +MODULE_LICENSE("GPL");
> +MODULE_AUTHOR("Ankit Agrawal <ankita@nvidia.com>");
> +MODULE_DESCRIPTION("NVGRACE EGM - Module to support Extended GPU Memory on NVIDIA Grace Based systems");
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
2026-07-02 22:22 ` Armin Wolf
@ 2026-07-03 13:59 ` Jason Gunthorpe
0 siblings, 0 replies; 7+ messages in thread
From: Jason Gunthorpe @ 2026-07-03 13:59 UTC (permalink / raw)
To: Armin Wolf
Cc: Ankit Agrawal, gregkh, jai.luthra, ilpo.jarvinen, alex, vsethi,
mhonap, mochs, clg, linux-kernel
On Fri, Jul 03, 2026 at 12:22:21AM +0200, Armin Wolf wrote:
> > - Walk all PCI devices at module init time. For each one advertising
> > "nvidia,egm-pxm", read the EGM region properties (base SPA, size and
> > PXM domain) from the ACPI DSD table via the GPU device handle, using
> > "nvidia,egm-base-pa" and "nvidia,egm-size" properties.
> > - Maintain a module-level list (egm_chardevs) of discovered regions,
> > de-duplicating by PXM so each socket gets exactly one EGM device
> > regardless of how many GPUs it has (2 GPUs per EGM device in GB200).
>
> I assume that the PCI device itself is being handled by the GPU driver, right?
> In this case i suggest that you do the discovery inside the GPU driver during probing,
> otherwise you might run intro problems once PCI hotplug comes into play.
IMHO the ACPI modeling placing the egm-base-pa under the PCI device is
not really very good (but we are stuck with it). The EGM is an
entirely distinct object from the PCI device. The ACPI is just saying
that the PCI device has efficient access to the EGM.
It wouldn't make any sense to involve the GPU driver since the GPU
driver doesn't even run on the hypervisor.
Jason
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/4] platform/nvidia: Implement mmap and memory scrubbing for EGM chardev
2026-07-02 19:25 [PATCH 0/4] Introduce nvgrace-egm driver for Extended GPU Memory Ankit Agrawal
2026-07-02 19:25 ` [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions Ankit Agrawal
@ 2026-07-02 19:25 ` Ankit Agrawal
2026-07-02 19:25 ` [PATCH 3/4] platform/nvidia: Handle retired ECC pages and expose via ioctl Ankit Agrawal
2026-07-02 19:25 ` [PATCH 4/4] platform/nvidia: Register EGM PFNMAP range with memory_failure Ankit Agrawal
3 siblings, 0 replies; 7+ messages in thread
From: Ankit Agrawal @ 2026-07-02 19:25 UTC (permalink / raw)
To: gregkh, jai.luthra, ilpo.jarvinen
Cc: W_Armin, alex, jgg, vsethi, mhonap, mochs, ankita, clg,
linux-kernel
Setup the file operations (open, close and mmap) for the EGM char device.
Memory scrubbing:
- The EGM region is invisible to the host Linux kernel and is not
managed by the page allocator. So the driver is responsible for
zeroing it before handing it to a VM.
- Clear the entire EGM region on the first open() call using memremap()
and memset() in 1 GiB chunks with cond_resched() between iterations
to avoid RCU stalls on large regions.
- The region is handed to a single VM at a time and must be scrubbed
before each handover. Serialise openers with a mutex guarding open_count
and refuse a second opener with -EBUSY while the region is still held.
mmap:
- Implement nvgrace_egm_mmap() using remap_pfn_range(). The EGM region
is not part of the host EFI map and has no struct pages making
remap_pfn_range() the correct interface for mapping it directly into
QEMU's virtual address space.
- Validate that the requested range fits within the EGM region. Reject a
vm_pgoff that already lies outside the region before it is shifted into
a byte count so that the page offset cannot overflow and slip past the
range check.
Suggested-by: Aniket Agashe <aniketa@nvidia.com>
Suggested-by: Vikram Sethi <vsethi@nvidia.com>
Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
---
drivers/platform/nvidia/egm.c | 125 ++++++++++++++++++++++++++++++++--
1 file changed, 121 insertions(+), 4 deletions(-)
diff --git a/drivers/platform/nvidia/egm.c b/drivers/platform/nvidia/egm.c
index a340c4fcbc7a..09ab00213de2 100644
--- a/drivers/platform/nvidia/egm.c
+++ b/drivers/platform/nvidia/egm.c
@@ -3,6 +3,9 @@
* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved
*/
+#include <linux/mutex.h>
+#include <linux/sched/signal.h>
+#include <linux/sizes.h>
#include <linux/vfio_pci_core.h>
#define NVGRACE_EGM_DEV_NAME "egm"
@@ -19,6 +22,10 @@ struct gpu_node {
struct nvgrace_egm_dev {
struct device device;
struct cdev cdev;
+ /* serialises the first-open scrub */
+ struct mutex open_lock;
+ /* protected by open_lock */
+ unsigned int open_count;
phys_addr_t egmphys;
size_t egmlength;
u64 egmpxm;
@@ -39,21 +46,129 @@ static LIST_HEAD(egm_chardevs);
static int nvgrace_egm_open(struct inode *inode, struct file *file)
{
- return 0;
+ struct nvgrace_egm_dev *egm_dev =
+ container_of(inode->i_cdev, struct nvgrace_egm_dev, cdev);
+ phys_addr_t phys;
+ size_t remaining, chunk_size;
+ void *chunk_addr;
+ int ret;
+
+ file->private_data = egm_dev;
+
+ /*
+ * The EGM region is a physical carveout handed to a single VM at a
+ * time and must be scrubbed before each handover. Take open_lock
+ * killably around the scrub which can run for long time, so that
+ * the waiters stay killable.
+ */
+ if (mutex_lock_killable(&egm_dev->open_lock))
+ return -EINTR;
+
+ /*
+ * Refuse a second opener while the region is still held. release()
+ * runs only when the last reference to the struct file drops but an
+ * mmap keeps that reference (and hence open_count) alive past
+ * close for the lifetime of the VMA. So open_count returns to zero
+ * only when every mapping is gone. A concurrent opener cannot be
+ * handed the region: it cannot be re-scrubbed without destroying the
+ * current consumer's live data.
+ */
+ if (egm_dev->open_count) {
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ /*
+ * nvgrace-egm module is responsible to manage the EGM memory as
+ * the host kernel has no knowledge of it. Clear the region before
+ * handing over to userspace.
+ *
+ * The EGM region can be very large (hundreds of GiB). So Map and zero
+ * one chunk at a time rather than mapping the whole region at once.
+ */
+ phys = egm_dev->egmphys;
+ remaining = egm_dev->egmlength;
+
+ while (remaining > 0) {
+ /*
+ * The scrub holds the lock for a long time; let it be
+ * SIGKILL'd.
+ */
+ if (fatal_signal_pending(current)) {
+ ret = -EINTR;
+ goto unlock;
+ }
+
+ chunk_size = min(remaining, SZ_1G);
+
+ chunk_addr = memremap(phys, chunk_size, MEMREMAP_WB);
+ if (!chunk_addr) {
+ ret = -ENOMEM;
+ goto unlock;
+ }
+
+ memset(chunk_addr, 0, chunk_size);
+ memunmap(chunk_addr);
+ cond_resched();
+
+ phys += chunk_size;
+ remaining -= chunk_size;
+ }
+
+ /*
+ * Mark the device open only after the scrub completes so that a
+ * concurrent opener cannot observe a non-zero count and proceed
+ * before the region has been cleared.
+ */
+ egm_dev->open_count = 1;
+ ret = 0;
+
+unlock:
+ mutex_unlock(&egm_dev->open_lock);
+ return ret;
}
static int nvgrace_egm_release(struct inode *inode, struct file *file)
{
+ struct nvgrace_egm_dev *egm_dev =
+ container_of(inode->i_cdev, struct nvgrace_egm_dev, cdev);
+
+ guard(mutex)(&egm_dev->open_lock);
+
+ if (!--egm_dev->open_count)
+ file->private_data = NULL;
+
return 0;
}
static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma)
{
+ struct nvgrace_egm_dev *egm_dev = file->private_data;
+ u64 req_len, pgoff, end;
+ unsigned long start_pfn, num_pages;
+
+ pgoff = vma->vm_pgoff;
+ num_pages = egm_dev->egmlength >> PAGE_SHIFT;
+
+ /* Reject a page offset that already lies outside the EGM region. */
+ if (pgoff >= num_pages)
+ return -EINVAL;
+
+ if (check_sub_overflow(vma->vm_end, vma->vm_start, &req_len) ||
+ check_add_overflow(PHYS_PFN(egm_dev->egmphys), pgoff, &start_pfn) ||
+ check_add_overflow(PFN_PHYS(pgoff), req_len, &end))
+ return -EOVERFLOW;
+
+ if (end > egm_dev->egmlength)
+ return -EINVAL;
+
/*
- * Mapping the EGM region into userspace is implemented by a later
- * patch. Until then refuse the mmap.
+ * EGM memory is invisible to the host kernel and is not managed
+ * by it. Map the usermode VMA to the EGM region.
*/
- return -EOPNOTSUPP;
+ return remap_pfn_range(vma, vma->vm_start,
+ start_pfn, req_len,
+ vma->vm_page_prot);
}
static const struct file_operations file_ops = {
@@ -122,6 +237,7 @@ static void egm_chardev_release(struct device *dev)
struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device);
remove_gpus(egm_chardev);
+ mutex_destroy(&egm_chardev->open_lock);
kfree(egm_chardev);
}
@@ -155,6 +271,7 @@ static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,
egm_chardev->egmphys = egmphys;
egm_chardev->egmlength = egmlength;
egm_chardev->egmpxm = egmpxm;
+ mutex_init(&egm_chardev->open_lock);
INIT_LIST_HEAD(&egm_chardev->gpus);
egm_chardev->device.devt = MKDEV(MAJOR(dev), egm_chardev->egmpxm);
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3/4] platform/nvidia: Handle retired ECC pages and expose via ioctl
2026-07-02 19:25 [PATCH 0/4] Introduce nvgrace-egm driver for Extended GPU Memory Ankit Agrawal
2026-07-02 19:25 ` [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions Ankit Agrawal
2026-07-02 19:25 ` [PATCH 2/4] platform/nvidia: Implement mmap and memory scrubbing for EGM chardev Ankit Agrawal
@ 2026-07-02 19:25 ` Ankit Agrawal
2026-07-02 19:25 ` [PATCH 4/4] platform/nvidia: Register EGM PFNMAP range with memory_failure Ankit Agrawal
3 siblings, 0 replies; 7+ messages in thread
From: Ankit Agrawal @ 2026-07-02 19:25 UTC (permalink / raw)
To: gregkh, jai.luthra, ilpo.jarvinen
Cc: W_Armin, alex, jgg, vsethi, mhonap, mochs, ankita, clg,
linux-kernel
EGM regions may contain pages retired due to ECC/poison errors that
are known to the system firmware beforehand. SBIOS records the list
in a dedicated carveout region whose SPA is advertised via the ACPI
DSD property "nvidia,egm-retired-pages-data-base". The carveout begins
with a u64 count followed by that many u64 system physical addresses
each the base SPA of a retired 64K page within the EGM region.
The EGM region is linear, so the SPA - EGM_base calculation can be
used to determine the offset within the EGM region. Moreover, the region
is assigned linearly to the VM. So offset within EGM region on the host
would be same as on the VM. Hence the retired pages offset is calculated
and stored by the nvgrace-egm. An xarray is used to track the retired
page offsets. These are exposed to the userspace (Qemu) which may use the
information to skip the retired pages while forming the memory mapping
for the VM.
Note that a retired page SPA SBIOS entry is for 64k size, but the
xarray list tracks offsets in PAGE_SIZE. So SZ_64K/PAGE_SIZE entries
are added to the xarray to ensure comaptibility with 4k OS.
Patch implements the following:
Retired page tracking:
- Map the carveout region with memremap() and validate the count.
- convert each physical address to an in-EGM-region offset and
populate a per-EGM-device hashtable (keyed by that offset).
Userspace ioctl:
- Introduce EGM_RETIRED_PAGES_LIST ioctl and add a new UAPI header
Userspace (QEMU) calls this ioctl to retrieve the list of retired
page offsets. QEMU then communicates the list to the VM so the
guest can take appropriate action.
Update MAINTAINERS to list include/uapi/linux/egm.h
Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
---
.../userspace-api/ioctl/ioctl-number.rst | 1 +
MAINTAINERS | 1 +
drivers/platform/nvidia/egm.c | 211 +++++++++++++++++-
include/uapi/linux/egm.h | 28 +++
4 files changed, 231 insertions(+), 10 deletions(-)
create mode 100644 include/uapi/linux/egm.h
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 3f0ef1e27eb0..94c9ca4a89ff 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -306,6 +306,7 @@ Code Seq# Include File Comments
'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API
'v' C0-FF linux/meye.h conflict!
'w' all CERN SCI driver
+'x' 00-1F include/uapi/linux/egm.h NVIDIA nvgrace-egm driver
'y' 00-1F packet based user level communications
<mailto:zapman@interlan.net>
'z' 00-3F CAN bus card conflict!
diff --git a/MAINTAINERS b/MAINTAINERS
index 2edf903a7746..e77c22a9e598 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -19348,6 +19348,7 @@ NVIDIA GRACE EGM PLATFORM DRIVER
M: Ankit Agrawal <ankita@nvidia.com>
S: Supported
F: drivers/platform/nvidia/
+F: include/uapi/linux/egm.h
NVIDIA VRS RTC DRIVER
M: Shubhi Garg <shgarg@nvidia.com>
diff --git a/drivers/platform/nvidia/egm.c b/drivers/platform/nvidia/egm.c
index 09ab00213de2..df963caedaa8 100644
--- a/drivers/platform/nvidia/egm.c
+++ b/drivers/platform/nvidia/egm.c
@@ -7,10 +7,28 @@
#include <linux/sched/signal.h>
#include <linux/sizes.h>
#include <linux/vfio_pci_core.h>
+#include <linux/xarray.h>
+#include <uapi/linux/egm.h>
#define NVGRACE_EGM_DEV_NAME "egm"
#define MAX_EGM_NODES 4
+/*
+ * Presence marker stored in nvgrace_egm_dev.retired_pages. Only the
+ * page offset within the region carries information and the value
+ * just needs to be a non-NULL xa entry.
+ */
+#define EGM_RETIRED_MARK xa_mk_value(1)
+
+/*
+ * The structure match the format of the retired pages information populated
+ * by the system firmware.
+ */
+struct fw_egm_retired_pages {
+ u64 num_retired_pages;
+ __u64 retired_page_addr[4096];
+};
+
static dev_t dev;
static struct class *class;
@@ -26,8 +44,14 @@ struct nvgrace_egm_dev {
struct mutex open_lock;
/* protected by open_lock */
unsigned int open_count;
+ /*
+ * Set of retired page offsets within the
+ * EGM region and keyed by page index.
+ */
+ struct xarray retired_pages;
phys_addr_t egmphys;
size_t egmlength;
+ phys_addr_t retiredpagesphys;
u64 egmpxm;
struct list_head gpus;
};
@@ -44,6 +68,9 @@ struct nvgrace_egm_dev_entry {
*/
static LIST_HEAD(egm_chardevs);
+static void cleanup_retired_pages(struct nvgrace_egm_dev *egm_dev);
+static int nvgrace_egm_fetch_retired_pages(struct nvgrace_egm_dev *egm_dev);
+
static int nvgrace_egm_open(struct inode *inode, struct file *file)
{
struct nvgrace_egm_dev *egm_dev =
@@ -171,11 +198,68 @@ static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma)
vma->vm_page_prot);
}
+static long nvgrace_egm_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ unsigned long minsz = offsetofend(struct egm_retired_pages_list, count);
+ struct egm_retired_pages_list info;
+ void __user *uarg = (void __user *)arg;
+ struct nvgrace_egm_dev *egm_dev = file->private_data;
+
+ if (copy_from_user(&info, uarg, minsz))
+ return -EFAULT;
+
+ if (info.argsz < minsz || !egm_dev)
+ return -EINVAL;
+
+ switch (cmd) {
+ case EGM_RETIRED_PAGES_LIST: {
+ unsigned long retired_page_struct_size = sizeof(struct egm_retired_pages_info);
+ struct egm_retired_pages_info tmp;
+ unsigned long page_index;
+ void *entry;
+ int count = 0, fill = 0;
+
+ xa_for_each(&egm_dev->retired_pages, page_index, entry)
+ count++;
+
+ if (info.argsz < (minsz + count * retired_page_struct_size)) {
+ info.argsz = minsz + count * retired_page_struct_size;
+ info.count = 0;
+ goto done;
+ }
+
+ xa_for_each(&egm_dev->retired_pages, page_index, entry) {
+ if (fill >= count)
+ break;
+
+ tmp.offset = (u64)page_index << PAGE_SHIFT;
+ tmp.size = PAGE_SIZE;
+
+ if (copy_to_user((u8 __user *)uarg + minsz +
+ fill * retired_page_struct_size,
+ &tmp, retired_page_struct_size))
+ return -EFAULT;
+
+ fill++;
+ }
+
+ info.count = fill;
+ break;
+ }
+ default:
+ return -EINVAL;
+ }
+
+done:
+ return copy_to_user(uarg, &info, minsz) ? -EFAULT : 0;
+}
+
static const struct file_operations file_ops = {
.owner = THIS_MODULE,
.open = nvgrace_egm_open,
.release = nvgrace_egm_release,
.mmap = nvgrace_egm_mmap,
+ .unlocked_ioctl = nvgrace_egm_ioctl,
};
static int nvgrace_egm_create_gpu_links(struct nvgrace_egm_dev *egm_dev,
@@ -236,13 +320,15 @@ static void egm_chardev_release(struct device *dev)
{
struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device);
+ cleanup_retired_pages(egm_chardev);
remove_gpus(egm_chardev);
mutex_destroy(&egm_chardev->open_lock);
kfree(egm_chardev);
}
static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,
- u64 egmpxm)
+ u64 egmpxm,
+ u64 retiredpagesphys)
{
struct nvgrace_egm_dev *egm_chardev;
unsigned int baseminor = MINOR(dev);
@@ -271,7 +357,9 @@ static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,
egm_chardev->egmphys = egmphys;
egm_chardev->egmlength = egmlength;
egm_chardev->egmpxm = egmpxm;
+ egm_chardev->retiredpagesphys = retiredpagesphys;
mutex_init(&egm_chardev->open_lock);
+ xa_init(&egm_chardev->retired_pages);
INIT_LIST_HEAD(&egm_chardev->gpus);
egm_chardev->device.devt = MKDEV(MAJOR(dev), egm_chardev->egmpxm);
@@ -284,6 +372,10 @@ static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,
if (ret)
goto error_exit;
+ ret = nvgrace_egm_fetch_retired_pages(egm_chardev);
+ if (ret)
+ goto error_exit;
+
ret = cdev_device_add(&egm_chardev->cdev, &egm_chardev->device);
if (ret)
goto error_exit;
@@ -302,6 +394,85 @@ static void del_egm_chardev(struct nvgrace_egm_dev *egm_chardev)
put_device(&egm_chardev->device);
}
+static void cleanup_retired_pages(struct nvgrace_egm_dev *egm_dev)
+{
+ /* Entries are value marks, so xa_destroy() frees the table itself. */
+ xa_destroy(&egm_dev->retired_pages);
+}
+
+static int nvgrace_egm_fetch_retired_pages(struct nvgrace_egm_dev *egm_dev)
+{
+ struct fw_egm_retired_pages *egm_retired;
+ u64 count;
+ int index, ret = 0;
+
+ /* No retired-pages region was advertised; nothing to populate. */
+ if (!egm_dev->retiredpagesphys)
+ return 0;
+
+ egm_retired = memremap(egm_dev->retiredpagesphys,
+ sizeof(*egm_retired), MEMREMAP_WB);
+ if (!egm_retired)
+ return -ENOMEM;
+
+ count = egm_retired->num_retired_pages;
+ if (count > ARRAY_SIZE(egm_retired->retired_page_addr)) {
+ memunmap(egm_retired);
+ return -EINVAL;
+ }
+
+ for (index = 0; index < count && !ret; index++) {
+ phys_addr_t base = egm_retired->retired_page_addr[index];
+ int sub;
+
+ /*
+ * Entries come from SBIOS. Skip any base outside the EGM
+ * region [egmphys, egmphys + egmlength).
+ */
+ if (base < egm_dev->egmphys ||
+ base >= egm_dev->egmphys + egm_dev->egmlength) {
+ dev_warn_ratelimited(&egm_dev->device,
+ "Ignoring retired page %pa outside EGM region\n",
+ &base);
+ continue;
+ }
+
+ /*
+ * Since the EGM is linearly mapped, the offset in the
+ * carveout is the same offset in the VM system memory.
+ *
+ * Calculate the offset to communicate to the usermode
+ * apps.
+ *
+ * The retired page entry represent a retired region of
+ * size 64K. So on a 4K kernel, each entry spans 0x10
+ * 4K sub-pages; add one entry per sub-page so that any
+ * lookup at 4K granularity hits the right retired range.
+ */
+ for (sub = 0; sub < SZ_64K / PAGE_SIZE; sub++) {
+ unsigned long offset = base +
+ (phys_addr_t)sub * PAGE_SIZE - egm_dev->egmphys;
+ void *old;
+
+ old = xa_store(&egm_dev->retired_pages,
+ offset >> PAGE_SHIFT, EGM_RETIRED_MARK,
+ GFP_KERNEL);
+ if (xa_is_err(old)) {
+ ret = xa_err(old);
+ break;
+ }
+ }
+ }
+
+ memunmap(egm_retired);
+
+ /*
+ * On failure the partially populated table is freed by
+ * egm_chardev_release() via the caller's error path.
+ */
+ return ret;
+}
+
static char *egm_devnode(const struct device *device, umode_t *mode)
{
if (mode)
@@ -347,22 +518,39 @@ static int has_egm_property(struct pci_dev *pdev, u64 *pegmpxm)
}
static int fetch_egm_property(struct pci_dev *pdev, u64 *pegmphys,
- u64 *pegmlength)
+ u64 *pegmlength, u64 *pretiredpagesphys)
{
int ret;
/*
- * The memory information is present in the system ACPI tables as DSD
- * properties nvidia,egm-base-pa and nvidia,egm-size.
+ * The EGM memory information is present in the system ACPI tables
+ * as DSD properties nvidia,egm-base-pa and nvidia,egm-size.
*/
ret = device_property_read_u64(&pdev->dev, "nvidia,egm-size",
pegmlength);
if (ret)
- return ret;
+ goto error_exit;
ret = device_property_read_u64(&pdev->dev, "nvidia,egm-base-pa",
pegmphys);
+ if (ret)
+ goto error_exit;
+
+ /*
+ * SBIOS puts the list of retired pages on a region exposed as
+ * "nvidia,egm-retired-pages-data-base". Older firmware may not expose
+ * it. Treat an absent or zero property as 0 retired pages so the
+ * EGM device is still created (with an empty retired-pages table).
+ */
+ ret = device_property_read_u64(&pdev->dev,
+ "nvidia,egm-retired-pages-data-base",
+ pretiredpagesphys);
+ if (ret) {
+ *pretiredpagesphys = 0;
+ ret = 0;
+ }
+error_exit:
return ret;
}
@@ -409,12 +597,13 @@ static int nvgrace_egm_create_pci_egm_devs(void)
int ret;
for_each_pci_dev(pdev) {
- u64 egmphys, egmlength, egmpxm;
+ u64 egmphys, egmlength, egmpxm, retiredpagesphys;
if (has_egm_property(pdev, &egmpxm))
continue;
- ret = fetch_egm_property(pdev, &egmphys, &egmlength);
+ ret = fetch_egm_property(pdev, &egmphys, &egmlength,
+ &retiredpagesphys);
if (ret)
continue;
@@ -432,7 +621,8 @@ static int nvgrace_egm_create_pci_egm_devs(void)
return -ENOMEM;
}
- egm_dev = setup_egm_chardev(egmphys, egmlength, egmpxm);
+ egm_dev = setup_egm_chardev(egmphys, egmlength, egmpxm,
+ retiredpagesphys);
if (!egm_dev) {
kfree(egm_entry);
pci_dev_put(pdev);
@@ -478,12 +668,13 @@ static int nvgrace_egm_lowest_pxm(u64 *pmin_pxm)
bool found = false;
for_each_pci_dev(pdev) {
- u64 egmphys, egmlength, egmpxm;
+ u64 egmphys, egmlength, egmpxm, retiredpagesphys;
if (has_egm_property(pdev, &egmpxm))
continue;
- if (fetch_egm_property(pdev, &egmphys, &egmlength))
+ if (fetch_egm_property(pdev, &egmphys, &egmlength,
+ &retiredpagesphys))
continue;
if (!found || egmpxm < *pmin_pxm) {
diff --git a/include/uapi/linux/egm.h b/include/uapi/linux/egm.h
new file mode 100644
index 000000000000..ee165056c2f6
--- /dev/null
+++ b/include/uapi/linux/egm.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ */
+
+#ifndef _UAPI_LINUX_EGM_H
+#define _UAPI_LINUX_EGM_H
+
+#include <linux/types.h>
+
+#define EGM_TYPE ('x')
+
+struct egm_retired_pages_info {
+ __aligned_u64 offset;
+ __aligned_u64 size;
+};
+
+struct egm_retired_pages_list {
+ __u32 argsz;
+ /* out */
+ __u32 count;
+ /* out */
+ struct egm_retired_pages_info retired_pages[];
+};
+
+#define EGM_RETIRED_PAGES_LIST _IOWR(EGM_TYPE, 0x01, struct egm_retired_pages_list)
+
+#endif /* _UAPI_LINUX_EGM_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 4/4] platform/nvidia: Register EGM PFNMAP range with memory_failure
2026-07-02 19:25 [PATCH 0/4] Introduce nvgrace-egm driver for Extended GPU Memory Ankit Agrawal
` (2 preceding siblings ...)
2026-07-02 19:25 ` [PATCH 3/4] platform/nvidia: Handle retired ECC pages and expose via ioctl Ankit Agrawal
@ 2026-07-02 19:25 ` Ankit Agrawal
3 siblings, 0 replies; 7+ messages in thread
From: Ankit Agrawal @ 2026-07-02 19:25 UTC (permalink / raw)
To: gregkh, jai.luthra, ilpo.jarvinen
Cc: W_Armin, alex, jgg, vsethi, mhonap, mochs, ankita, clg,
linux-kernel
EGM carveout memory is mapped directly into userspace (QEMU) and is not
added to the kernel. It is not managed by the kernel page allocator and
has no struct pages. The module can thus utilize the Linux memory manager's
memory_failure mechanism for regions with no struct pages. The Linux MM
code exposes register/unregister APIs allowing modules to register such
memory regions for memory_failure handling. On the occurrence of the
memory_failure, Linux MM calls the registered function if the PFN is
within the range.
These dynamically occurring memory_failure pages are tracked alongside
the statically populated list by the system firmware in the xarray. So
the userspace can get both the information through the common
EGM_RETIRED_PAGES_LIST ioctl.
Register the EGM PFN range with the memory_failure infrastructure on
first open() and unregister it on the last close(). Provide a
pfn_to_vma_pgoff callback that:
- Validates that the reported PFN falls within the EGM region and the
current VMA.
- Converts the PFN to a file offset within the EGM region.
- Records the poisoned offset in the existing retired-pages hashtable
so it is reported to userspace via EGM_RETIRED_PAGES_LIST alongside
statically configured retired pages from SBIOS.
Assisted-by: Claude:claude-opus-4.8
Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
---
drivers/platform/nvidia/egm.c | 123 +++++++++++++++++++++++++++++++---
1 file changed, 115 insertions(+), 8 deletions(-)
diff --git a/drivers/platform/nvidia/egm.c b/drivers/platform/nvidia/egm.c
index df963caedaa8..39c4be768f75 100644
--- a/drivers/platform/nvidia/egm.c
+++ b/drivers/platform/nvidia/egm.c
@@ -3,6 +3,7 @@
* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved
*/
+#include <linux/memory-failure.h>
#include <linux/mutex.h>
#include <linux/sched/signal.h>
#include <linux/sizes.h>
@@ -40,7 +41,7 @@ struct gpu_node {
struct nvgrace_egm_dev {
struct device device;
struct cdev cdev;
- /* serialises the first-open scrub */
+ /* serialises the first-open scrub + PFN registration */
struct mutex open_lock;
/* protected by open_lock */
unsigned int open_count;
@@ -49,6 +50,7 @@ struct nvgrace_egm_dev {
* EGM region and keyed by page index.
*/
struct xarray retired_pages;
+ struct pfn_address_space pfn_address_space;
phys_addr_t egmphys;
size_t egmlength;
phys_addr_t retiredpagesphys;
@@ -71,6 +73,104 @@ static LIST_HEAD(egm_chardevs);
static void cleanup_retired_pages(struct nvgrace_egm_dev *egm_dev);
static int nvgrace_egm_fetch_retired_pages(struct nvgrace_egm_dev *egm_dev);
+static int pfn_memregion_offset(struct nvgrace_egm_dev *egm_dev,
+ unsigned long pfn,
+ pgoff_t *pfn_offset_in_region)
+{
+ unsigned long start_pfn, num_pages;
+
+ start_pfn = PHYS_PFN(egm_dev->egmphys);
+ num_pages = egm_dev->egmlength >> PAGE_SHIFT;
+
+ if (pfn < start_pfn || pfn >= start_pfn + num_pages)
+ return -EFAULT;
+
+ *pfn_offset_in_region = pfn - start_pfn;
+
+ return 0;
+}
+
+static int track_ecc_offset(struct nvgrace_egm_dev *egm_dev,
+ unsigned long mem_offset)
+{
+ void *old;
+
+ /*
+ * This runs from the pfn_to_vma_pgoff callback, which the
+ * memory_failure path invokes under rcu_read_lock() (see
+ * collect_procs_pfn()). Sleeping is not allowed in that atomic
+ * context and so use GFP_ATOMIC rather than GFP_KERNEL.
+ *
+ * Storing by page index makes a repeated report at the same offset a
+ * harmless overwrite, so no explicit de-duplication is needed.
+ */
+ old = xa_store(&egm_dev->retired_pages, mem_offset >> PAGE_SHIFT,
+ EGM_RETIRED_MARK, GFP_ATOMIC);
+ if (xa_is_err(old))
+ return xa_err(old);
+
+ return 0;
+}
+
+static int nvgrace_egm_pfn_to_vma_pgoff(struct vm_area_struct *vma,
+ unsigned long pfn,
+ pgoff_t *pgoff)
+{
+ struct nvgrace_egm_dev *egm_dev = vma->vm_file->private_data;
+ pgoff_t vma_offset_in_region = vma->vm_pgoff;
+ pgoff_t pfn_offset_in_region;
+ int ret;
+
+ ret = pfn_memregion_offset(egm_dev, pfn, &pfn_offset_in_region);
+ if (ret)
+ return ret;
+
+ /* Ensure PFN is not before VMA's start within the region */
+ if (pfn_offset_in_region < vma_offset_in_region)
+ return -EFAULT;
+
+ /* Ensure PFN is not past the VMA's end within the region */
+ if (pfn_offset_in_region >= vma_offset_in_region + vma_pages(vma))
+ return -EFAULT;
+
+ /*
+ * The VMA's vm_pgoff is the page offset of its start within the EGM
+ * region, so the region offset of the pfn is already the pgoff to
+ * report.
+ */
+ *pgoff = pfn_offset_in_region;
+
+ /*
+ * Record the poisoned offset for reporting via
+ * EGM_RETIRED_PAGES_LIST. This bookkeeping is best-effort: its
+ * failure must not be reported as "vma does not map pfn", or the
+ * owning process would not be signalled for the poisoned page.
+ */
+ track_ecc_offset(egm_dev, *pgoff << PAGE_SHIFT);
+
+ return 0;
+}
+
+static int
+nvgrace_egm_register_pfn_range(struct inode *inode,
+ struct nvgrace_egm_dev *egm_dev)
+{
+ unsigned long pfn, nr_pages;
+ int ret;
+
+ pfn = PHYS_PFN(egm_dev->egmphys);
+ nr_pages = egm_dev->egmlength >> PAGE_SHIFT;
+
+ egm_dev->pfn_address_space.node.start = pfn;
+ egm_dev->pfn_address_space.node.last = pfn + nr_pages - 1;
+ egm_dev->pfn_address_space.mapping = inode->i_mapping;
+ egm_dev->pfn_address_space.pfn_to_vma_pgoff = nvgrace_egm_pfn_to_vma_pgoff;
+
+ ret = register_pfn_address_space(&egm_dev->pfn_address_space);
+
+ return ret;
+}
+
static int nvgrace_egm_open(struct inode *inode, struct file *file)
{
struct nvgrace_egm_dev *egm_dev =
@@ -84,9 +184,9 @@ static int nvgrace_egm_open(struct inode *inode, struct file *file)
/*
* The EGM region is a physical carveout handed to a single VM at a
- * time and must be scrubbed before each handover. Take open_lock
- * killably around the scrub which can run for long time, so that
- * the waiters stay killable.
+ * time and must be scrubbed and registered with memory_failure before
+ * each handover. Take open_lock killably around the scrub which can
+ * run for long time, so that the waiters stay killable.
*/
if (mutex_lock_killable(&egm_dev->open_lock))
return -EINTR;
@@ -142,10 +242,15 @@ static int nvgrace_egm_open(struct inode *inode, struct file *file)
remaining -= chunk_size;
}
+ ret = nvgrace_egm_register_pfn_range(inode, egm_dev);
+ if (ret && ret != -EOPNOTSUPP)
+ goto unlock;
+
/*
- * Mark the device open only after the scrub completes so that a
- * concurrent opener cannot observe a non-zero count and proceed
- * before the region has been cleared.
+ * Only mark the device open once the scrub and registration have
+ * succeeded. On failure open_count stays 0, so this opener returns
+ * an error (release() never runs for it) and the next opener
+ * retries the scrub and registration from a clean state.
*/
egm_dev->open_count = 1;
ret = 0;
@@ -162,8 +267,10 @@ static int nvgrace_egm_release(struct inode *inode, struct file *file)
guard(mutex)(&egm_dev->open_lock);
- if (!--egm_dev->open_count)
+ if (!--egm_dev->open_count) {
+ unregister_pfn_address_space(&egm_dev->pfn_address_space);
file->private_data = NULL;
+ }
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 7+ messages in thread