Re: [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions

[Alex Williamson <alex@shazbot.org>]

On Thu, 2 Jul 2026 19:25:29 +0000
Ankit Agrawal <ankita@nvidia.com> wrote:

> The Extended GPU Memory (EGM) feature enables the GPU to access system
> memory allocations within and across nodes through a high-bandwidth path
> on Grace-based systems. The GPU can utilize system memory on the same
> socket, a different socket or a different node in a multi-node setup.
> 
> When EGM mode is enabled through SBIOS, host system memory is partitioned
> into two parts: a Hypervisor region for the host OS and a
> Hypervisor-Invisible (HI) region for the VM. Only the hypervisor region
> is part of the host EFI map and visible to the host OS on bootup. The
> HI partition is interchangeably called the EGM region. Its base SPA and
> size are exposed through ACPI DSDT properties.
> 
> Whilst the EGM region is accessible on the host, it is not
> added/hot-plugged to the kernel. The HI region is assigned to a VM
> by mapping the QEMU VMA to the SPA using remap_pfn_range(). So
> it looks like the following:
> 
>   |---- Sysmem ----|                  VM view
>   |                |
>   |IPA <-> SPA map |
>   |                |
>   |--- HI / EGM ---|-- Host Mem --|   Host view
> 
> Introduce the nvgrace-egm platform driver module and implement full EGM
> region discovery, char device exposure and sysfs metadata.
> 
> Build infrastructure:
> - Kconfig, Makefile entries, MAINTAINERS entry.
> - An 'egm' device class and a range of char device minor numbers.
>   MAX_EGM_NODES is set to 4 to matches the largest 4-socket Grace
>   configuration.
> 
> Discovery:
> - Walk all PCI devices at module init time. For each one advertising
>   "nvidia,egm-pxm", read the EGM region properties (base SPA, size and
>   PXM domain) from the ACPI DSD table via the GPU device handle, using
>   "nvidia,egm-base-pa" and "nvidia,egm-size" properties.
> - Maintain a module-level list (egm_chardevs) of discovered regions,
>   de-duplicating by PXM so each socket gets exactly one EGM device
>   regardless of how many GPUs it has (2 GPUs per EGM device in GB200).
> 
> Char device creation:
> - Each EGM region is exposed as /dev/egmX, where X is the PXM number.
>   Embed struct device and struct cdev inside nvgrace_egm_dev so the
>   device lifecycle follows the standard kernel device model.
> - Implement setup_egm_chardev() and del_egm_chardev() for setup and
>   teardown. The devnode callback sets device permissions to 0600.
> - Stub file operations (open, release and mmap) are registered.
> 
> GPU tracking and sysfs:
> - Track which PCI devices are associated with each EGM region in a
>   per-device GPU list (struct gpu_node).
> - Expose sysfs symlinks from each EGM device to its associated GPU PCI
>   devices to allow the userspace to determine GPU-to-socket affinity and
>   replicate the host EGM topology in the VM. Helps admin understand
>   GPU association.
> - Expose the EGM region size via an egm_size sysfs attribute so QEMU
>   can determine the memory-backend-file size. On a 2-socket Grace
>   Blackwell setup this appears as:
>     Socket0: /sys/devices/virtual/egm/egm4/egm_size
>     Socket1: /sys/devices/virtual/egm/egm5/egm_size
> 
> Introduce nvgrace_egm_create_pci_egm_devs() and its counterpart
> nvgrace_egm_destroy_pci_devs() that are called from module init/exit.
> 
> Suggested-by: Alex Williamson <alex@shazbot.org>
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Suggested-by: Aniket Agashe <aniketa@nvidia.com>
> Suggested-by: Matthew R. Ochs <mochs@nvidia.com>
> Assisted-by: Claude:claude-opus-4.8
> Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
> ---
>  MAINTAINERS                      |   5 +
>  drivers/platform/Kconfig         |   1 +
>  drivers/platform/Makefile        |   1 +
>  drivers/platform/nvidia/Kconfig  |  10 +
>  drivers/platform/nvidia/Makefile |   3 +
>  drivers/platform/nvidia/egm.c    | 459 +++++++++++++++++++++++++++++++
>  6 files changed, 479 insertions(+)
>  create mode 100644 drivers/platform/nvidia/Kconfig
>  create mode 100644 drivers/platform/nvidia/Makefile
>  create mode 100644 drivers/platform/nvidia/egm.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 15011f5752a9..2edf903a7746 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -19344,6 +19344,11 @@ L:	linux-acpi@vger.kernel.org
>  S:	Maintained
>  F:	drivers/acpi/apei/ghes-nvidia.c
>  
> +NVIDIA GRACE EGM PLATFORM DRIVER
> +M:	Ankit Agrawal <ankita@nvidia.com>
> +S:	Supported
> +F:	drivers/platform/nvidia/
> +
>  NVIDIA VRS RTC DRIVER
>  M:	Shubhi Garg <shgarg@nvidia.com>
>  L:	linux-tegra@vger.kernel.org
> diff --git a/drivers/platform/Kconfig b/drivers/platform/Kconfig
> index 312788f249c9..21f2dc901301 100644
> --- a/drivers/platform/Kconfig
> +++ b/drivers/platform/Kconfig
> @@ -22,3 +22,4 @@ source "drivers/platform/arm64/Kconfig"
>  source "drivers/platform/raspberrypi/Kconfig"
>  
>  source "drivers/platform/wmi/Kconfig"
> +source "drivers/platform/nvidia/Kconfig"
> diff --git a/drivers/platform/Makefile b/drivers/platform/Makefile
> index fa322e7f8716..96de8fa26ed5 100644
> --- a/drivers/platform/Makefile
> +++ b/drivers/platform/Makefile
> @@ -15,3 +15,4 @@ obj-$(CONFIG_SURFACE_PLATFORMS)	+= surface/
>  obj-$(CONFIG_ARM64_PLATFORM_DEVICES)	+= arm64/
>  obj-$(CONFIG_BCM2835_VCHIQ)	+= raspberrypi/
>  obj-$(CONFIG_ACPI_WMI)		+= wmi/
> +obj-$(CONFIG_NVGRACE_EGM)	+= nvidia/
> diff --git a/drivers/platform/nvidia/Kconfig b/drivers/platform/nvidia/Kconfig
> new file mode 100644
> index 000000000000..8554a58b1141
> --- /dev/null
> +++ b/drivers/platform/nvidia/Kconfig
> @@ -0,0 +1,10 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config NVGRACE_EGM
> +	tristate "EGM driver for NVIDIA Grace Hopper and Blackwell Superchip"
> +	depends on ARM64 || (COMPILE_TEST && 64BIT)
> +	help
> +	  Extended GPU Memory (EGM) support for the GPU in the NVIDIA Grace
> +	  based chips required to avail the CPU memory as additional
> +	  cross-node/cross-socket memory for GPU using KVM/qemu.
> +
> +	  If you don't know what to do here, say N.
> diff --git a/drivers/platform/nvidia/Makefile b/drivers/platform/nvidia/Makefile
> new file mode 100644
> index 000000000000..a89f820908fb
> --- /dev/null
> +++ b/drivers/platform/nvidia/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +obj-$(CONFIG_NVGRACE_EGM) += nvgrace-egm.o
> +nvgrace-egm-y := egm.o
> diff --git a/drivers/platform/nvidia/egm.c b/drivers/platform/nvidia/egm.c
> new file mode 100644
> index 000000000000..a340c4fcbc7a
> --- /dev/null
> +++ b/drivers/platform/nvidia/egm.c
> @@ -0,0 +1,459 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved
> + */
> +
> +#include <linux/vfio_pci_core.h>

Bogus include.

> +
> +#define NVGRACE_EGM_DEV_NAME "egm"
> +#define MAX_EGM_NODES 4
> +
> +static dev_t dev;
> +static struct class *class;

This should move to using class_register() rather than class_create()
as the latter is being phased out[1].  This then becomes a static const
struct class:

  static const struct class egm_class = {                                                          
      .name = NVGRACE_EGM_DEV_NAME,                                                                
      .devnode = egm_devnode,                                                                      
      .dev_groups = attr_groups,                                                                   
  }; 

[1]https://lore.kernel.org/all/2023040244-duffel-pushpin-f738@gregkh/

> +
> +struct gpu_node {
> +	struct list_head list;
> +	struct pci_dev *pdev;
> +};
> +
> +struct nvgrace_egm_dev {
> +	struct device device;
> +	struct cdev cdev;
> +	phys_addr_t egmphys;
> +	size_t egmlength;
> +	u64 egmpxm;
> +	struct list_head gpus;
> +};

I can't spot any consistency in when nvgrace is prefixed to structures
and function and when it's not.

> +
> +struct nvgrace_egm_dev_entry {
> +	struct list_head list;
> +	struct nvgrace_egm_dev *egm_dev;
> +};
> +
> +/*
> + * Track egm device lists. Note that there is one device per socket.
> + * All the GPUs belonging to the same sockets are associated with
> + * the EGM device for that socket.
> + */
> +static LIST_HEAD(egm_chardevs);
> +
> +static int nvgrace_egm_open(struct inode *inode, struct file *file)
> +{
> +	return 0;
> +}
> +
> +static int nvgrace_egm_release(struct inode *inode, struct file *file)
> +{
> +	return 0;
> +}
> +
> +static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma)
> +{
> +	/*
> +	 * Mapping the EGM region into userspace is implemented by a later
> +	 * patch. Until then refuse the mmap.
> +	 */
> +	return -EOPNOTSUPP;
> +}
> +
> +static const struct file_operations file_ops = {
> +	.owner = THIS_MODULE,
> +	.open = nvgrace_egm_open,
> +	.release = nvgrace_egm_release,
> +	.mmap = nvgrace_egm_mmap,
> +};
> +
> +static int nvgrace_egm_create_gpu_links(struct nvgrace_egm_dev *egm_dev,
> +					struct pci_dev *pdev)
> +{
> +	int ret;
> +
> +	ret = sysfs_create_link(&egm_dev->device.kobj,
> +				&pdev->dev.kobj,
> +				dev_name(&pdev->dev));
> +
> +	if (ret && ret != -EEXIST)
> +		return ret;

AIUI, there are 2 GPUs per egm, and we're creating symlinks from the
single egm object to the individual, unique, GPUs.  Why do we ignore
-EEXIST?

> +
> +	return 0;
> +}
> +
> +static void remove_egm_symlinks(struct nvgrace_egm_dev *egm_dev,
> +				struct pci_dev *pdev)
> +{
> +	sysfs_remove_link(&egm_dev->device.kobj, dev_name(&pdev->dev));
> +}

Consistent naming between the symmetric functions would be nice.

> +
> +static int add_gpu(struct nvgrace_egm_dev *egm_dev, struct pci_dev *pdev)
> +{
> +	struct gpu_node *node;
> +
> +	node = kzalloc_obj(*node, GFP_KERNEL);
> +	if (!node)
> +		return -ENOMEM;
> +
> +	/*
> +	 * The pdev is stored for the lifetime of the EGM device but it is
> +	 * obtained from a for_each_pci_dev() walk that drops its reference as
> +	 * it advances. Take a reference here so the stored pointer stays valid
> +	 * even if the GPU is later unbound; remove_gpus() drops it.
> +	 */
> +	node->pdev = pci_dev_get(pdev);
> +
> +	list_add_tail(&node->list, &egm_dev->gpus);
> +
> +	return nvgrace_egm_create_gpu_links(egm_dev, pdev);
> +}

If the EEXIST issue above is fabricated and both symlink functions are
just wrappers with a single caller, we should drop them, ex:

	node = kzalloc_obj(...);
	if (!node)
		return -ENOMEM;

	ret = sysfs_create_link(...);
	if (ret) {
		kfree(node);
		return ret;
	}

	node->pdev = pci_dev_get(pdev);

	list_add_tail(...);

	return 0;

> +
> +static void remove_gpus(struct nvgrace_egm_dev *egm_dev)
> +{
> +	struct gpu_node *node, *tmp;
> +
> +	list_for_each_entry_safe(node, tmp, &egm_dev->gpus, list) {
> +		remove_egm_symlinks(egm_dev, node->pdev);
> +		list_del(&node->list);
> +		pci_dev_put(node->pdev);
> +		kfree(node);
> +	}
> +}

Style choice, but this could drain with:

	struct gpu_node *gpu;

	while ((gpu = list_first_entry_or_null(&egm_dev->gpus,...)) {
		sysfs_remove_link(...)
		...
	}

> +
> +static void egm_chardev_release(struct device *dev)
> +{
> +	struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device);
> +
> +	remove_gpus(egm_chardev);
> +	kfree(egm_chardev);
> +}

Fix subject/verb consistency in naming throughout, egm_chardev_release()
above (subject-verb), setup_egm_chardev() below (verb-subject).

> +
> +static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,
> +						 u64 egmpxm)
> +{
> +	struct nvgrace_egm_dev *egm_chardev;
> +	unsigned int baseminor = MINOR(dev);
> +	int ret;
> +
> +	/*
> +	 * Only MAX_EGM_NODES minors from baseminor were reserved. Reject a PXM
> +	 * outside that window.
> +	 */
> +	if (egmpxm < baseminor || egmpxm - baseminor >= MAX_EGM_NODES) {
> +		pr_err("nvgrace-egm: EGM proximity domain %llu outside reserved minor window [%u, %u)\n",
> +		       egmpxm, baseminor, baseminor + MAX_EGM_NODES);
> +		goto create_err;
> +	}

See below, this is an artificial requirement.

> +
> +	egm_chardev = kzalloc_obj(*egm_chardev, GFP_KERNEL);
> +	if (!egm_chardev)
> +		goto create_err;
> +
> +	device_initialize(&egm_chardev->device);
> +
> +	/*
> +	 * Use the proximity domain number as the device minor number.
> +	 * So the EGM corresponding to node X would be /dev/egmX.
> +	 */

See below, what does this give us?

> +	egm_chardev->egmphys = egmphys;
> +	egm_chardev->egmlength = egmlength;
> +	egm_chardev->egmpxm = egmpxm;
> +	INIT_LIST_HEAD(&egm_chardev->gpus);
> +
> +	egm_chardev->device.devt = MKDEV(MAJOR(dev), egm_chardev->egmpxm);
> +	egm_chardev->device.class = class;
> +	egm_chardev->device.release = egm_chardev_release;
> +	cdev_init(&egm_chardev->cdev, &file_ops);
> +	egm_chardev->cdev.owner = THIS_MODULE;
> +
> +	ret = dev_set_name(&egm_chardev->device, "egm%llu", egm_chardev->egmpxm);
> +	if (ret)
> +		goto error_exit;
> +
> +	ret = cdev_device_add(&egm_chardev->cdev, &egm_chardev->device);
> +	if (ret)
> +		goto error_exit;
> +
> +	return egm_chardev;
> +
> +error_exit:
> +	put_device(&egm_chardev->device);
> +create_err:
> +	return NULL;
> +}
> +
> +static void del_egm_chardev(struct nvgrace_egm_dev *egm_chardev)
> +{
> +	cdev_device_del(&egm_chardev->cdev, &egm_chardev->device);
> +	put_device(&egm_chardev->device);
> +}
> +
> +static char *egm_devnode(const struct device *device, umode_t *mode)
> +{
> +	if (mode)
> +		*mode = 0600;
> +
> +	return NULL;
> +}
> +
> +static ssize_t egm_size_show(struct device *dev, struct device_attribute *attr,
> +			     char *buf)
> +{
> +	struct nvgrace_egm_dev *egm_dev =
> +		container_of(dev, struct nvgrace_egm_dev, device);
> +
> +	return sysfs_emit(buf, "0x%zx\n", egm_dev->egmlength);
> +}
> +
> +static DEVICE_ATTR_RO(egm_size);
> +
> +static struct attribute *attrs[] = {
> +	&dev_attr_egm_size.attr,
> +	NULL,
> +};
> +
> +static const struct attribute_group attr_group = {
> +	.attrs = attrs,
> +};
> +
> +static const struct attribute_group *attr_groups[] = {
> +	&attr_group,
> +	NULL
> +};
> +
> +/*
> + * Determine if the EGM feature is enabled. If disabled, there
> + * will be no EGM properties populated in the ACPI tables and this
> + * fetch would fail.
> + */
> +static int has_egm_property(struct pci_dev *pdev, u64 *pegmpxm)
> +{
> +	return device_property_read_u64(&pdev->dev, "nvidia,egm-pxm",
> +					pegmpxm);
> +}

What's the value of splitting this out from the properties below?
They're only used together.  One function could fail on the first
failed property.  The function above has a non-intuitive convention,
named as a bool test, but returns an errno.  The "fetch" function below
has the more intuitive convention.

> +
> +static int fetch_egm_property(struct pci_dev *pdev, u64 *pegmphys,
> +			      u64 *pegmlength)
> +{
> +	int ret;
> +
> +	/*
> +	 * The memory information is present in the system ACPI tables as DSD
> +	 * properties nvidia,egm-base-pa and nvidia,egm-size.
> +	 */
> +	ret = device_property_read_u64(&pdev->dev, "nvidia,egm-size",
> +				       pegmlength);
> +	if (ret)
> +		return ret;
> +
> +	ret = device_property_read_u64(&pdev->dev, "nvidia,egm-base-pa",
> +				       pegmphys);
> +
> +	return ret;
> +}
> +
> +static struct nvgrace_egm_dev *get_egm_dev(u64 egmpxm)
> +{
> +	struct nvgrace_egm_dev_entry *egm_entry;
> +
> +	list_for_each_entry(egm_entry, &egm_chardevs, list) {
> +		/*
> +		 * A system could have multiple GPUs associated with an
> +		 * EGM region and will have the same set of EGM region
> +		 * information. Return the existing EGM device if already
> +		 * registered through a different GPU on the same socket.
> +		 */
> +		if (egm_entry->egm_dev->egmpxm == egmpxm)
> +			return egm_entry->egm_dev;
> +	}
> +
> +	return NULL;
> +}
> +
> +static void nvgrace_egm_destroy_pci_devs(void)
> +{
> +	struct nvgrace_egm_dev_entry *entry, *tmp;
> +
> +	list_for_each_entry_safe(entry, tmp, &egm_chardevs, list) {
> +		list_del(&entry->list);
> +		del_egm_chardev(entry->egm_dev);
> +		kfree(entry);
> +	}
> +}
> +
> +/*
> + * Walk all PCI devices and for each one that has a companion ACPI object
> + * advertising EGM properties, create an auxiliary device for that EGM range.
> + * Multiple GPUs on the same socket share a single EGM region (same proximity
> + * domain). So de-duplicate by skipping a pxm that is already registered.
> + */
> +static int nvgrace_egm_create_pci_egm_devs(void)
> +{
> +	struct nvgrace_egm_dev_entry *egm_entry;
> +	struct nvgrace_egm_dev *egm_dev;
> +	struct pci_dev *pdev = NULL;
> +	int ret;
> +
> +	for_each_pci_dev(pdev) {
> +		u64 egmphys, egmlength, egmpxm;
> +
> +		if (has_egm_property(pdev, &egmpxm))
> +			continue;
> +
> +		ret = fetch_egm_property(pdev, &egmphys, &egmlength);
> +		if (ret)
> +			continue;
> +
> +		egm_dev = get_egm_dev(egmpxm);
> +		if (egm_dev) {
> +			ret = add_gpu(egm_dev, pdev);
> +			if (ret)
> +				goto free_dev;
> +			continue;
> +		}
> +
> +		egm_entry = kzalloc_obj(*egm_entry, GFP_KERNEL);
> +		if (!egm_entry) {
> +			pci_dev_put(pdev);
> +			return -ENOMEM;
> +		}
> +
> +		egm_dev = setup_egm_chardev(egmphys, egmlength, egmpxm);
> +		if (!egm_dev) {
> +			kfree(egm_entry);
> +			pci_dev_put(pdev);
> +			return -EINVAL;
> +		}
> +
> +		egm_entry->egm_dev = egm_dev;
> +
> +		ret = add_gpu(egm_entry->egm_dev, pdev);
> +		if (ret) {
> +			del_egm_chardev(egm_dev);
> +			kfree(egm_entry);
> +			goto free_dev;
> +		}

Bare returns with pci_dev_put() mixed with partial unwinds and gotos is
hard to review and hard to maintain.  Make this consistent.  Failed
functions should cleanup any state they modified, drop the PCI
reference and unwind based on the state of the egm_chardevs and
egm_entry.

> +
> +		list_add_tail(&egm_entry->list, &egm_chardevs);
> +	}
> +
> +	return 0;
> +
> +free_dev:
> +	/*
> +	 * Reached only via goto from inside the loop where the pdev still
> +	 * holds the reference from the last pci_get_device(). So drop it here.
> +	 */
> +	pci_dev_put(pdev);
> +	nvgrace_egm_destroy_pci_devs();
> +	return ret;
> +}
> +
> +/*
> + * The char device minor is the EGM proximity domain number which is sparse
> + * and need not start at 0. Find the lowest proximity domain among the EGM
> + * devices so the minor range can be reserved starting there. Returns -ENODEV
> + * if the system has no EGM devices.
> + *
> + * EGM PXMs are assumed contiguous within [lowest, lowest + MAX_EGM_NODES),
> + * which holds for the supported Grace configurations.
> + */
> +static int nvgrace_egm_lowest_pxm(u64 *pmin_pxm)
> +{
> +	struct pci_dev *pdev = NULL;
> +	bool found = false;
> +
> +	for_each_pci_dev(pdev) {
> +		u64 egmphys, egmlength, egmpxm;
> +
> +		if (has_egm_property(pdev, &egmpxm))
> +			continue;
> +
> +		if (fetch_egm_property(pdev, &egmphys, &egmlength))
> +			continue;
> +
> +		if (!found || egmpxm < *pmin_pxm) {
> +			*pmin_pxm = egmpxm;
> +			found = true;
> +		}
> +	}
> +
> +	if (!found)
> +		return -ENODEV;
> +
> +	/*
> +	 * The PXM is used verbatim as the minor (only MINORBITS wide). Validate
> +	 * the window would not overflow MINORMASK and be truncated.
> +	 */
> +	if (*pmin_pxm + (MAX_EGM_NODES - 1) > MINORMASK) {
> +		pr_err("nvgrace-egm: lowest EGM proximity domain %llu\n"
> +		       "exceeds the minor range\n", *pmin_pxm);
> +		return -ERANGE;
> +	}
> +
> +	return 0;
> +}
> +
> +static int __init nvgrace_egm_init(void)
> +{
> +	u64 min_pxm;
> +	int ret;
> +
> +	/*
> +	 * Each EGM region is exposed as /dev/egmX with the proximity domain X
> +	 * as its minor number. Reserve MAX_EGM_NODES minors starting at
> +	 * the lowest proximity domain so that every egmX device falls within
> +	 * the reserved range. With no EGM devices present, load without
> +	 * reserving any resources.
> +	 */
> +	ret = nvgrace_egm_lowest_pxm(&min_pxm);
> +	if (ret == -ENODEV)
> +		return 0;
> +	else if (ret)
> +		return ret;
> +
> +	ret = alloc_chrdev_region(&dev, min_pxm, MAX_EGM_NODES,
> +				  NVGRACE_EGM_DEV_NAME);

It's not clear to me what it's buying us to require the minor numbers
match the PXM.  OTOH, it's making us walk all the PCI devices twice and
introduces built in fragility to our minor allocation.  Just get any
MAX_EGM_NODES and use an IDA to track allocated minors.  The chardev
still exposes the PXM in the name and the lowest_pxm function goes away.

> +	if (ret < 0)
> +		return ret;
> +
> +	class = class_create(NVGRACE_EGM_DEV_NAME);
> +	if (IS_ERR(class)) {
> +		unregister_chrdev_region(dev, MAX_EGM_NODES);
> +		return PTR_ERR(class);
> +	}
> +
> +	class->devnode = egm_devnode;
> +	class->dev_groups = attr_groups;
> +
> +	ret = nvgrace_egm_create_pci_egm_devs();
> +	if (ret)
> +		goto cleanup_pci_devs;
> +
> +	return 0;
> +
> +cleanup_pci_devs:
> +	nvgrace_egm_destroy_pci_devs();
> +	class_destroy(class);
> +	unregister_chrdev_region(dev, MAX_EGM_NODES);
> +
> +	return ret;
> +}
> +
> +static void __exit nvgrace_egm_cleanup(void)
> +{
> +	/*
> +	 * When no EGM devices were present, the module loaded inertly and
> +	 * reserved nothing. @class stays NULL and there is nothing to undo.
> +	 */
> +	if (!class)
> +		return;

We should instead make nvgrace_egm_create_pci_egm_devs() return -ENODEV
if no devices are found, remove the inertly loaded with nothing to do
state, then we don't need to handle this case.  We also won't have the
class pointer once we move to class_register().  Thanks,

Alex

> +
> +	nvgrace_egm_destroy_pci_devs();
> +	class_destroy(class);
> +	unregister_chrdev_region(dev, MAX_EGM_NODES);
> +}
> +
> +module_init(nvgrace_egm_init);
> +module_exit(nvgrace_egm_cleanup);
> +
> +MODULE_LICENSE("GPL");
> +MODULE_AUTHOR("Ankit Agrawal <ankita@nvidia.com>");
> +MODULE_DESCRIPTION("NVGRACE EGM - Module to support Extended GPU Memory on NVIDIA Grace Based systems");