[PATCH rdma-next] RDMA/mana_ib: Enable RDMA devices on the CDX bus

[PATCH rdma-next] RDMA/mana_ib: Enable RDMA devices on the CDX bus

[Konstantin Taranov]

From: Zhiyue Qiu <zhiyueqiu@microsoft.com>

Enable MANA RDMA on CDX by depending on the shared GDMA core and matching
the CDX auxiliary devices.

Map non-PCI userspace doorbells with pgprot_device() to match the CDX
transport, and report the Microsoft vendor ID directly.

Signed-off-by: Zhiyue Qiu <zhiyueqiu@microsoft.com>
Co-developed-by: Konstantin Taranov <kotaranov@microsoft.com>
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
 drivers/infiniband/hw/mana/Kconfig  | 10 +++++-----
 drivers/infiniband/hw/mana/device.c |  2 ++
 drivers/infiniband/hw/mana/main.c   |  8 +++++---
 3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/mana/Kconfig b/drivers/infiniband/hw/mana/Kconfig
index 546640657..6eae7fa91 100644
--- a/drivers/infiniband/hw/mana/Kconfig
+++ b/drivers/infiniband/hw/mana/Kconfig
@@ -1,10 +1,10 @@
 # SPDX-License-Identifier: GPL-2.0-only
 config MANA_INFINIBAND
 	tristate "Microsoft Azure Network Adapter support"
-	depends on NETDEVICES && ETHERNET && PCI && MICROSOFT_MANA
+	depends on NETDEVICES && ETHERNET && MICROSOFT_GDMA_CORE
 	help
 	  This driver provides low-level RDMA support for Microsoft Azure
-	  Network Adapter (MANA). MANA supports RDMA features that can be used
-	  for workloads (e.g. DPDK, MPI etc) that uses RDMA verbs to directly
-	  access hardware from user-mode processes in Microsoft Azure cloud
-	  environment.
+	  Network Adapter (MANA) on the PCI or CDX bus. MANA supports RDMA
+	  features that workloads such as DPDK and MPI can use to directly
+	  access hardware from user-mode processes in the Microsoft Azure
+	  cloud environment.
diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c
index 62c7803ca..33e474907 100644
--- a/drivers/infiniband/hw/mana/device.c
+++ b/drivers/infiniband/hw/mana/device.c
@@ -257,6 +257,8 @@ static void mana_ib_remove(struct auxiliary_device *adev)
 static const struct auxiliary_device_id mana_id_table[] = {
 	{ .name = "mana.rdma", },
 	{ .name = "mana.eth", },
+	{ .name = "mana_cdx.rdma", },
+	{ .name = "mana_cdx.eth", },
 	{},
 };
 
diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index 858bbc325..c63ac20e7 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -572,7 +572,10 @@ int mana_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)
 	pfn = (gc->phys_db_page_base +
 	       gc->db_page_size * mana_ucontext->doorbell) >>
 	      PAGE_SHIFT;
-	prot = pgprot_writecombine(vma->vm_page_prot);
+	if (dev_is_pci(gc->dev))
+		prot = pgprot_writecombine(vma->vm_page_prot);
+	else
+		prot = pgprot_device(vma->vm_page_prot);
 
 	ret = rdma_user_mmap_io(ibcontext, vma, pfn, PAGE_SIZE, prot,
 				NULL);
@@ -623,14 +626,13 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
 			 struct ib_udata *uhw)
 {
 	struct mana_ib_dev *dev = container_of(ibdev, struct mana_ib_dev, ib_dev);
-	struct pci_dev *pdev = to_pci_dev(mdev_to_gc(dev)->dev);
 	int err;
 
 	err = ib_no_udata_io(uhw);
 	if (err)
 		return err;
 
-	props->vendor_id = pdev->vendor;
+	props->vendor_id = PCI_VENDOR_ID_MICROSOFT;
 	props->vendor_part_id = dev->gdma_dev->dev_id.type;
 	props->max_mr_size = MANA_IB_MAX_MR_SIZE;
 	props->page_size_cap = dev->adapter_caps.page_size_cap;

base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.43.0

Re: [PATCH rdma-next] RDMA/mana_ib: Enable RDMA devices on the CDX bus

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] RDMA/mana_ib: NULL dma_parms in mana_ib_probe() on CDX devices
--

commit 4f88c8811c923b9c16d7e0dbf8674249f05b4f08
Author: Zhiyue Qiu <zhiyueqiu@microsoft.com>

RDMA/mana_ib: Enable RDMA devices on the CDX bus

This commit enables MANA RDMA on the CDX bus by depending on the shared GDMA
core, matching CDX auxiliary devices, mapping non-PCI userspace doorbells
with pgprot_device(), and reporting the Microsoft vendor ID directly.

> diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c
> index 62c7803ca6c60..33e47490755f6 100644
> --- a/drivers/infiniband/hw/mana/device.c
> +++ b/drivers/infiniband/hw/mana/device.c
> @@ -257,6 +257,8 @@ static void mana_ib_remove(struct auxiliary_device *adev)
>  static const struct auxiliary_device_id mana_id_table[] = {
>  	{ .name = "mana.rdma", },
>  	{ .name = "mana.eth", },
> +	{ .name = "mana_cdx.rdma", },
> +	{ .name = "mana_cdx.eth", },

[Severity: High]
When mana_ib_probe() binds to mana_cdx.rdma or mana_cdx.eth, does passing
gc->dev to ib_register_device() trigger a warning and fall back to a 64 KiB
DMA segment size limit because gc->dev->dma_parms is NULL?

In drivers/infiniband/hw/mana/device.c, mana_ib_probe() registers the IB
device with gc->dev as the dma_device:

	ret = ib_register_device(&dev->ib_dev, mana_ib_is_rnic(dev) ? "mana_%d" : "manae_%d",
				 gc->dev);

And in drivers/infiniband/core/device.c, ib_register_device() checks:

	WARN_ON(dma_device && !dma_device->dma_parms);
	device->dma_device = dma_device;

Unlike struct pci_dev, struct cdx_device does not attach a dma_parms
structure, which is noted in mana_cdx_setup_hwc_irqs() in
drivers/net/ethernet/microsoft/mana/gdma_cdx.c:

	/* The PCI path also calls dma_set_max_seg_size(), but that is
	 * deliberately omitted here: the CDX core does not attach a
	 * dma_parms to the device, so the helper would only WARN, and MANA
	 * never builds scatter-gather lists, so the limit is never read.
	 */

In addition to triggering the warning in ib_register_device() on probe,
mana_ib_create_queue() and mana_ib_reg_user_mr() build scatter-gather lists
via ib_umem_get_cq_buf_or_va() and ib_umem_get_va(), where
__ib_umem_get_va() calls ib_dma_max_seg_size() -> dma_get_max_seg_size() and
falls back to SZ_64K instead of UINT_MAX when dma_parms is NULL.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103115032.736554-1-kotaranov@linux.microsoft.com?part=1

Re: [PATCH rdma-next] RDMA/mana_ib: Enable RDMA devices on the CDX bus

[Leon Romanovsky]

On Tue, Nov 03, 2026 at 12:05:41PM +0000, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] RDMA/mana_ib: NULL dma_parms in mana_ib_probe() on CDX devices
> --
> 
> commit 4f88c8811c923b9c16d7e0dbf8674249f05b4f08
> Author: Zhiyue Qiu <zhiyueqiu@microsoft.com>
> 
> RDMA/mana_ib: Enable RDMA devices on the CDX bus
> 
> This commit enables MANA RDMA on the CDX bus by depending on the shared GDMA
> core, matching CDX auxiliary devices, mapping non-PCI userspace doorbells
> with pgprot_device(), and reporting the Microsoft vendor ID directly.
> 
> > diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c
> > index 62c7803ca6c60..33e47490755f6 100644
> > --- a/drivers/infiniband/hw/mana/device.c
> > +++ b/drivers/infiniband/hw/mana/device.c
> > @@ -257,6 +257,8 @@ static void mana_ib_remove(struct auxiliary_device *adev)
> >  static const struct auxiliary_device_id mana_id_table[] = {
> >  	{ .name = "mana.rdma", },
> >  	{ .name = "mana.eth", },
> > +	{ .name = "mana_cdx.rdma", },
> > +	{ .name = "mana_cdx.eth", },
> 
> [Severity: High]
> When mana_ib_probe() binds to mana_cdx.rdma or mana_cdx.eth, does passing
> gc->dev to ib_register_device() trigger a warning and fall back to a 64 KiB
> DMA segment size limit because gc->dev->dma_parms is NULL?
> 
> In drivers/infiniband/hw/mana/device.c, mana_ib_probe() registers the IB
> device with gc->dev as the dma_device:
> 
> 	ret = ib_register_device(&dev->ib_dev, mana_ib_is_rnic(dev) ? "mana_%d" : "manae_%d",
> 				 gc->dev);
> 
> And in drivers/infiniband/core/device.c, ib_register_device() checks:
> 
> 	WARN_ON(dma_device && !dma_device->dma_parms);
> 	device->dma_device = dma_device;
> 
> Unlike struct pci_dev, struct cdx_device does not attach a dma_parms
> structure, which is noted in mana_cdx_setup_hwc_irqs() in
> drivers/net/ethernet/microsoft/mana/gdma_cdx.c:
> 
> 	/* The PCI path also calls dma_set_max_seg_size(), but that is
> 	 * deliberately omitted here: the CDX core does not attach a
> 	 * dma_parms to the device, so the helper would only WARN, and MANA
> 	 * never builds scatter-gather lists, so the limit is never read.
> 	 */
> 
> In addition to triggering the warning in ib_register_device() on probe,
> mana_ib_create_queue() and mana_ib_reg_user_mr() build scatter-gather lists
> via ib_umem_get_cq_buf_or_va() and ib_umem_get_va(), where
> __ib_umem_get_va() calls ib_dma_max_seg_size() -> dma_get_max_seg_size() and
> falls back to SZ_64K instead of UINT_MAX when dma_parms is NULL.

The analysis looks correct to me. How was it tested?

Thanks

> 
> -- 
> Sashiko AI review · https://sashiko.dev/#/patchset/20261103115032.736554-1-kotaranov@linux.microsoft.com?part=1
>