From: Konstantin Taranov <kotaranov@linux.microsoft.com>
To: kotaranov@microsoft.com, snsanghvi@microsoft.com,
longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org
Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH rdma-next v2] RDMA/mana_ib: Enable RDMA devices on the CDX bus
Date: Tue, 17 Nov 2026 10:20:43 -0800 [thread overview]
Message-ID: <20261117182043.149267-1-kotaranov@linux.microsoft.com> (raw)
From: Zhiyue Qiu <zhiyueqiu@microsoft.com>
Enable MANA RDMA on CDX by depending on the shared GDMA core and matching
the CDX auxiliary devices.
Map non-PCI userspace doorbells with pgprot_device() to match the CDX
transport, and report the Microsoft vendor ID directly.
Allocate missing DMA parameters before GDMA setup using managed memory.
Register a cleanup action to clear the device pointer before the memory
is freed.
Signed-off-by: Zhiyue Qiu <zhiyueqiu@microsoft.com>
Co-developed-by: Konstantin Taranov <kotaranov@microsoft.com>
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
---
Changes in v2:
- Allocate missing CDX DMA parameters before GDMA setup.
drivers/infiniband/hw/mana/Kconfig | 10 +++----
drivers/infiniband/hw/mana/device.c | 2 ++
drivers/infiniband/hw/mana/main.c | 9 ++++--
.../net/ethernet/microsoft/mana/gdma_cdx.c | 30 +++++++++++++++----
4 files changed, 38 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/hw/mana/Kconfig b/drivers/infiniband/hw/mana/Kconfig
index 546640657bac..6eae7fa91d95 100644
--- a/drivers/infiniband/hw/mana/Kconfig
+++ b/drivers/infiniband/hw/mana/Kconfig
@@ -1,10 +1,10 @@
# SPDX-License-Identifier: GPL-2.0-only
config MANA_INFINIBAND
tristate "Microsoft Azure Network Adapter support"
- depends on NETDEVICES && ETHERNET && PCI && MICROSOFT_MANA
+ depends on NETDEVICES && ETHERNET && MICROSOFT_GDMA_CORE
help
This driver provides low-level RDMA support for Microsoft Azure
- Network Adapter (MANA). MANA supports RDMA features that can be used
- for workloads (e.g. DPDK, MPI etc) that uses RDMA verbs to directly
- access hardware from user-mode processes in Microsoft Azure cloud
- environment.
+ Network Adapter (MANA) on the PCI or CDX bus. MANA supports RDMA
+ features that workloads such as DPDK and MPI can use to directly
+ access hardware from user-mode processes in the Microsoft Azure
+ cloud environment.
diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c
index 62c7803ca6c6..33e47490755f 100644
--- a/drivers/infiniband/hw/mana/device.c
+++ b/drivers/infiniband/hw/mana/device.c
@@ -257,6 +257,8 @@ static void mana_ib_remove(struct auxiliary_device *adev)
static const struct auxiliary_device_id mana_id_table[] = {
{ .name = "mana.rdma", },
{ .name = "mana.eth", },
+ { .name = "mana_cdx.rdma", },
+ { .name = "mana_cdx.eth", },
{},
};
diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
index 858bbc325eb2..afb8fa002f73 100644
--- a/drivers/infiniband/hw/mana/main.c
+++ b/drivers/infiniband/hw/mana/main.c
@@ -572,7 +572,11 @@ int mana_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)
pfn = (gc->phys_db_page_base +
gc->db_page_size * mana_ucontext->doorbell) >>
PAGE_SHIFT;
- prot = pgprot_writecombine(vma->vm_page_prot);
+ /* Match the Device attributes used by the CDX transport. */
+ if (dev_is_pci(gc->dev))
+ prot = pgprot_writecombine(vma->vm_page_prot);
+ else
+ prot = pgprot_device(vma->vm_page_prot);
ret = rdma_user_mmap_io(ibcontext, vma, pfn, PAGE_SIZE, prot,
NULL);
@@ -623,14 +627,13 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
struct ib_udata *uhw)
{
struct mana_ib_dev *dev = container_of(ibdev, struct mana_ib_dev, ib_dev);
- struct pci_dev *pdev = to_pci_dev(mdev_to_gc(dev)->dev);
int err;
err = ib_no_udata_io(uhw);
if (err)
return err;
- props->vendor_id = pdev->vendor;
+ props->vendor_id = PCI_VENDOR_ID_MICROSOFT;
props->vendor_part_id = dev->gdma_dev->dev_id.type;
props->max_mr_size = MANA_IB_MAX_MR_SIZE;
props->page_size_cap = dev->adapter_caps.page_size_cap;
diff --git a/drivers/net/ethernet/microsoft/mana/gdma_cdx.c b/drivers/net/ethernet/microsoft/mana/gdma_cdx.c
index 0d01e3d2f0d9..6c6ddc14ef7e 100644
--- a/drivers/net/ethernet/microsoft/mana/gdma_cdx.c
+++ b/drivers/net/ethernet/microsoft/mana/gdma_cdx.c
@@ -3,6 +3,7 @@
#include <linux/module.h>
#include <linux/bitmap.h>
#include <linux/cdx/cdx_bus.h>
+#include <linux/device.h>
#include <linux/msi.h>
#include <linux/xarray.h>
@@ -61,11 +62,8 @@ static int mana_cdx_setup_hwc_irqs(struct gdma_context *gc)
return err;
}
- /* The PCI path also calls dma_set_max_seg_size(), but that is
- * deliberately omitted here: the CDX core does not attach a
- * dma_parms to the device, so the helper would only WARN, and MANA
- * never builds scatter-gather lists, so the limit is never read.
- */
+ dma_set_max_seg_size(gc->dev, UINT_MAX);
+ dma_set_seg_boundary(gc->dev, DMA_BIT_MASK(32));
err = cdx_enable_msi(cdx_dev);
if (err) {
@@ -196,6 +194,13 @@ static const struct gdma_bus_ops mana_cdx_bus_ops = {
.dev_reset = mana_cdx_dev_reset,
};
+static void mana_cdx_clear_dma_parms(void *data)
+{
+ struct device *dev = data;
+
+ dev->dma_parms = NULL;
+}
+
static int mana_cdx_gd_probe(struct cdx_device *cdx_dev)
{
struct gdma_context *gc;
@@ -241,6 +246,21 @@ static int mana_cdx_gd_probe(struct cdx_device *cdx_dev)
gc->bar0_va = bar0_va;
gc->dev = &cdx_dev->dev;
gc->bus_ops = &mana_cdx_bus_ops;
+ if (!gc->dev->dma_parms) {
+ gc->dev->dma_parms = devm_kzalloc(gc->dev,
+ sizeof(*gc->dev->dma_parms),
+ GFP_KERNEL);
+ if (!gc->dev->dma_parms) {
+ err = -ENOMEM;
+ goto unmap_bar;
+ }
+
+ /* Clear the pointer before devres frees the allocation. */
+ err = devm_add_action_or_reset(gc->dev, mana_cdx_clear_dma_parms,
+ gc->dev);
+ if (err)
+ goto unmap_bar;
+ }
/* gc->is_pf is intentionally left clear: this device uses the VF
* register layout, and the vPort and MAC filter registration that
--
2.43.0
next reply other threads:[~2026-11-17 18:20 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 18:20 Konstantin Taranov [this message]
2026-11-17 18:36 ` [PATCH rdma-next v2] RDMA/mana_ib: Enable RDMA devices on the CDX bus sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261117182043.149267-1-kotaranov@linux.microsoft.com \
--to=kotaranov@linux.microsoft.com \
--cc=jgg@ziepe.ca \
--cc=kotaranov@microsoft.com \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=snsanghvi@microsoft.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.