Linux RDMA and InfiniBand development
 help / color / mirror / Atom feed
From: Leon Romanovsky <leon@kernel.org>
To: Konstantin Taranov <kotaranov@linux.microsoft.com>
Cc: kotaranov@microsoft.com, snsanghvi@microsoft.com,
	longli@microsoft.com, jgg@ziepe.ca, linux-rdma@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH rdma-next v2] RDMA/mana_ib: Enable RDMA devices on the CDX bus
Date: Sun, 22 Nov 2026 14:16:23 +0200	[thread overview]
Message-ID: <20261122121623.GC14811@unreal> (raw)
In-Reply-To: <20261117182043.149267-1-kotaranov@linux.microsoft.com>

On Tue, Nov 17, 2026 at 10:20:43AM -0800, Konstantin Taranov wrote:
> From: Zhiyue Qiu <zhiyueqiu@microsoft.com>
> 
> Enable MANA RDMA on CDX by depending on the shared GDMA core and matching
> the CDX auxiliary devices.
> 
> Map non-PCI userspace doorbells with pgprot_device() to match the CDX
> transport, and report the Microsoft vendor ID directly.
> 
> Allocate missing DMA parameters before GDMA setup using managed memory.
> Register a cleanup action to clear the device pointer before the memory
> is freed.

The commit message merely lists the code changes in the patch. It should
explain why you are using pgprot_device() and why you report Microsoft
vendor ID directly.

> 
> Signed-off-by: Zhiyue Qiu <zhiyueqiu@microsoft.com>
> Co-developed-by: Konstantin Taranov <kotaranov@microsoft.com>
> Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
> ---
> Changes in v2:
> - Allocate missing CDX DMA parameters before GDMA setup.
> 
>  drivers/infiniband/hw/mana/Kconfig            | 10 +++----
>  drivers/infiniband/hw/mana/device.c           |  2 ++
>  drivers/infiniband/hw/mana/main.c             |  9 ++++--
>  .../net/ethernet/microsoft/mana/gdma_cdx.c    | 30 +++++++++++++++----
>  4 files changed, 38 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/mana/Kconfig b/drivers/infiniband/hw/mana/Kconfig
> index 546640657bac..6eae7fa91d95 100644
> --- a/drivers/infiniband/hw/mana/Kconfig
> +++ b/drivers/infiniband/hw/mana/Kconfig
> @@ -1,10 +1,10 @@
>  # SPDX-License-Identifier: GPL-2.0-only
>  config MANA_INFINIBAND
>  	tristate "Microsoft Azure Network Adapter support"
> -	depends on NETDEVICES && ETHERNET && PCI && MICROSOFT_MANA
> +	depends on NETDEVICES && ETHERNET && MICROSOFT_GDMA_CORE
>  	help
>  	  This driver provides low-level RDMA support for Microsoft Azure
> -	  Network Adapter (MANA). MANA supports RDMA features that can be used
> -	  for workloads (e.g. DPDK, MPI etc) that uses RDMA verbs to directly
> -	  access hardware from user-mode processes in Microsoft Azure cloud
> -	  environment.
> +	  Network Adapter (MANA) on the PCI or CDX bus. MANA supports RDMA
> +	  features that workloads such as DPDK and MPI can use to directly
> +	  access hardware from user-mode processes in the Microsoft Azure
> +	  cloud environment.

You should rely less on AI and review its output more carefully. The
only point you needed to add was that MAND works on both PCI and CDX buses.
Instead, you rewrote the entire text. The original text was not limited to
PCI, so all of the changes above are unnecessary.

> diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c
> index 62c7803ca6c6..33e47490755f 100644
> --- a/drivers/infiniband/hw/mana/device.c
> +++ b/drivers/infiniband/hw/mana/device.c
> @@ -257,6 +257,8 @@ static void mana_ib_remove(struct auxiliary_device *adev)
>  static const struct auxiliary_device_id mana_id_table[] = {
>  	{ .name = "mana.rdma", },
>  	{ .name = "mana.eth", },
> +	{ .name = "mana_cdx.rdma", },
> +	{ .name = "mana_cdx.eth", },
>  	{},
>  };
>  
> diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
> index 858bbc325eb2..afb8fa002f73 100644
> --- a/drivers/infiniband/hw/mana/main.c
> +++ b/drivers/infiniband/hw/mana/main.c
> @@ -572,7 +572,11 @@ int mana_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)
>  	pfn = (gc->phys_db_page_base +
>  	       gc->db_page_size * mana_ucontext->doorbell) >>
>  	      PAGE_SHIFT;
> -	prot = pgprot_writecombine(vma->vm_page_prot);
> +	/* Match the Device attributes used by the CDX transport. */
> +	if (dev_is_pci(gc->dev))
> +		prot = pgprot_writecombine(vma->vm_page_prot);
> +	else
> +		prot = pgprot_device(vma->vm_page_prot);
>  
>  	ret = rdma_user_mmap_io(ibcontext, vma, pfn, PAGE_SIZE, prot,
>  				NULL);
> @@ -623,14 +627,13 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props,
>  			 struct ib_udata *uhw)
>  {
>  	struct mana_ib_dev *dev = container_of(ibdev, struct mana_ib_dev, ib_dev);
> -	struct pci_dev *pdev = to_pci_dev(mdev_to_gc(dev)->dev);
>  	int err;
>  
>  	err = ib_no_udata_io(uhw);
>  	if (err)
>  		return err;
>  
> -	props->vendor_id = pdev->vendor;
> +	props->vendor_id = PCI_VENDOR_ID_MICROSOFT;
>  	props->vendor_part_id = dev->gdma_dev->dev_id.type;
>  	props->max_mr_size = MANA_IB_MAX_MR_SIZE;
>  	props->page_size_cap = dev->adapter_caps.page_size_cap;
> diff --git a/drivers/net/ethernet/microsoft/mana/gdma_cdx.c b/drivers/net/ethernet/microsoft/mana/gdma_cdx.c
> index 0d01e3d2f0d9..6c6ddc14ef7e 100644
> --- a/drivers/net/ethernet/microsoft/mana/gdma_cdx.c
> +++ b/drivers/net/ethernet/microsoft/mana/gdma_cdx.c
> @@ -3,6 +3,7 @@
>  #include <linux/module.h>
>  #include <linux/bitmap.h>
>  #include <linux/cdx/cdx_bus.h>
> +#include <linux/device.h>
>  #include <linux/msi.h>
>  #include <linux/xarray.h>
>  
> @@ -61,11 +62,8 @@ static int mana_cdx_setup_hwc_irqs(struct gdma_context *gc)
>  		return err;
>  	}
>  
> -	/* The PCI path also calls dma_set_max_seg_size(), but that is
> -	 * deliberately omitted here: the CDX core does not attach a
> -	 * dma_parms to the device, so the helper would only WARN, and MANA
> -	 * never builds scatter-gather lists, so the limit is never read.
> -	 */
> +	dma_set_max_seg_size(gc->dev, UINT_MAX);
> +	dma_set_seg_boundary(gc->dev, DMA_BIT_MASK(32));

PCI code declares almost same code in probe function and not in mana_XXX_setup_hwc_irqs().

>  
>  	err = cdx_enable_msi(cdx_dev);
>  	if (err) {
> @@ -196,6 +194,13 @@ static const struct gdma_bus_ops mana_cdx_bus_ops = {
>  	.dev_reset		= mana_cdx_dev_reset,
>  };
>  
> +static void mana_cdx_clear_dma_parms(void *data)
> +{
> +	struct device *dev = data;
> +
> +	dev->dma_parms = NULL;

Will you reuse dev after the device has been released?

> +}
> +
>  static int mana_cdx_gd_probe(struct cdx_device *cdx_dev)
>  {
>  	struct gdma_context *gc;
> @@ -241,6 +246,21 @@ static int mana_cdx_gd_probe(struct cdx_device *cdx_dev)
>  	gc->bar0_va = bar0_va;
>  	gc->dev = &cdx_dev->dev;
>  	gc->bus_ops = &mana_cdx_bus_ops;
> +	if (!gc->dev->dma_parms) {
> +		gc->dev->dma_parms = devm_kzalloc(gc->dev,
> +						  sizeof(*gc->dev->dma_parms),
> +						  GFP_KERNEL);
> +		if (!gc->dev->dma_parms) {
> +			err = -ENOMEM;
> +			goto unmap_bar;
> +		}
> +
> +		/* Clear the pointer before devres frees the allocation. */
> +		err = devm_add_action_or_reset(gc->dev, mana_cdx_clear_dma_parms,
> +					       gc->dev);
> +		if (err)
> +			goto unmap_bar;

Why do you need devm_XXX interfaces if device can't be accessed after it
is released?

Thanks

> +	}
>  
>  	/* gc->is_pf is intentionally left clear: this device uses the VF
>  	 * register layout, and the vPort and MAC filter registration that
> -- 
> 2.43.0
> 

      parent reply	other threads:[~2026-11-22 12:16 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 18:20 [PATCH rdma-next v2] RDMA/mana_ib: Enable RDMA devices on the CDX bus Konstantin Taranov
2026-11-17 18:36 ` sashiko-bot
2026-11-22 12:16 ` Leon Romanovsky [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261122121623.GC14811@unreal \
    --to=leon@kernel.org \
    --cc=jgg@ziepe.ca \
    --cc=kotaranov@linux.microsoft.com \
    --cc=kotaranov@microsoft.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=longli@microsoft.com \
    --cc=snsanghvi@microsoft.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox