Linux RDMA and InfiniBand development
 help / color / mirror / Atom feed
From: Leon Romanovsky <leon@kernel.org>
To: Wei Hu <weh@linux.microsoft.com>
Cc: longli@kernel.org, kotaranov@microsoft.com, kuba@kernel.org,
	davem@davemloft.net, pabeni@redhat.com, edumazet@google.com,
	andrew+netdev@lunn.ch, jgg@ziepe.ca, haiyangz@microsoft.com,
	wei.liu@kernel.org, decui@microsoft.com,
	shradhagupta@linux.microsoft.com, horms@kernel.org,
	ernis@linux.microsoft.com, stephen@networkplumber.org,
	netdev@vger.kernel.org, linux-rdma@vger.kernel.org,
	linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
	gargaditya@linux.microsoft.com, kees+treewide@kernel.org,
	kees@kernel.org, mawasthi@linux.microsoft.com, weh@microsoft.com,
	edumazet@kernel.org, shacharr@microsoft.com
Subject: Re: [PATCH net-next v11 4/4] net: mana: add dynamic HWC queue depth with reinit path
Date: Tue, 17 Nov 2026 20:09:31 +0200	[thread overview]
Message-ID: <20261117180931.GE162713@unreal> (raw)
In-Reply-To: <100c6b7a71e86c8d7afd0c55f6b231f01db184d5.1794751992.git.weh@linux.microsoft.com>

On Tue, Nov 17, 2026 at 04:01:01PM +0000, Wei Hu wrote:
> From: Long Li <longli@microsoft.com>
> 
> The HWC starts with depth-one bootstrap queues. Treat the PF-reported depth
> as a maximum and rebuild the channel exactly for reports from 2 through
> 128. Keep the untouched bootstrap queues for depth one and reports above
> 128, and require a rebuilt channel to repeat the selected depth.
> 
> Treat the reported request and response sizes as directional maxima.
> Rebuild only when both reports equal the 4096-byte bootstrap allocation.
> Keep allocation strides separate from these limits, and share the effective
> host-to-PF request budget with the GDMA and RDMA request builders. Bound
> RDMA CREATE and ADD_PAGES batches by that budget, checking that the headers
> and at least one page address fit before subtracting their sizes.
> 
> Require nonzero depth, request-size and response-size reports before
> deciding whether to keep the bootstrap queues. Apply the same validation
> to a restored bootstrap channel. A missing or zero required report fails
> with -EPROTO, even at depth one; do not silently substitute guessed limits.
> 
> Reject oversized messages before acquiring a slot or publishing caller
> storage. A retained, stopped channel still returns -ENODEV rather than
> reporting a size error against uninitialized negotiated limits.
> 
> Keep each candidate private through queue creation, establishment and the
> direct EQ test. Publish only the final tested channel. If a rebuild fails,
> including its functional test, tear it down and restore fresh, tested
> bootstrap queues. Stop if teardown is not acknowledged; never reuse
> PF-owned mappings. Restore the previous live timeout budget before the
> fallback so a failed test's fail-fast reduction does not poison the retry,
> without reviving terminal cancellation.
> 
> Invalidate the previous HWC doorbell before every ESTABLISH_HWC. Validate
> the BAR0 page geometry and reported index before accepting a new doorbell,
> and suppress MMIO while it is invalid. The original driver accepted an
> unchecked device-supplied index in its MMIO address calculation; this also
> fixes that existing out-of-bounds MMIO path. Backports need the complete
> ownership and validation dependencies, not the entire feature stack by
> default. A malformed doorbell completes initialization with -EPROTO without
> relying on later INIT_DONE events that might require re-arming through that
> doorbell.
> 
> Use the initial PF report to select the queue geometry. The PF directs
> this choice; the absence of VF size fields in ESTABLISH_HWC does not require
> keeping the bootstrap geometry. Driver-version advertisement occurs after
> HWC establishment and cannot negotiate behavior needed during creation.
> 
> Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
> Link: https://lore.kernel.org/r/20260908035201.402424-5-longli@microsoft.com
> Link: https://lore.kernel.org/r/179196575846.434549.7933818740976196422@kernel.org
> Link: https://lore.kernel.org/r/179455240227.2199510.2666165032389090040@kernel.org
> Signed-off-by: Long Li <longli@microsoft.com>
> Signed-off-by: Wei Hu <weh@microsoft.com>
> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
> Copilot-Session: 8c2361e6-c0f4-4b0c-bf01-b5e4cdacec50
> ---
>  drivers/infiniband/hw/mana/main.c             |  19 +-
>  .../net/ethernet/microsoft/mana/gdma_main.c   |  26 +-
>  .../net/ethernet/microsoft/mana/hw_channel.c  | 316 +++++++++++++++---
>  include/net/mana/gdma.h                       |   2 +
>  include/net/mana/hw_channel.h                 |  19 +-
>  5 files changed, 317 insertions(+), 65 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c
> index 7d5a8fc7380d..6e036a25a9f7 100644
> --- a/drivers/infiniband/hw/mana/main.c
> +++ b/drivers/infiniband/hw/mana/main.c
> @@ -427,28 +427,38 @@ static int mana_ib_gd_create_dma_region(struct mana_ib_dev *dev, struct ib_umem
>  	unsigned int create_req_msg_size;
>  	struct hw_channel_context *hwc;
>  	struct ib_block_iter biter;
> -	size_t max_pgs_add_cmd = 0;
> +	size_t max_pgs_add_cmd;
>  	size_t max_pgs_create_cmd;
>  	struct gdma_context *gc;
>  	size_t num_pages_total;
>  	unsigned int tail = 0;
> +	u32 req_limit;
>  	u64 *page_addr_list;
>  	void *request_buf;
>  	int err = 0;
>  
>  	gc = mdev_to_gc(dev);
>  	hwc = gc->hwc.driver_data;
> +	req_limit = mana_hwc_get_max_req_size(hwc);
> +	if (req_limit < struct_size(create_req, page_addr_list, 1) ||
> +	    req_limit < struct_size(add_req, page_addr_list, 1)) {
> +		ibdev_dbg(&dev->ib_dev,
> +			  "HWC request budget %u is too small for DMA region commands\n",
> +			  req_limit);
> +		return -EMSGSIZE;
> +	}

No. Make sure you also call mana_ib_gd_create_dma_region() with valid parameters.

Thanks

  reply	other threads:[~2026-11-17 18:09 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 16:00 [PATCH net-next v11 0/4] net: mana: concurrent HWC requests and dynamic queue depth Wei Hu
2026-11-17 16:00 ` [PATCH net-next v11 1/4] net: mana: prepare HWC ownership for safe reinitialization Wei Hu
2026-11-17 16:00 ` [PATCH net-next v11 2/4] net: mana: give each HWC message slot its own completion state Wei Hu
2026-11-17 16:01 ` [PATCH net-next v11 3/4] net: mana: support concurrent HWC requests Wei Hu
2026-11-17 18:07   ` Leon Romanovsky
2026-11-17 16:01 ` [PATCH net-next v11 4/4] net: mana: add dynamic HWC queue depth with reinit path Wei Hu
2026-11-17 18:09   ` Leon Romanovsky [this message]
2026-11-17 16:03 ` [PATCH net-next v11 0/4] net: mana: concurrent HWC requests and dynamic queue depth netdev-bot+sinfo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261117180931.GE162713@unreal \
    --to=leon@kernel.org \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=decui@microsoft.com \
    --cc=edumazet@google.com \
    --cc=edumazet@kernel.org \
    --cc=ernis@linux.microsoft.com \
    --cc=gargaditya@linux.microsoft.com \
    --cc=haiyangz@microsoft.com \
    --cc=horms@kernel.org \
    --cc=jgg@ziepe.ca \
    --cc=kees+treewide@kernel.org \
    --cc=kees@kernel.org \
    --cc=kotaranov@microsoft.com \
    --cc=kuba@kernel.org \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=longli@kernel.org \
    --cc=mawasthi@linux.microsoft.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=shacharr@microsoft.com \
    --cc=shradhagupta@linux.microsoft.com \
    --cc=stephen@networkplumber.org \
    --cc=weh@linux.microsoft.com \
    --cc=weh@microsoft.com \
    --cc=wei.liu@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox