The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Aditya Garg <gargaditya@linux.microsoft.com>
To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
	decui@microsoft.com, longli@microsoft.com, andrew+netdev@lunn.ch,
	davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
	pabeni@redhat.com, kotaranov@microsoft.com, horms@kernel.org,
	ernis@linux.microsoft.com, dipayanroy@linux.microsoft.com,
	shradhagupta@linux.microsoft.com, kees@kernel.org,
	sgeorgejohn@microsoft.com, ssengar@linux.microsoft.com,
	gargaditya@linux.microsoft.com, gargaditya@microsoft.com,
	linux-hyperv@vger.kernel.org, netdev@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org
Subject: [PATCH net-next 0/2] net: mana: Avoid DMA queue allocation failure under memory fragmentation
Date: Fri,  7 Aug 2026 13:56:34 -0700	[thread overview]
Message-ID: <20260807210002.1695263-1-gargaditya@linux.microsoft.com> (raw)

The MANA driver can fail to bring up its queues on systems with high
memory utilization because every GDMA queue ring is allocated as a
single dma_alloc_coherent() of the whole power-of-2 ring size. Under
memory fragmentation these high-order allocations may fail, preventing
the driver from creating queues when opening the interface, after a VF
reset, or when reconfiguring channels, ring parameters or MTU.

Per-queue sizes that are problematic, with depth and size given as
(default, max) over the ethtool ring settings:

  ring                  entry  depth          size
  ------------------------------------------------------------
  TX completion queue   64 B   (256, 16384)   (16 KB, 1024 KB)
  TX send queue         32 B   (256, 16384)   ( 8 KB,  512 KB)
  RX completion queue   64 B   (1024, 8192)   (64 KB,  512 KB)
  RX receive queue      32 B   (1024, 8192)   (32 KB,  256 KB)
  event queue           16 B   2048 (fixed)   32 KB

This series addresses the issue by:
  1. Routing all CPU-side ring access through mana_gd_ring_ptr() and
     mana_gd_ring_contig_avail(). On a contiguous ring these reduce to
     simple arithmetic, so this patch is a pure refactor.
  2. Falling back in mana_gd_alloc_memory() to a vector of scattered
     order-0 coherent pages when the contiguous allocation fails. The
     device sees the same page-list format either way, as
     mana_gd_create_dma_region() already describes a ring as a list of
     MANA_PAGE_SIZE addresses. The HW channel stays contiguous, as
     advertising a scattered page list needs the HW channel itself.

Throughput testing confirms no regression. Since the fallback only
triggers under memory fragmentation, the scattered-page path was enabled
unconditionally for all eligible GDMA queue rings during testing (iperf3,
Gbit/s):

                 Baseline    Patched     Patched
  Connections   Contiguous  Contiguous  Scattered
  -----------------------------------------------
  1                  46.1        46.2       46.1
  16                 182         182        182
  32                 182         182        182
  64                 182         182        182

Aditya Garg (2):
  net: mana: Route ring-buffer access through offset-based helpers
  net: mana: Fall back to scattered pages for GDMA queues

 .../net/ethernet/microsoft/mana/gdma_main.c   | 219 +++++++++++++++---
 .../net/ethernet/microsoft/mana/hw_channel.c  |   2 +-
 drivers/net/ethernet/microsoft/mana/mana_en.c |   3 +
 include/net/mana/gdma.h                       |  19 +-
 4 files changed, 205 insertions(+), 38 deletions(-)

-- 
2.43.0


             reply	other threads:[~2026-08-07 21:00 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-07 20:56 Aditya Garg [this message]
2026-08-07 20:56 ` [PATCH net-next 1/2] net: mana: Route ring-buffer access through offset-based helpers Aditya Garg
2026-08-07 20:56 ` [PATCH net-next 2/2] net: mana: Fall back to scattered pages for GDMA queues Aditya Garg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260807210002.1695263-1-gargaditya@linux.microsoft.com \
    --to=gargaditya@linux.microsoft.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=decui@microsoft.com \
    --cc=dipayanroy@linux.microsoft.com \
    --cc=edumazet@google.com \
    --cc=ernis@linux.microsoft.com \
    --cc=gargaditya@microsoft.com \
    --cc=haiyangz@microsoft.com \
    --cc=horms@kernel.org \
    --cc=kees@kernel.org \
    --cc=kotaranov@microsoft.com \
    --cc=kuba@kernel.org \
    --cc=kys@microsoft.com \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=longli@microsoft.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=sgeorgejohn@microsoft.com \
    --cc=shradhagupta@linux.microsoft.com \
    --cc=ssengar@linux.microsoft.com \
    --cc=wei.liu@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox