Re: [PATCH net-next 0/2] net: mana: Avoid DMA queue allocation failure under memory fragmentation

[Aditya Garg <gargaditya@linux.microsoft.com>]

On 13-08-2026 17:12, Paolo Abeni wrote:
> On 8/7/26 10:56 PM, Aditya Garg wrote:
>> The MANA driver can fail to bring up its queues on systems with high
>> memory utilization because every GDMA queue ring is allocated as a
>> single dma_alloc_coherent() of the whole power-of-2 ring size. Under
>> memory fragmentation these high-order allocations may fail, preventing
>> the driver from creating queues when opening the interface, after a VF
>> reset, or when reconfiguring channels, ring parameters or MTU.
>>
>> Per-queue sizes that are problematic, with depth and size given as
>> (default, max) over the ethtool ring settings:
>>
>>    ring                  entry  depth          size
>>    ------------------------------------------------------------
>>    TX completion queue   64 B   (256, 16384)   (16 KB, 1024 KB)
>>    TX send queue         32 B   (256, 16384)   ( 8 KB,  512 KB)
>>    RX completion queue   64 B   (1024, 8192)   (64 KB,  512 KB)
>>    RX receive queue      32 B   (1024, 8192)   (32 KB,  256 KB)
>>    event queue           16 B   2048 (fixed)   32 KB
>>
>> This series addresses the issue by:
>>    1. Routing all CPU-side ring access through mana_gd_ring_ptr() and
>>       mana_gd_ring_contig_avail(). On a contiguous ring these reduce to
>>       simple arithmetic, so this patch is a pure refactor.
>>    2. Falling back in mana_gd_alloc_memory() to a vector of scattered
>>       order-0 coherent pages when the contiguous allocation fails. The
>>       device sees the same page-list format either way, as
>>       mana_gd_create_dma_region() already describes a ring as a list of
>>       MANA_PAGE_SIZE addresses. The HW channel stays contiguous, as
>>       advertising a scattered page list needs the HW channel itself.
>>
>> Throughput testing confirms no regression. Since the fallback only
>> triggers under memory fragmentation, the scattered-page path was enabled
>> unconditionally for all eligible GDMA queue rings during testing (iperf3,
>> Gbit/s):
>>
>>                   Baseline    Patched     Patched
>>    Connections   Contiguous  Contiguous  Scattered
>>    -----------------------------------------------
>>    1                  46.1        46.2       46.1
>>    16                 182         182        182
>>    32                 182         182        182
>>    64                 182         182        182
> Given the above, I suggest to follow-up and remove entirely the single
> dma_alloc_coherent() allocation; you will get simpler code and less
> pressure for later dma users.
> 
> /P

Thanks for the review, Paolo!

I got these throughput numbers from an idle system running only iperf3, 
so we're not sure whether they would remain the same with other 
workloads running in parallel. I'll try to get these numbers as well.

Also, our hardware team indicated that this fallback leads to additional 
resource consumption on their side. I'll check with them and post a 
follow-up.

Regards,
Aditya