[PATCH for-next] RDMA/efa: Use __GFP_RETRY_MAYFAIL for ring allocations

[PATCH for-next] RDMA/efa: Use __GFP_RETRY_MAYFAIL for ring allocations

[Michael Margolin]

RQ and CQ ring buffers require physically contiguous memory. On
systems under memory pressure, high-order allocations can fail
even when sufficient free memory exists because the allocator
gives up after a single reclaim and compaction attempt for
costly-order requests that lack __GFP_RETRY_MAYFAIL.

Add __GFP_RETRY_MAYFAIL to the GFP flags in efa_zalloc_mapped()
to allow retries with escalating compaction priority and release
of highatomic reserves before giving up.

Reviewed-by: Firas Jahjah <firasj@amazon.com>
Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
Signed-off-by: Michael Margolin <mrgolin@amazon.com>
---
 drivers/infiniband/hw/efa/efa_verbs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
index 2d28d68efe77..0b0396e190f2 100644
--- a/drivers/infiniband/hw/efa/efa_verbs.c
+++ b/drivers/infiniband/hw/efa/efa_verbs.c
@@ -190,7 +190,7 @@ static void *efa_zalloc_mapped(struct efa_dev *dev, dma_addr_t *dma_addr,
 {
 	void *addr;
 
-	addr = alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO);
+	addr = alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO | __GFP_RETRY_MAYFAIL);
 	if (!addr)
 		return NULL;
 
-- 
2.47.3

Re: [PATCH for-next] RDMA/efa: Use __GFP_RETRY_MAYFAIL for ring allocations

[sashiko-bot]

> RQ and CQ ring buffers require physically contiguous memory. On
> systems under memory pressure, high-order allocations can fail
> even when sufficient free memory exists because the allocator
> gives up after a single reclaim and compaction attempt for
> costly-order requests that lack __GFP_RETRY_MAYFAIL.
> 
> Add __GFP_RETRY_MAYFAIL to the GFP flags in efa_zalloc_mapped()
> to allow retries with escalating compaction priority and release
> of highatomic reserves before giving up.
> 
> Reviewed-by: Firas Jahjah <firasj@amazon.com>
> Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
> Signed-off-by: Michael Margolin <mrgolin@amazon.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260917105101.9365-1-mrgolin@amazon.com?part=1

Re: [PATCH for-next] RDMA/efa: Use __GFP_RETRY_MAYFAIL for ring allocations

[Leon Romanovsky]

On Thu, Sep 17, 2026 at 10:51:01AM +0000, Michael Margolin wrote:
> RQ and CQ ring buffers require physically contiguous memory. On
> systems under memory pressure, high-order allocations can fail
> even when sufficient free memory exists because the allocator
> gives up after a single reclaim and compaction attempt for
> costly-order requests that lack __GFP_RETRY_MAYFAIL.

What sizes did you see fail, and in which flow?

Thanks

> 
> Add __GFP_RETRY_MAYFAIL to the GFP flags in efa_zalloc_mapped()
> to allow retries with escalating compaction priority and release
> of highatomic reserves before giving up.
> 
> Reviewed-by: Firas Jahjah <firasj@amazon.com>
> Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
> Signed-off-by: Michael Margolin <mrgolin@amazon.com>
> ---
>  drivers/infiniband/hw/efa/efa_verbs.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
> index 2d28d68efe77..0b0396e190f2 100644
> --- a/drivers/infiniband/hw/efa/efa_verbs.c
> +++ b/drivers/infiniband/hw/efa/efa_verbs.c
> @@ -190,7 +190,7 @@ static void *efa_zalloc_mapped(struct efa_dev *dev, dma_addr_t *dma_addr,
>  {
>  	void *addr;
>  
> -	addr = alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO);
> +	addr = alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO | __GFP_RETRY_MAYFAIL);
>  	if (!addr)
>  		return NULL;
>  
> -- 
> 2.47.3
> 

Re: [PATCH for-next] RDMA/efa: Use __GFP_RETRY_MAYFAIL for ring allocations

[Jason Gunthorpe]

On Wed, Sep 23, 2026 at 01:51:27PM +0300, Leon Romanovsky wrote:
> On Thu, Sep 17, 2026 at 10:51:01AM +0000, Michael Margolin wrote:
> > RQ and CQ ring buffers require physically contiguous memory. On
> > systems under memory pressure, high-order allocations can fail
> > even when sufficient free memory exists because the allocator
> > gives up after a single reclaim and compaction attempt for
> > costly-order requests that lack __GFP_RETRY_MAYFAIL.
> 
> What sizes did you see fail, and in which flow?

.. and many other drivers has spent alot of effort not having high
order allocations like this because they just don't work.

Jason

Re: [PATCH for-next] RDMA/efa: Use __GFP_RETRY_MAYFAIL for ring allocations

[Michael Margolin]

On Wed, Sep 23, 2026 at 08:57:55AM -0300, Jason Gunthorpe wrote:
> On Wed, Sep 23, 2026 at 01:51:27PM +0300, Leon Romanovsky wrote:
> > On Thu, Sep 17, 2026 at 10:51:01AM +0000, Michael Margolin wrote:
> > > RQ and CQ ring buffers require physically contiguous memory. On
> > > systems under memory pressure, high-order allocations can fail
> > > even when sufficient free memory exists because the allocator
> > > gives up after a single reclaim and compaction attempt for
> > > costly-order requests that lack __GFP_RETRY_MAYFAIL.
> > 
> > What sizes did you see fail, and in which flow?

In the recent cases we have seen (in customer/production environment)
it during order 7 (512KB) allocations for 32K entries RQ.
> 
> .. and many other drivers has spent alot of effort not having high
> order allocations like this because they just don't work.
> 

Unfortunately we cannot support non contiguous queues in some of the
devices but we are going to add this support where possible.

Michael

> Jason

Re: [PATCH for-next] RDMA/efa: Use __GFP_RETRY_MAYFAIL for ring allocations

[Jason Gunthorpe]

On Thu, Sep 17, 2026 at 10:51:01AM +0000, Michael Margolin wrote:
> RQ and CQ ring buffers require physically contiguous memory. On
> systems under memory pressure, high-order allocations can fail
> even when sufficient free memory exists because the allocator
> gives up after a single reclaim and compaction attempt for
> costly-order requests that lack __GFP_RETRY_MAYFAIL.
> 
> Add __GFP_RETRY_MAYFAIL to the GFP flags in efa_zalloc_mapped()
> to allow retries with escalating compaction priority and release
> of highatomic reserves before giving up.
> 
> Reviewed-by: Firas Jahjah <firasj@amazon.com>
> Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
> Signed-off-by: Michael Margolin <mrgolin@amazon.com>
> ---
>  drivers/infiniband/hw/efa/efa_verbs.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Applied to for-next, thanks

Jason