[PATCH net] net: mana: never hand a non-page_pool buffer to XDP

[PATCH net] net: mana: never hand a non-page_pool buffer to XDP

[Gunter Woytowitz via B4 Relay]

From: Gunter Woytowitz <gunter@vcinity.io>

mana_create_rxq() registers MEM_TYPE_PAGE_POOL for the rxq
unconditionally, so every buffer XDP can see must be owned by that
page_pool: on XDP_REDIRECT the frame is freed through __xdp_return()
-> page_pool_put_full_page().

mana_xdp_set() assigns apc->bpf_prog before calling
mana_pre_alloc_rxbufs(), which allocates with dev_alloc_pages(), and
mana_fill_rx_oob() prefers those buffers whenever mpc->rxbufs_pre is
set, leaving from_pool false. So for a port that is up when a program
is attached, the entire re-created ring is filled with pages the
page_pool does not own.

The page_pool then sees pp_ref_count == 0 when such a frame is
returned, so the atomic_long_sub_return() in page_pool_unref_netmem()
goes negative and trips its WARN_ON(ret < 0), once per redirected
frame. Observed on a 5.14-based distro kernel, where that warning sits
at helpers.h:269:

  WARNING: CPU: 3 PID: 0 at include/net/page_pool/helpers.h:269
                                            __xdp_return+0x2b3/0x2c0
  mana_process_rx_cqe -> mana_run_xdp -> mana_rx_skb -> xsk_map_redirect
                                            -> __xdp_return

On a VM booted with console=ttyS0 the resulting stack traces peg the
console thread and the machine becomes unusable.

Fill from the page_pool when a program is attached, using
mana_xdp_get() -- the predicate mana_get_rxbuf_cfg() already uses to
choose the XDP buffer geometry. With no program attached nothing
changes, so the pre-allocation still does its job of keeping
mana_attach() from failing on allocation.

Leaving the pre-allocated buffers unconsumed is safe:
mana_pre_dealloc_rxbufs() dma-unmaps and put_page()s the remainder,
and every caller (mana_xdp_set(), mana_change_mtu(), and both ethtool
ring and channel paths) already runs it after mana_attach().

The rxq->xdp_save_va reuse in mana_get_rxfrag() also leaves from_pool
false, but that cache is fed only by the drop path's non-pool branch,
which this change makes unreachable while a program is attached, so
it needs no fix.

Found and fixed on a 5.14-based distro kernel running AF_XDP over
MANA in copy mode: 24M+ redirected frames with no warnings, where the
unpatched driver warned on essentially every redirected frame. All of
the code involved is unchanged in mainline.

Fixes: b1d13f7a3b53 ("net: mana: Add page pool for RX buffers")
Signed-off-by: Gunter Woytowitz <gunter@vcinity.io>
---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 7a1ac853e..a2cb66753 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -2925,7 +2925,14 @@ static int mana_fill_rx_oob(struct mana_recv_buf_oob *rx_oob, u32 mem_key,
 	dma_addr_t da;
 	void *va;
 
-	if (mpc->rxbufs_pre)
+	/* The pre-allocated buffers come from dev_alloc_pages(), not from the
+	 * rxq's page_pool. With a program attached any buffer can reach
+	 * __xdp_return(), which returns it to the pool the rxq registered, so
+	 * fill from the pool instead. The unused pre-allocated buffers are
+	 * released by the mana_pre_dealloc_rxbufs() every caller already runs
+	 * after mana_attach().
+	 */
+	if (mpc->rxbufs_pre && !mana_xdp_get(mpc))
 		va = mana_get_rxbuf_pre(rxq, &da);
 	else
 		va = mana_get_rxfrag(rxq, dev, &da, &from_pool, &pp_page,

---
base-commit: 50d05c7c76c96b90462f24debacca971d2e86713
change-id: 20260910-mana-xdp-pagepool-d61a74902b9f

Best regards,
--  
Gunter Woytowitz <gunter@vcinity.io>

Re: [PATCH net] net: mana: never hand a non-page_pool buffer to XDP

[Simon Horman]

On Thu, Sep 10, 2026 at 11:17:47AM -0400, Gunter Woytowitz wrote:
> mana_create_rxq() registers MEM_TYPE_PAGE_POOL for the rxq
> unconditionally, so every buffer XDP can see must be owned by that
> page_pool: on XDP_REDIRECT the frame is freed through __xdp_return()
> -> page_pool_put_full_page().
> 
> mana_xdp_set() assigns apc->bpf_prog before calling
> mana_pre_alloc_rxbufs(), which allocates with dev_alloc_pages(), and
> mana_fill_rx_oob() prefers those buffers whenever mpc->rxbufs_pre is
> set, leaving from_pool false. So for a port that is up when a program
> is attached, the entire re-created ring is filled with pages the
> page_pool does not own.
> 
> The page_pool then sees pp_ref_count == 0 when such a frame is
> returned, so the atomic_long_sub_return() in page_pool_unref_netmem()
> goes negative and trips its WARN_ON(ret < 0), once per redirected
> frame. Observed on a 5.14-based distro kernel, where that warning sits
> at helpers.h:269:
> 
>   WARNING: CPU: 3 PID: 0 at include/net/page_pool/helpers.h:269
>                                             __xdp_return+0x2b3/0x2c0
>   mana_process_rx_cqe -> mana_run_xdp -> mana_rx_skb -> xsk_map_redirect
>                                             -> __xdp_return
> 
> On a VM booted with console=ttyS0 the resulting stack traces peg the
> console thread and the machine becomes unusable.
> 
> Fill from the page_pool when a program is attached, using
> mana_xdp_get() -- the predicate mana_get_rxbuf_cfg() already uses to
> choose the XDP buffer geometry. With no program attached nothing
> changes, so the pre-allocation still does its job of keeping
> mana_attach() from failing on allocation.
> 
> Leaving the pre-allocated buffers unconsumed is safe:
> mana_pre_dealloc_rxbufs() dma-unmaps and put_page()s the remainder,
> and every caller (mana_xdp_set(), mana_change_mtu(), and both ethtool
> ring and channel paths) already runs it after mana_attach().
> 
> The rxq->xdp_save_va reuse in mana_get_rxfrag() also leaves from_pool
> false, but that cache is fed only by the drop path's non-pool branch,
> which this change makes unreachable while a program is attached, so
> it needs no fix.
> 
> Found and fixed on a 5.14-based distro kernel running AF_XDP over
> MANA in copy mode: 24M+ redirected frames with no warnings, where the
> unpatched driver warned on essentially every redirected frame. All of
> the code involved is unchanged in mainline.
> 
> Fixes: b1d13f7a3b53 ("net: mana: Add page pool for RX buffers")
> Signed-off-by: Gunter Woytowitz <gunter@vcinity.io>

Unfortunately the CI failed to apply this patch to net.
Which is curious, because I am able to apply it locally.

But perhaps it would be best to (rebase and?) repost
after waiting for the usual 24h[*] to elapse.

[*] https://docs.kernel.org/process/maintainer-netdev.html

-- 
pw-bot: changes-requested

Re: [PATCH net] net: mana: never hand a non-page_pool buffer to XDP

[Gunter Woytowitz]

I had a very hard time creating the request I submitted,  and don’t have the time to try and do that again.
If you think it is a worthy change you can incorporate it some other way.


> On Sep 11, 2026, at 7:12 AM, Simon Horman <horms@kernel.org> wrote:
>
> On Thu, Sep 10, 2026 at 11:17:47AM -0400, Gunter Woytowitz wrote:
>> mana_create_rxq() registers MEM_TYPE_PAGE_POOL for the rxq
>> unconditionally, so every buffer XDP can see must be owned by that
>> page_pool: on XDP_REDIRECT the frame is freed through __xdp_return()
>> -> page_pool_put_full_page().
>>
>> mana_xdp_set() assigns apc->bpf_prog before calling
>> mana_pre_alloc_rxbufs(), which allocates with dev_alloc_pages(), and
>> mana_fill_rx_oob() prefers those buffers whenever mpc->rxbufs_pre is
>> set, leaving from_pool false. So for a port that is up when a program
>> is attached, the entire re-created ring is filled with pages the
>> page_pool does not own.
>>
>> The page_pool then sees pp_ref_count == 0 when such a frame is
>> returned, so the atomic_long_sub_return() in page_pool_unref_netmem()
>> goes negative and trips its WARN_ON(ret < 0), once per redirected
>> frame. Observed on a 5.14-based distro kernel, where that warning sits
>> at helpers.h:269:
>>
>>  WARNING: CPU: 3 PID: 0 at include/net/page_pool/helpers.h:269
>>                                            __xdp_return+0x2b3/0x2c0
>>  mana_process_rx_cqe -> mana_run_xdp -> mana_rx_skb -> xsk_map_redirect
>>                                            -> __xdp_return
>>
>> On a VM booted with console=ttyS0 the resulting stack traces peg the
>> console thread and the machine becomes unusable.
>>
>> Fill from the page_pool when a program is attached, using
>> mana_xdp_get() -- the predicate mana_get_rxbuf_cfg() already uses to
>> choose the XDP buffer geometry. With no program attached nothing
>> changes, so the pre-allocation still does its job of keeping
>> mana_attach() from failing on allocation.
>>
>> Leaving the pre-allocated buffers unconsumed is safe:
>> mana_pre_dealloc_rxbufs() dma-unmaps and put_page()s the remainder,
>> and every caller (mana_xdp_set(), mana_change_mtu(), and both ethtool
>> ring and channel paths) already runs it after mana_attach().
>>
>> The rxq->xdp_save_va reuse in mana_get_rxfrag() also leaves from_pool
>> false, but that cache is fed only by the drop path's non-pool branch,
>> which this change makes unreachable while a program is attached, so
>> it needs no fix.
>>
>> Found and fixed on a 5.14-based distro kernel running AF_XDP over
>> MANA in copy mode: 24M+ redirected frames with no warnings, where the
>> unpatched driver warned on essentially every redirected frame. All of
>> the code involved is unchanged in mainline.
>>
>> Fixes: b1d13f7a3b53 ("net: mana: Add page pool for RX buffers")
>> Signed-off-by: Gunter Woytowitz <gunter@vcinity.io>
>
> Unfortunately the CI failed to apply this patch to net.
> Which is curious, because I am able to apply it locally.
>
> But perhaps it would be best to (rebase and?) repost
> after waiting for the usual 24h[*] to elapse.
>
> [*] https://docs.kernel.org/process/maintainer-netdev.html
>
> --
> pw-bot: changes-requested


This email may contain confidential and privileged information and is intended solely for the use of the addressee(s). Unless you are the addressee or are authorized to receive messages for the addressee, you may not use, copy, disseminate, or disclose the information or any attachments to any third party. If you have received this correspondence in error, please notify the sender immediately and delete this email. Your cooperation and understanding are greatly appreciated. Attention Federal Customers: Please note this email platform is NOT approved to communicate (send or receive) CUI. For questions on the approved system to communicate CUI, please contact your designated Vcinity Representative.

Re: [PATCH net] net: mana: never hand a non-page_pool buffer to XDP

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Bypassing pre-allocated buffers when XDP is attached defeats anti-OOM protections, risking permanent interface disablement.

Pre-existing issues:
- [Critical] Dangling pointer in apc->bpf_prog leading to Use-After-Free and Double Free if mana_pre_alloc_rxbufs fails.
--

commit 47baeb3edbb2306dafb469762d4ed1350e2c78f3
Author: Gunter Woytowitz <gunter@vcinity.io>

net: mana: never hand a non-page_pool buffer to XDP

This commit ensures that all buffers handled by XDP are allocated from
the page_pool to prevent warnings when __xdp_return() frees them. It
bypasses the pre-allocated buffers when an XDP program is attached,
falling back to dynamic allocation. Unused pre-allocated buffers are
safely freed by existing cleanup paths.

> diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
> index 7a1ac853e3abc..a2cb667530a06 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_en.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
> @@ -2925,7 +2925,14 @@ static int mana_fill_rx_oob(struct mana_recv_buf_oob *rx_oob, u32 mem_key,
>  	dma_addr_t da;
>  	void *va;
>  
> -	if (mpc->rxbufs_pre)
> +	/* The pre-allocated buffers come from dev_alloc_pages(), not from the
> +	 * rxq's page_pool. With a program attached any buffer can reach
> +	 * __xdp_return(), which returns it to the pool the rxq registered, so
> +	 * fill from the pool instead. The unused pre-allocated buffers are
> +	 * released by the mana_pre_dealloc_rxbufs() every caller already runs
> +	 * after mana_attach().
> +	 */
> +	if (mpc->rxbufs_pre && !mana_xdp_get(mpc))
>  		va = mana_get_rxbuf_pre(rxq, &da);
>  	else
>  		va = mana_get_rxfrag(rxq, dev, &da, &from_pool, &pp_page,

[Severity: High]
Does bypassing the pre-allocated buffers when XDP is attached defeat the
anti-OOM protections?

When attaching an XDP program under memory pressure, mana_pre_alloc_rxbufs()
succeeds, but the dynamic page_pool allocations in mana_fill_rx_oob()
can now fail.

If mana_attach() fails halfway through due to these allocation failures:

drivers/net/ethernet/microsoft/mana/mana_bpf.c:mana_xdp_set() {
    ...
		err = mana_attach(ndev);
		if (err) {
			netdev_err(ndev,
				   "mana_attach failed at xdp set: %d\n", err);
			NL_SET_ERR_MSG_MOD(extack,
					   "XDP: Re-config failed at attach");
			goto err_dealloc_rxbuffs;
		}
    ...
}

Could this leave the network interface permanently detached until manually
reset, effectively doubling the peak memory requirement during the XDP attach
process?


[Severity: Critical]
This is a pre-existing issue, but I noticed a potential use-after-free
in mana_xdp_set() during error handling.

drivers/net/ethernet/microsoft/mana/mana_bpf.c:mana_xdp_set() {
    ...
	apc->bpf_prog = prog;

	if (apc->port_is_up) {
		/* Pre-allocate buffers to prevent failure in mana_attach */
		err = mana_pre_alloc_rxbufs(apc, ndev->mtu, apc->num_queues);
		if (err) {
			NL_SET_ERR_MSG_MOD(extack,
					   "XDP: Insufficient memory for tx/rx re-config");
			return err;
		}
    ...
}

If mana_pre_alloc_rxbufs() fails, we return early without restoring
apc->bpf_prog to old_prog.

Can this leave a dangling pointer in apc->bpf_prog?

The caller dev_xdp_attach() in net/core/dev.c drops the reference to the
program on error, freeing it. A subsequent call to mana_xdp_set() will read
this pointer as old_prog and call bpf_prog_put(old_prog).

Will this result in a double-free of the BPF program?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260910-mana-xdp-pagepool-v1-1-dab140d91fed@vcinity.io?part=1