From: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
To: Orgad Shaneh <orgads@gmail.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>,
Wei Liu <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
"Jakub Kicinski" <kuba@kernel.org>,
Paolo Abeni <pabeni@redhat.com>, <netdev@vger.kernel.org>,
<linux-hyperv@vger.kernel.org>, <linux-rdma@vger.kernel.org>,
<bpf@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
Wei Hu <weh@microsoft.com>,
"Konstantin Taranov" <kotaranov@microsoft.com>,
Simon Horman <horms@kernel.org>,
"Alexei Starovoitov" <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
"Jesper Dangaard Brouer" <hawk@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
Stanislav Fomichev <sdf@fomichev.me>,
Long Li <longli@microsoft.com>,
Dipayaan Roy <dipayanroy@linux.microsoft.com>,
Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>,
Aditya Garg <gargaditya@linux.microsoft.com>,
Erni Sri Satya Vennela <ernis@linux.microsoft.com>,
Muhammad Falak R Wani <falakreyaz@gmail.com>,
Jesse Brandeburg <jbrandeb@kernel.org>
Subject: Re: [PATCH net] net: mana: fix XDP_REDIRECT of pre-allocated RX buffers
Date: Wed, 4 Nov 2026 17:22:24 +0100 [thread overview]
Message-ID: <autbwKmZPPWEj0Dq@boxer> (raw)
In-Reply-To: <20261102064324.806414-1-orgads@gmail.com>
On Mon, Nov 02, 2026 at 06:43:24AM +0000, Orgad Shaneh wrote:
> mana_pre_alloc_rxbufs() fills the RX rings with plain pages from
> dev_alloc_pages() (from_pool == false), but the rxq is registered with
> MEM_TYPE_PAGE_POOL and mana_run_xdp() builds every xdp_buff with that
> rxq info. mana_rx_skb() frees a !from_pool buffer correctly on
> XDP_PASS, XDP_TX and XDP_DROP, but a redirected buffer is freed through
> its memory model: __xdp_return() hands the plain page to
> page_pool_put_full_netmem(). This triggers the WARN in
> include/net/page_pool/helpers.h, breaks the page refcount, RX refills
> fail and the queue stops receiving.
>
> The pre-allocated buffers fill the whole ring of every rxq created by
> an MTU change, by ethtool -L / -G and, since
> commit 730ff06d3f5c ("net: mana: Use page pool fragments for RX buffers
> instead of full pages to improve memory efficiency."), by every XDP
> attach and detach. So on current kernels the first ring of redirected
> packets after attaching an XDP program always hits this.
>
> Reproduced on 7.2.11, and on 6.12.113 after ethtool -L, on an Azure
> MANA VF: an XDP program that redirects UDP to an XSKMAP with an AF_XDP
> socket in copy mode gives WARNINGs at __xdp_return+0x249 (stack
> __xsk_map_redirect, xdp_do_redirect, mana_run_xdp, mana_poll), then
> the queue stops.
>
> Give each rxq a second xdp_rxq_info registered with
> MEM_TYPE_PAGE_SHARED, the model these buffers had before the page pool
> was added, and use it for buffers that are not from the pool. Those
> buffers are already DMA-unmapped by mana_refill_rx_oob() before XDP
> runs, and page_frag_free() frees them whatever their order. Allocating
> the pre-allocated buffers from the page pool instead would need the
> pools to exist before the rxqs and to outlive them, a much larger
> change.
>
> Found, fixed and tested with an AI coding assistant (Claude Code).
>
> Fixes: b1d13f7a3b53 ("net: mana: Add page pool for RX buffers")
> Signed-off-by: Orgad Shaneh <orgads@gmail.com>
> ---
> Tested on an Azure D2s_v6 MANA VF, net at cb284914e749 with and
> without this patch (same config), XDP program redirecting UDP to an
> AF_XDP socket in copy mode on queue 0, 25k pps offered right after
> attach:
> - without: 41 WARNINGs at __xdp_return+0x252, 0 packets relayed,
> hc_rx_discards_no_wqe 301470.
> - with: no WARNING, 25k of 25k pps relayed, 0 loss; also over three
> more attach/detach cycles, two ethtool -L combined 1/2 cycles and an
> MTU 4000/1500 change before attach. MemAvailable unchanged.
> W=1 build of the mana directories: no new warnings.
>
> drivers/net/ethernet/microsoft/mana/mana_bpf.c | 9 +++++++--
> drivers/net/ethernet/microsoft/mana/mana_en.c | 11 ++++++++++-
> include/net/mana/mana.h | 5 ++++-
> 3 files changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
> index 7c55f8724441..494267b3069a 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_bpf.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
> @@ -78,7 +78,8 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,
> }
>
> u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
> - struct xdp_buff *xdp, void *buf_va, uint pkt_len)
> + struct xdp_buff *xdp, void *buf_va, bool from_pool,
> + uint pkt_len)
> {
> struct mana_stats_rx *rx_stats;
> struct bpf_prog *prog;
> @@ -90,7 +91,11 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
> if (!prog)
> goto out;
>
> - xdp_init_buff(xdp, PAGE_SIZE, &rxq->xdp_rxq);
> + /* A redirected buffer is freed by its memory model, so a buffer not
> + * from the page pool must not claim MEM_TYPE_PAGE_POOL.
> + */
> + xdp_init_buff(xdp, PAGE_SIZE,
> + from_pool ? &rxq->xdp_rxq : &rxq->xdp_rxq_shared);
> xdp_prepare_buff(xdp, buf_va, rxq->headroom, pkt_len, true);
>
> act = bpf_prog_run_xdp(prog, xdp);
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
> index 6b7c027e497e..7547aeb3235d 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_en.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
> @@ -2157,7 +2157,7 @@ static void mana_rx_skb(void *buf_va, bool from_pool,
> return;
> }
>
> - act = mana_run_xdp(ndev, rxq, &xdp, buf_va, pkt_len);
> + act = mana_run_xdp(ndev, rxq, &xdp, buf_va, from_pool, pkt_len);
I wonder if it wouldn't be better to work with rxoob objects which seem to
carry the from_pool setting?
>
> if (act == XDP_REDIRECT && !rxq->xdp_rc)
> return;
> @@ -2947,6 +2947,9 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
> if (xdp_rxq_info_is_reg(&rxq->xdp_rxq))
> xdp_rxq_info_unreg(&rxq->xdp_rxq);
>
> + if (xdp_rxq_info_is_reg(&rxq->xdp_rxq_shared))
> + xdp_rxq_info_unreg(&rxq->xdp_rxq_shared);
> +
> if (rxq->rxobj != INVALID_MANA_HANDLE)
> mana_destroy_wq_obj(apc, GDMA_RQ, rxq->rxobj);
>
> @@ -3225,6 +3228,12 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
> WARN_ON(xdp_rxq_info_reg_mem_model(&rxq->xdp_rxq, MEM_TYPE_PAGE_POOL,
> rxq->page_pool));
>
> + /* The pre-allocated buffers (mana_pre_alloc_rxbufs) are plain pages */
> + WARN_ON(xdp_rxq_info_reg(&rxq->xdp_rxq_shared, ndev, rxq_idx,
> + cq->napi.napi_id));
> + WARN_ON(xdp_rxq_info_reg_mem_model(&rxq->xdp_rxq_shared,
> + MEM_TYPE_PAGE_SHARED, NULL));
> +
> /* Initialize the DIM work before enabling NAPI, so that a poll
> * cannot reach net_dim() with an uninitialized cq->dim.work.
> */
> diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
> index 71d726b1f784..11ceb5946ee0 100644
> --- a/include/net/mana/mana.h
> +++ b/include/net/mana/mana.h
> @@ -420,6 +420,8 @@ struct mana_rxq {
>
> struct bpf_prog __rcu *bpf_prog;
> struct xdp_rxq_info xdp_rxq;
> + /* For buffers not from the page pool (from_pool == false) */
> + struct xdp_rxq_info xdp_rxq_shared;
> void *xdp_save_va; /* for reusing */
> bool xdp_flush;
> int xdp_rc; /* XDP redirect return code */
> @@ -695,7 +697,8 @@ void mana_xdp_tx(struct sk_buff *skb, struct net_device *ndev);
> int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,
> u32 flags);
> u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
> - struct xdp_buff *xdp, void *buf_va, uint pkt_len);
> + struct xdp_buff *xdp, void *buf_va, bool from_pool,
> + uint pkt_len);
> struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);
> void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog);
> int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);
>
> base-commit: cb284914e749388eef402938bdb3115fe1f6d337
> --
> 2.52.0
>
>
next prev parent reply other threads:[~2026-11-04 16:22 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-02 6:43 [PATCH net] net: mana: fix XDP_REDIRECT of pre-allocated RX buffers Orgad Shaneh
2026-11-04 16:22 ` Maciej Fijalkowski [this message]
2026-11-04 20:10 ` Orgad Shaneh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=autbwKmZPPWEj0Dq@boxer \
--to=maciej.fijalkowski@intel.com \
--cc=andrew+netdev@lunn.ch \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=decui@microsoft.com \
--cc=dipayanroy@linux.microsoft.com \
--cc=edumazet@kernel.org \
--cc=ernis@linux.microsoft.com \
--cc=falakreyaz@gmail.com \
--cc=gargaditya@linux.microsoft.com \
--cc=haiyangz@microsoft.com \
--cc=hamzamahfooz@linux.microsoft.com \
--cc=hawk@kernel.org \
--cc=horms@kernel.org \
--cc=jbrandeb@kernel.org \
--cc=john.fastabend@gmail.com \
--cc=kotaranov@microsoft.com \
--cc=kuba@kernel.org \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=netdev@vger.kernel.org \
--cc=orgads@gmail.com \
--cc=pabeni@redhat.com \
--cc=sdf@fomichev.me \
--cc=weh@microsoft.com \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox