[PATCH bpf] net: return page_pool frags to the pool in bpf_xdp_shrink_data()

[PATCH bpf] net: return page_pool frags to the pool in bpf_xdp_shrink_data()

[Dudu Lu]

bpf_xdp_shrink_data() releases a fully-shrunk frag through
__xdp_return(netmem, mem_type, false, zc_frag), selecting the freer from
the rxq's mem type.  Under generic XDP the rxq's xdp_rxq_info often has no
mem model registered (e.g. loopback), so mem_type stays
MEM_TYPE_PAGE_SHARED == 0 even though the frag was allocated from a page
pool by skb_pp_cow_data().  __xdp_return() then frees the page through
page_frag_free() instead of page_pool_put_full_netmem(), so the page is
returned to the buddy allocator while its page_pool accounting is still
live:

  BUG: Bad page state in process A-146-S3_S1  pfn:4694b
  page dumped because: page_pool leak
   page_frag_free+0xf0/0x11c
   __xdp_return+0x160/0x640
   bpf_xdp_adjust_tail+0x5a4/0xbac
   bpf_prog_run_generic_xdp+0x4a8/0x1204

Before falling back to __xdp_return(), try the page_pool path: if the
frag really is a page_pool page, napi_pp_put_page() recycles it through
the pool and we are done.

Fixes: 4dce1a0d7cf3 ("bpf: Support pulling non-linear xdp data")
Assisted-by: LLM
Signed-off-by: Dudu Lu <phx0fer@gmail.com>
---
Tested base
-----------
Detected on Linux 7.3-rc4 (commit 93f51579e7df), arm64 QEMU guest.
Re-checked against mainline f47d223627f8 (v7.3-341): the affected code
is unchanged between the two, so the bug is still present there.
Checked against linux-next next-20261021 (commit 9a482184d910): the
affected code is unchanged there as well, so the bug is still present in
linux-next too.

Reproducer status
-----------------
A reproducer was written and executed on an arm64 QEMU guest (generic XDP on
loopback); the kernel reported "BUG: Bad page state ... page_pool leak".  The
reproducer source is not posted here per Documentation/process/security-bugs.rst;
it is available on request.

Triggering conditions
---------------------
The ability to load a generic-XDP program (CAP_BPF / CAP_NET_ADMIN) on a
device whose rxq has no mem model but whose skbs are page_pool-backed (loopback
via skb_cow_data_for_xdp()).

Notes
-----
Built and checkpatch-clean: this patch was applied on top of 7.3-rc4
(commit 93f51579e7df) and a full arm64 kernel (defconfig) was built with
it -- no new warnings or errors.  scripts/checkpatch.pl reports 0 errors,
0 warnings.

 net/core/filter.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/net/core/filter.c b/net/core/filter.c
index 70dc621672f2..94bc367cee12 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -36,6 +36,7 @@
 #include <net/protocol.h>
 #include <net/netlink.h>
 #include <linux/skbuff.h>
+#include <linux/skbuff_ref.h>
 #include <linux/skmsg.h>
 #include <net/sock.h>
 #include <net/flow_dissector.h>
@@ -4380,6 +4381,18 @@ static bool bpf_xdp_shrink_data(struct xdp_buff *xdp, skb_frag_t *frag,
 	}
 
 	if (release) {
+		/* The frag may be page_pool-backed even when the rxq's
+		 * xdp_rxq_info carries no mem model (generic XDP on a
+		 * driver that never registers one, e.g. loopback).  In that
+		 * case mem_type is MEM_TYPE_PAGE_SHARED and __xdp_return()
+		 * would free the page through page_frag_free(), leaving the
+		 * page_pool's accounting dangling -- "page_pool leak".  Route
+		 * page_pool pages back through the pool instead.
+		 */
+		if (IS_ENABLED(CONFIG_PAGE_POOL) &&
+		    mem_type == MEM_TYPE_PAGE_SHARED && napi_pp_put_page(netmem))
+			return release;
+
 		__xdp_return(netmem, mem_type, false, zc_frag);
 	} else {
 		if (!tail)

Re: [PATCH bpf] net: return page_pool frags to the pool in bpf_xdp_shrink_data()

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2975 bytes --]

> net: return page_pool frags to the pool in bpf_xdp_shrink_data()

TL;DR: Commit message only: the Fixes tag (4dce1a0d7cf3) may be wrong;
e6d5dbdd20aa (v6.9, generic XDP multi-buff) seems the real origin, so stable
backport scope is understated.

>     net: return page_pool frags to the pool in bpf_xdp_shrink_data()
>
>     bpf_xdp_shrink_data() releases a fully-shrunk frag through
>     __xdp_return(netmem, mem_type, false, zc_frag), selecting the freer from
>     the rxq's mem type.  Under generic XDP the rxq's xdp_rxq_info often has no
>     mem model registered (e.g. loopback), so mem_type stays
>     MEM_TYPE_PAGE_SHARED == 0 even though the frag was allocated from a page
>     pool by skb_pp_cow_data().  __xdp_return() then frees the page through
>     page_frag_free() instead of page_pool_put_full_netmem(), so the page is
>     returned to the buddy allocator while its page_pool accounting is still
>     live:
>
>       BUG: Bad page state in process A-146-S3_S1  pfn:4694b
>       page dumped because: page_pool leak
>        page_frag_free+0xf0/0x11c
>        __xdp_return+0x160/0x640
>        bpf_xdp_adjust_tail+0x5a4/0xbac
>        bpf_prog_run_generic_xdp+0x4a8/0x1204
>
>     Before falling back to __xdp_return(), try the page_pool path: if the
>     frag really is a page_pool page, napi_pp_put_page() recycles it through
>     the pool and we are done.
>
>     Fixes: 4dce1a0d7cf3 ("bpf: Support pulling non-linear xdp data")
>     Assisted-by: LLM
>     Signed-off-by: Dudu Lu <phx0fer@gmail.com>

Is the Fixes: tag pointing at the right commit?

4dce1a0d7cf3 only added bpf_xdp_pull_data() as a new caller of
bpf_xdp_shrink_data().  The splat in the commit message goes through
bpf_xdp_adjust_tail() -> bpf_xdp_frags_shrink_tail() -> __xdp_return(),
which already freed released frags based on xdp->rxq->mem.type
(MEM_TYPE_PAGE_SHARED when no mem model is registered, as in generic XDP)
since c5114710c8ce ("xsk: fix usage of multi-buffer BPF helpers for ZC XDP")
and bf25146a5595 ("bpf: add frags support to the bpf_xdp_adjust_tail() API").

The mismatch only became possible when e6d5dbdd20aa made
netif_receive_generic_xdp() build non-linear xdp_buffs from page_pool backed
frags via skb_pp_cow_data(), while the generic XDP rxq has no registered mem
model.  From then on the adjust_tail shrink path freed page_pool pages with
page_frag_free(), and the "page_pool leak" check already existed via
dba1b8a7ab68 ("mm/page_pool: catch page_pool memory leaks").

e6d5dbdd20aa landed in v6.9 while 4dce1a0d7cf3 landed in v6.18, so the
current tag understates the stable backport scope.  Should this instead
include:

  Fixes: e6d5dbdd20aa ("xdp: add multi-buff support for xdp running in generic mode")


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39631548606

Re: [PATCH bpf] net: return page_pool frags to the pool in bpf_xdp_shrink_data()

[Alexei Starovoitov]

On Wed, Oct 21, 2026 at 08:56 AM Dudu Lu <phx0fer@gmail.com> wrote:
> Before falling back to __xdp_return(), try the page_pool path: if the
> frag really is a page_pool page, napi_pp_put_page() recycles it through
> the pool and we are done.

This makes it worse.
netmem_is_pp() says that the page is still in a pool. It doesn't say
what kind of reference the frag holds.

On XDP_TX and XDP_REDIRECT veth_xdp_rcv_skb() does get_page() on every
frag in veth_xdp_get(), consume_skb() drops the page_pool references
and the frame leaves as MEM_TYPE_PAGE_SHARED. A frag of 2k or less
shares its page with other frags, so the page is still in
rq->page_pool when the prog of the peer trims that frag in
veth_xdp_rcv_one(). Today page_frag_free() drops the reference that
get_page() took. With this patch napi_pp_put_page() drops pp_ref_count
for the second time and the page reference is leaked.

Jiayuan's v1 in August had the same check in the same place.
See why looking at the page doesn't work:
https://lore.kernel.org/bpf/01fd251d-1134-4251-9c11-79943e367ffe@linux.dev/

rxq->mem.type has to be right when the prog runs.
v3 of that series adds a flag to xdp_buff for that:
https://lore.kernel.org/bpf/20260911135711.109338-1-jiayuan.chen@linux.dev/
I think rxq on stack, like in cpu_map_bpf_prog_run_xdp(), is enough.
dev->_rx[i].xdp_rxq never has a mem model and in
bpf_prog_run_generic_xdp() the frags of a nonlinear skb always come
from skb_pp_cow_data().
Below leaves out veth. It needs the same for skbs, plus
https://lore.kernel.org/bpf/20260917185814.0f944e78@kernel.org/

Something like this (compile tested only):

diff --git a/net/core/dev.c b/net/core/dev.c
index e76762e..ecdac67 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5542,6 +5542,7 @@ u32 bpf_prog_run_generic_xdp(struct sk_buff *skb, struct xdp_buff *xdp,
 	void *orig_data, *orig_data_end, *hard_start;
 	struct netdev_rx_queue *rxqueue;
 	bool orig_bcast, orig_host;
+	struct xdp_rxq_info rxq;
 	u32 mac_len, frame_sz;
 	__be16 orig_eth_type;
 	struct ethhdr *eth;
@@ -5565,6 +5566,12 @@ u32 bpf_prog_run_generic_xdp(struct sk_buff *skb, struct xdp_buff *xdp,
 	if (skb_is_nonlinear(skb)) {
 		skb_shinfo(skb)->xdp_frags_size = skb->data_len;
 		xdp_buff_set_frags_flag(xdp);
+		/* The frags came from skb_pp_cow_data(). xdp_rxq is shared
+		 * with other CPUs and with BPF_PROG_TEST_RUN, so use a copy.
+		 */
+		rxq = rxqueue->xdp_rxq;
+		rxq.mem.type = MEM_TYPE_PAGE_POOL;
+		xdp->rxq = &rxq;
 	} else {
 		xdp_buff_clear_frags_flag(xdp);
 	}
@@ -5577,6 +5584,7 @@ u32 bpf_prog_run_generic_xdp(struct sk_buff *skb, struct xdp_buff *xdp,
 	orig_eth_type = eth->h_proto;

 	act = bpf_prog_run_xdp(xdp_prog, xdp);
+	xdp->rxq = &rxqueue->xdp_rxq;

 	/* check if bpf_xdp_adjust_head was used */
 	off = xdp->data - orig_data;

xdp fixes go via net tree.

pw-bot: cr