[PATCH bpf] bpf: free page_pool frags via the page_pool path in bpf_xdp_shrink_data

[Jiayuan Chen <jiayuan.chen@linux.dev>]

syzbot reported a "Bad page state ... page_pool leak" when a generic XDP
program shrinks an skb into its frags on the tun write() path:

  page_frag_free
  __xdp_return
  bpf_xdp_shrink_data
  bpf_xdp_frags_shrink_tail
  bpf_xdp_adjust_tail

When a program shrinks a whole frag, bpf_xdp_shrink_data() frees it via
__xdp_return() using xdp->rxq->mem.type. For skb-backed XDP the skb is
first rebuilt into page_pool memory (skb_cow_data_for_xdp() for generic
XDP, skb_pp_cow_data() for veth), so the frag is a page_pool page. But the
rxq was registered as MEM_TYPE_PAGE_SHARED, so __xdp_return() calls
page_frag_free() on a page_pool page: its base refcount drops to 0 and the
page is freed to the buddy allocator with pp_magic still set.

The rxq mem model cannot be relied on here because the rxq is shared and
does not describe the frag's real memory. The netdev generic rxq is used
both by generic XDP (page_pool frags) and by bpf_prog_test_run_xdp(), which
borrows the loopback rxq for plain alloc_page() frags; veth uses one rxq
for cow'd (page_pool) skbs and for redirected frames of any memory type. A
single rxq->mem.type can be wrong in either direction.

The memory type is really a property of the page. Check the frag itself
with netmem_is_pp() and return page_pool frags to their pool, keeping the
rxq mem type only for non page_pool pages. The page_pool is taken from the
page (netmem_get_pp()), so no rxq registration is involved.

Fixes: e6d5dbdd20aa ("xdp: add multi-buff support for xdp running in generic mode")
Fixes: 0ebab78cbcbf ("net: veth: add page_pool for page recycling")
Reported-by: syzbot+237bbeed8dfe0699b7f5@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=237bbeed8dfe0699b7f5
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
---
 net/core/filter.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/net/core/filter.c b/net/core/filter.c
index 3423734124a5..4536bd147c25 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -89,6 +89,7 @@
 #include <net/ip6_route.h>
 
 #include "dev.h"
+#include "netmem_priv.h"
 
 /* Keep the struct bpf_fib_lookup small so that it fits into a cacheline */
 static_assert(sizeof(struct bpf_fib_lookup) == 64, "struct bpf_fib_lookup size check");
@@ -4293,6 +4294,14 @@ static bool bpf_xdp_shrink_data(struct xdp_buff *xdp, skb_frag_t *frag,
 	if (mem_type == MEM_TYPE_XSK_BUFF_POOL) {
 		netmem = 0;
 		zc_frag = bpf_xdp_shrink_data_zc(xdp, shrink, tail, release);
+	} else if (netmem_is_pp(netmem)) {
+		/* The rxq mem model does not always describe how a released
+		 * frag must be freed: the generic-XDP and veth paths run the
+		 * program on a shared rxq while the frag has been cow'd into
+		 * page_pool memory. Trust the page itself and return a
+		 * page_pool frag to its pool regardless of rxq->mem.type.
+		 */
+		mem_type = MEM_TYPE_PAGE_POOL;
 	}
 
 	if (release) {
-- 
2.43.0