From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794524124; b=HeUvj9Gw8YNIk5UFa0CC1vVABcBgvVTnUQpVW7WyxltJjpgKIllkP5WLtKWeszdilr+z sjZftLZ3ZYXF6SvyUTSr9KIC1I03ex/O7kjwKmgKmzE8pmcF8oIUkZL3IyDOv7usq44lW PLnUBisu5Qu1JD1wFspAq/zXL417hoA3zC4z+48ato3nbjKEtlRHkQozmozq77Rui6TFL TdCzkqoN7/xkqrZnDjcYj4u7PUIjFdv0Gs9u6q2i1zeAD5usJIdzjnNNwUGkrD2qgx9nc GpNXtoeKJGXPeHirscww3XDppkdmAp0J0kioEHBD6fC/PZp4joHWAzDtX3rcyucwmrQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794524124; h=Received:DKIM-Signature:From:Date:Subject:MIME-Version:Content-Type: Content-Transfer-Encoding:Message-Id:References:In-Reply-To:To:Cc: X-Mailer:X-Developer-Signature:X-Developer-Key; bh=YtuAsfhpH9Wi2rdBATYS1f2ORNrPFRRfK6WhBQnQnFY=; b=pvHtqYCZCd6SO5+F8qgtFFW2C78NQvdLGlCgMEFBkA3AUjO5dcv9VBI3/fAs52KJWS7t zv414ouaPxdb00RO3UBI0CHZnLtxNDmMHfDJkKeJgNkJU3kx1+lyE7AHsf+eni2g1111b Cs0Taon7Bz7SIqAq1ra2nHrb9ZqkSMTppCrJ3kcnPnA14383TcjFlGoolfDW8hiBtoFHe LkDOWw82WfCKsEztdKkVN6/IstUo3mJW/iEt1JFG43LysFQ92J/nUc9pMuy99mcpIDulx Cp0IRoB+YRsVFGh6zcby5ODKi/CN5uU1Xtr8z3bFz0BOJXoMwns0UEWm9sumnbAzg7Q== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=Gdwb1klD; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 835D7606A31C; Thu, 12 Nov 2026 22:55:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=Gdwb1klD Received: by smtp.kernel.org (Postfix) with ESMTPSA id 67AFD1F000C4; Thu, 12 Nov 2026 22:55:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1794524124; bh=YtuAsfhpH9Wi2rdBATYS1f2ORNrPFRRfK6WhBQnQnFY=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=Gdwb1klDz3rDpk33v9KdY79kgGQ0z0CsT+1QzSa3AHhNT1VfKpS4dY+elh8Vh8zzH 7200U0MJ77EZV1wdBXuZV029q8TE1VwwQjCttb9LRSTGvr5Z8FCdqbrkWZsQnnsD9V zJu7Bl4EyiJZbUfuXY9DaKDPb2tzs/HZl3g4HENFg2Gt727/oin3dVOfZ84cKN+xee Cj1BrMCXhArTBYRhjfo1rmIzOL8h5ZNgAxfQMZTz1+rgjr5vBHHN1uGBOOqpWJHRt1 w8G68tIb6/C7RB1968O36s+e1pTDqLz9pQSndAdpjgIfMxsNgYQmqtl5kv1aupQh36 mlJrgkpih+DeQ== From: "David Hildenbrand (Arm)" Date: Thu, 12 Nov 2026 23:53:25 +0100 Subject: [PATCH v4 6/9] net: limit folio refcount when copying skb frag references X-Mailing-List: netdev@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261112-refcount_overflow_stable-v4-6-ad3deef0bd23@kernel.org> References: <20261112-refcount_overflow_stable-v4-0-ad3deef0bd23@kernel.org> In-Reply-To: <20261112-refcount_overflow_stable-v4-0-ad3deef0bd23@kernel.org> To: Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , "Matthew Wilcox (Oracle)" , Jan Kara , Andrew Morton , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Matthew Brost , Joshua Hahn , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Peter Xu , Johannes Weiner , Song Liu , Rik van Riel , Miklos Szeredi , Alexander Viro , Christian Brauner , Hugh Dickins , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , Arnd Bergmann , Jason Gunthorpe , Kevin Tian , "Joerg Roedel (AMD)" , Robin Murphy , Steve Sistare , John Hubbard , "Jose A. Perez de Azpillaga" , Josef Bacik , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Neal Cardwell , Kuniyuki Iwashima Cc: "David Hildenbrand (Arm)" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, fuse-devel@lists.linux.dev, linux-arch@vger.kernel.org, iommu@lists.linux.dev, netdev@vger.kernel.org, stable@vger.kernel.org, Pedro Falcato X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7095; i=david@kernel.org; h=from:subject:message-id; bh=xba2U2hB9W37nLQj4Nz5OSKhUBoSCCtXjwLlqARMq2s=; b=owGbwMvMwCF2/OVpCdfQ3ZWMp9WSGLK+OWfpuWcejPu65Py2edJ3/FU1+pe87WU92bo7MubIj hdba+t6O0pZGMQ4GGTFFFmCgu8bvHy5rrbajY8ZZg4rE8gQBi5OAZjIxzMM/4PdsudUmN/lyp33 Vv52is1b5mo152nFAXNlZ+muSlvb+Ifhf1Q1c/2uRa6THvNGsTJ5PC0uf7Jpu6PM5y8MU7c9K/y 0kBMA X-Developer-Key: i=david@kernel.org; a=openpgp; fpr=1BD9CAAD735C4C3A460DFCCA4DDE10F700FF835A Various networking features copy frag references of an skb, for example when unsharing a cloned skb (AF_PACKET trimming a clone), when copying an skb (nft dup) or when building NFQUEUE messages. This is an effective way for multiplying folio references held by an skb, allowing for exceeding folio refcount limits. This is not specific to large pagecache folios, but with them it is a lot easier to reach the refcount limit, as page table mappings can cheaply create many references. Let's introduce skb_frag_try_ref() and skb_frags_try_ref(), which use try_get_page(), and use them in pskb_expand_head(), __pskb_copy_fclone(), skb_zerocopy() and pskb_carve_inside_*(). All of these can already fail, and their callers handle that. Fixes: 800d8c63b2e9 ("shmem: add huge pages support") Fixes: 99cb0dbd47a1 ("mm,thp: add read-only THP support for (non-shmem) FS") Fixes: 793917d997df ("mm/readahead: Add large folio readahead") Cc: stable@vger.kernel.org Acked-by: Pedro Falcato Signed-off-by: David Hildenbrand (Arm) --- include/linux/skbuff_ref.h | 53 ++++++++++++++++++++++++++++++++++++++++++++++ net/core/skbuff.c | 35 +++++++++++++++++++----------- 2 files changed, 76 insertions(+), 12 deletions(-) diff --git a/include/linux/skbuff_ref.h b/include/linux/skbuff_ref.h index b6f6da556ee90..560ddc863bbec 100644 --- a/include/linux/skbuff_ref.h +++ b/include/linux/skbuff_ref.h @@ -44,6 +44,33 @@ static __always_inline void skb_frag_ref(struct sk_buff *skb, int f) skb->pp_recycle); } +/** + * skb_frag_try_ref - try to acquire a reference on a paged fragment of an skb + * @skb: the buffer + * @f: the fragment offset + * + * Like skb_frag_ref(), but acquires non-page_pool page references using + * try_get_page(), refusing additional references to a page whose refcount is + * at its limit. + * + * Return: True if a reference was acquired, otherwise false. + */ +static __always_inline __must_check bool skb_frag_try_ref(struct sk_buff *skb, + int f) +{ + netmem_ref netmem = skb_frag_netmem(&skb_shinfo(skb)->frags[f]); + +#ifdef CONFIG_PAGE_POOL + if (skb->pp_recycle && napi_pp_get_page(netmem)) + return true; +#endif + if (netmem_is_net_iov(netmem)) { + net_iov_inc_ref(netmem_to_net_iov(netmem)); + return true; + } + return try_get_page(netmem_to_page(netmem)); +} + bool napi_pp_put_page(netmem_ref netmem); /** @@ -83,4 +110,30 @@ static __always_inline void skb_frag_unref(struct sk_buff *skb, int f) skb->pp_recycle); } +/** + * skb_frags_try_ref - try to acquire references on all paged fragments + * @skb: the buffer + * + * Try to acquire a reference on each paged fragment of @skb using + * skb_frag_try_ref(). On failure, acquired references are dropped. + * + * Must not be used on skbs with managed frag references. + * + * Return: True if all references were acquired, otherwise false. + */ +static inline __must_check bool skb_frags_try_ref(struct sk_buff *skb) +{ + int i; + + DEBUG_NET_WARN_ON_ONCE(skb_zcopy_managed(skb)); + for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) { + if (unlikely(!skb_frag_try_ref(skb, i))) { + while (i--) + skb_frag_unref(skb, i); + return false; + } + } + return true; +} + #endif /* _LINUX_SKBUFF_REF_H */ diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 8da53d5b76fc2..1d44e9a4152e5 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -2254,7 +2254,13 @@ struct sk_buff *__pskb_copy_fclone(struct sk_buff *skb, int headroom, n->pp_recycle = skb->pp_recycle; for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) { skb_shinfo(n)->frags[i] = skb_shinfo(skb)->frags[i]; - skb_frag_ref(n, i); + if (!skb_frag_try_ref(n, i)) { + /* Drop the references acquired so far. */ + skb_shinfo(n)->nr_frags = i; + kfree_skb(n); + n = NULL; + goto out; + } } skb_shinfo(n)->nr_frags = i; skb_shinfo(n)->flags |= skb_shinfo(skb)->flags & SKBFL_SHARED_FRAG; @@ -2298,7 +2304,6 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail, unsigned int size = osize + nhead + ntail; long off; u8 *data; - int i; if (unlikely(nhead < 0)) { DEBUG_NET_WARN_ON_ONCE(1); @@ -2335,12 +2340,10 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail, * be since all we did is relocate the values */ if (skb_cloned(skb)) { - if (skb_orphan_frags(skb, gfp_mask)) + if (skb_orphan_frags(skb, gfp_mask) || !skb_frags_try_ref(skb)) goto nofrags; if (skb_zcopy(skb)) net_zcopy_get(skb_uarg(skb)); - for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) - skb_frag_ref(skb, i); if (skb_has_frag_list(skb)) skb_clone_fraglist(skb); @@ -3896,7 +3899,8 @@ EXPORT_SYMBOL_GPL(skb_zerocopy_headlen); * * Return value: * 0: everything is OK - * -ENOMEM: couldn't orphan frags of @from due to lack of memory + * -ENOMEM: couldn't orphan frags of @from due to lack of memory, or + * couldn't take additional references to the frags of @from * -EFAULT: skb_copy_bits() found some problem with skb geometry, or readable head * payload would be mixed with unreadable frags. */ @@ -3961,7 +3965,11 @@ skb_zerocopy(struct sk_buff *to, struct sk_buff *from, int len, int hlen) len); skb_frag_size_set(&skb_shinfo(to)->frags[j], size); len -= size; - skb_frag_ref(to, j); + if (!skb_frag_try_ref(to, j)) { + /* The caller frees @to, dropping the acquired refs. */ + skb_shinfo(to)->nr_frags = j; + return -ENOMEM; + } j++; } skb_shinfo(to)->nr_frags = j; @@ -6966,7 +6974,6 @@ static void skb_carve_reset_headers(struct sk_buff *skb) static int pskb_carve_inside_header(struct sk_buff *skb, const u32 off, const int headlen, gfp_t gfp_mask) { - int i; unsigned int size = skb_end_offset(skb); int new_hlen = headlen - off; u8 *data; @@ -6994,14 +7001,12 @@ static int pskb_carve_inside_header(struct sk_buff *skb, const u32 off, frags[skb_shinfo(skb)->nr_frags])); if (skb_cloned(skb)) { /* drop the old head gracefully */ - if (skb_orphan_frags(skb, gfp_mask)) { + if (skb_orphan_frags(skb, gfp_mask) || !skb_frags_try_ref(skb)) { skb_kfree_head(data); return -ENOMEM; } if (skb_zcopy(skb)) net_zcopy_get(skb_uarg(skb)); - for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) - skb_frag_ref(skb, i); if (skb_has_frag_list(skb)) skb_clone_fraglist(skb); skb_release_data(skb, SKB_CONSUMED); @@ -7131,7 +7136,13 @@ static int pskb_carve_inside_nonlinear(struct sk_buff *skb, const u32 off, skb_frag_off_add(&shinfo->frags[0], off - pos); skb_frag_size_sub(&shinfo->frags[0], off - pos); } - skb_frag_ref(skb, i); + if (!skb_frag_try_ref(skb, i)) { + /* We referenced frags i - k ... i - 1 so far. */ + while (k--) + skb_frag_unref(skb, --i); + skb_kfree_head(data); + return -ENOMEM; + } k++; } pos += fsize; -- 2.56.0