[PATCH net-next] page_pool: always add GFP_NOWARN for ATOMIC allocations

[PATCH net-next] page_pool: always add GFP_NOWARN for ATOMIC allocations

[Jakub Kicinski]

Driver authors often forget to add GFP_NOWARN for page allocation
from the datapath. This is annoying to operators as OOMs are a fact
of life, and we pretty much expect network Rx to hit page allocation
failures during OOM. Make page pool add GFP_NOWARN for ATOMIC allocations
by default.

Don't compare to GFP_ATOMIC because it's a mask with 2 bits set.
We want a single bit so that the compiler can do an unconditional
mask and shift. clang builds the condition as:

    1c31: 89 e8                        	movl	%ebp, %eax
    1c33: 83 e0 20                     	andl	$0x20, %eax
    1c36: c1 e0 0d                     	shll	$0xd, %eax
    1c39: 09 e8                        	orl	%ebp, %eax

so there seems to be no need any more to use the old flag multiplication
tricks which is less readable. Pick the lowest bit out of GFP_ATOMIC
to limit the size of the instructions.

The specific change which makes me propose this is that bnxt, after
commit cd1fafe7da1f ("eth: bnxt: add support rx side device memory TCP"),
lost the GFP_NOWARN, again. It used to allocate with page_pool_dev_alloc_*
which added the NOWARN unconditionally. While switching to
__bnxt_alloc_rx_netmem() authors forgot to add NOWARN in the explicitly
specified flags.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: hawk@kernel.org
CC: ilias.apalodimas@linaro.org
CC: nathan@kernel.org
CC: nick.desaulniers+lkml@gmail.com
CC: morbo@google.com
CC: justinstitt@google.com
CC: llvm@lists.linux.dev
---
 net/core/page_pool.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index ba70569bd4b0..6ffce0e821e4 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -555,6 +555,13 @@ static noinline netmem_ref __page_pool_alloc_netmems_slow(struct page_pool *pool
 	netmem_ref netmem;
 	int i, nr_pages;
 
+	/* Unconditionally set NOWARN if allocating from the datapath.
+	 * Use a single bit from the ATOMIC mask to help compiler optimize.
+	 */
+	BUILD_BUG_ON(!(GFP_ATOMIC & __GFP_HIGH));
+	if (gfp & __GFP_HIGH)
+		gfp |= __GFP_NOWARN;
+
 	/* Don't support bulk alloc for high-order pages */
 	if (unlikely(pp_order))
 		return page_to_netmem(__page_pool_alloc_page_order(pool, gfp));
-- 
2.51.0

Re: [PATCH net-next] page_pool: always add GFP_NOWARN for ATOMIC allocations

[Mina Almasry]

On Mon, Sep 8, 2025 at 8:21 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> Driver authors often forget to add GFP_NOWARN for page allocation
> from the datapath. This is annoying to operators as OOMs are a fact
> of life, and we pretty much expect network Rx to hit page allocation
> failures during OOM. Make page pool add GFP_NOWARN for ATOMIC allocations
> by default.
>
> Don't compare to GFP_ATOMIC because it's a mask with 2 bits set.
> We want a single bit so that the compiler can do an unconditional
> mask and shift. clang builds the condition as:
>
>     1c31: 89 e8                         movl    %ebp, %eax
>     1c33: 83 e0 20                      andl    $0x20, %eax
>     1c36: c1 e0 0d                      shll    $0xd, %eax
>     1c39: 09 e8                         orl     %ebp, %eax
>
> so there seems to be no need any more to use the old flag multiplication
> tricks which is less readable. Pick the lowest bit out of GFP_ATOMIC
> to limit the size of the instructions.
>
> The specific change which makes me propose this is that bnxt, after
> commit cd1fafe7da1f ("eth: bnxt: add support rx side device memory TCP"),
> lost the GFP_NOWARN, again. It used to allocate with page_pool_dev_alloc_*

BTW, there is a page_pool_dev_alloc_netmems now that also add the
NOWARN and should have been devmem tcp compatible and maintained same
behavior I think.

> which added the NOWARN unconditionally. While switching to
> __bnxt_alloc_rx_netmem() authors forgot to add NOWARN in the explicitly
> specified flags.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> CC: hawk@kernel.org
> CC: ilias.apalodimas@linaro.org
> CC: nathan@kernel.org
> CC: nick.desaulniers+lkml@gmail.com
> CC: morbo@google.com
> CC: justinstitt@google.com
> CC: llvm@lists.linux.dev
> ---
>  net/core/page_pool.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/net/core/page_pool.c b/net/core/page_pool.c
> index ba70569bd4b0..6ffce0e821e4 100644
> --- a/net/core/page_pool.c
> +++ b/net/core/page_pool.c
> @@ -555,6 +555,13 @@ static noinline netmem_ref __page_pool_alloc_netmems_slow(struct page_pool *pool
>         netmem_ref netmem;
>         int i, nr_pages;
>
> +       /* Unconditionally set NOWARN if allocating from the datapath.
> +        * Use a single bit from the ATOMIC mask to help compiler optimize.
> +        */
> +       BUILD_BUG_ON(!(GFP_ATOMIC & __GFP_HIGH));
> +       if (gfp & __GFP_HIGH)
> +               gfp |= __GFP_NOWARN;
> +

I wonder if pp allocs are ever used for anything other than datapath
pages (and if not, we can add __GPF_NOWARN here unconditionally. But
this is good too I think.

Reviewed-by: Mina Almasry <almasrymina@google.com>

-- 
Thanks,
Mina

Re: [PATCH net-next] page_pool: always add GFP_NOWARN for ATOMIC allocations

[Jakub Kicinski]

On Mon, 8 Sep 2025 09:15:07 -0700 Mina Almasry wrote:
> > +       /* Unconditionally set NOWARN if allocating from the datapath.
> > +        * Use a single bit from the ATOMIC mask to help compiler optimize.
> > +        */
> > +       BUILD_BUG_ON(!(GFP_ATOMIC & __GFP_HIGH));
> > +       if (gfp & __GFP_HIGH)
> > +               gfp |= __GFP_NOWARN;
> > +  
> 
> I wonder if pp allocs are ever used for anything other than datapath
> pages (and if not, we can add __GPF_NOWARN here unconditionally. But
> this is good too I think.

datapath == in NAPI context, here. We still want the warning if 
the allocations fail with GFP_KERNEL, e.g. during ndo_open.

Re: [PATCH net-next] page_pool: always add GFP_NOWARN for ATOMIC allocations

[Jakub Kicinski]

On Mon,  8 Sep 2025 08:21:23 -0700 Jakub Kicinski wrote:
> Driver authors often forget to add GFP_NOWARN for page allocation
> from the datapath. This is annoying to operators as OOMs are a fact
> of life, and we pretty much expect network Rx to hit page allocation
> failures during OOM. Make page pool add GFP_NOWARN for ATOMIC allocations
> by default.

Hi Jesper! Are you okay with this? It's not a lot of instructions and
it's in the _slow() function, anyway. TBH I wrote the patch to fix the
driver (again) first but when writing the commit message I realized my
explanation why we can't fix this in the core was sounding like BS :$

Re: [PATCH net-next] page_pool: always add GFP_NOWARN for ATOMIC allocations

[Jesper Dangaard Brouer]

On 11/09/2025 02.52, Jakub Kicinski wrote:
> On Mon,  8 Sep 2025 08:21:23 -0700 Jakub Kicinski wrote:
>> Driver authors often forget to add GFP_NOWARN for page allocation
>> from the datapath. This is annoying to operators as OOMs are a fact
>> of life, and we pretty much expect network Rx to hit page allocation
>> failures during OOM. Make page pool add GFP_NOWARN for ATOMIC allocations
>> by default.
> 
> Hi Jesper! Are you okay with this? [2] It's not a lot of instructions and
> it's in the _slow() function, anyway. 

For this "_slow()" function I don't worry about the performance.
The optimization you did seems a bit premature... did you run the
page_pool benchmark to see if it is worth it?

> TBH I wrote the patch to fix the
> driver (again) first but when writing the commit message I realized my
> explanation why we can't fix this in the core was sounding like BS :$

It feels slightly strange to fix drivers misuse of the API in the core,
but again I'm not going to nack it, as it might be easier for us as
maintainers as it is hard to catch all cases during driver review.

All driver are suppose to use page_pool_dev_alloc[1] as it sets
(GFP_ATOMIC | __GFP_NOWARN).  Or page_pool_dev_alloc_netmems().

  [1] https://elixir.bootlin.com/linux/v6.16.6/source/include/net/
page_pool/helpers.h#L175-L179

The reason I added page_pool_dev_alloc() is because all driver used to
call a function named dev_alloc_page(), that also sets the appropriate
GFP flags.  So, I simple piggybacked on that design decision (which I
don't know whom came up with). Thus, I'm open to change.  Maybe DaveM
can remember/explain why "dev_alloc" was a requirement.

I'll let it be up to you,
--Jesper

[2] https://lore.kernel.org/all/20250908152123.97829-1-kuba@kernel.org/