From: "Björn Töpel" <bjorn@kernel.org>
To: Mina Almasry <almasrymina@google.com>
Cc: Magnus Karlsson <magnus.karlsson@intel.com>,
Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
Stanislav Fomichev <sdf@fomichev.me>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Simon Horman <horms@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Randy Dunlap <rdunlap@infradead.org>,
Alexander Duyck <alexanderduyck@fb.com>,
kernel-team@meta.com, Andrew Lunn <andrew+netdev@lunn.ch>,
Jesper Dangaard Brouer <hawk@kernel.org>,
Ilias Apalodimas <ilias.apalodimas@linaro.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
John Fastabend <john.fastabend@gmail.com>,
Pavel Begunkov <asml.silence@gmail.com>,
Jens Axboe <axboe@kernel.dk>, Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Song Liu <song@kernel.org>,
Yonghong Song <yonghong.song@linux.dev>,
Jiri Olsa <jolsa@kernel.org>,
Emil Tsalapatis <emil@etsalapatis.com>,
Ihor Solodrai <ihor.solodrai@linux.dev>,
netdev@vger.kernel.org, bpf@vger.kernel.org,
io-uring@vger.kernel.org,
"Mike Marciniszyn (Meta)" <mike.marciniszyn@gmail.com>,
Weiming Shi <bestswngs@gmail.com>,
Nikolay Aleksandrov <razor@blackwall.org>,
David Wei <dw@davidwei.uk>,
Alexander Lobakin <aleksander.lobakin@intel.com>,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
Stuart Clark <stuclar@meta.com>
Subject: Re: [PATCH net-next v1 02/11] net: Add memory provider capabilities
Date: Tue, 17 Nov 2026 18:06:52 +0100 [thread overview]
Message-ID: <87zev71j6b.fsf@all.your.base.are.belong.to.us> (raw)
In-Reply-To: <CAHS8izNVDU8+JzeuwDBmF24C-XqkO+ox01qnTvK665Y8pyWNLA@mail.gmail.com>
Mina Almasry <almasrymina@google.com> writes:
> On Sun, Nov 15, 2026 at 10:47 AM Björn Töpel <bjorn@kernel.org> wrote:
>>
>> Mina Almasry <almasrymina@google.com> writes:
>>
>> > On Tue, Nov 10, 2026 at 2:21 AM Björn Töpel <bjorn@kernel.org> wrote:
>> >>
>> >> The core treats every memory provider as unreadable: installing one
>> >> needs header split and no XDP program, and any page pool that allows
>> >> unreadable netmem takes it. An AF_XDP UMEM provider is readable, needs
>> >> XDP, and hands out whole buffers.
>> >>
>> >> Describe providers by MP_CAP_READABLE and MP_CAP_FRAG; devmem and
>> >> io_uring keep their behavior with MP_CAP_FRAG. A page pool takes a
>> >> readable provider only with the new PP_FLAG_MEM_PROVIDER:
>> >
>> > Why do we need this? I thought whatever creates the pool actually
>> > shouldn't get to pick whether the pool picks up the memory provider. I
>> > thought if core has configured the rxq for a memory provider then the
>> > pool is forced to take it.
>>
>> Thanks for looking at series, Mina. ...this needed some thinking.
>>
>> It isn't forced today either: only pools with
>> PP_FLAG_ALLOW_UNREADABLE_NETMEM take the queue's provider. Drivers set
>> it on the pool that holds payloads, and keep headers in host memory: in
>> a separate pool (fbnic, bnxt), or outside the page pool (gve, mlx5).
>> That works because every provider so far backs payloads only.
>>
>> UMEM doesn't. The socket gets whole packets, so UMEM has to back the
>> header buffers, and with multi-buffer the payloads too. The flag was my
>> way to get it there, but you're right that it's the wrong knob: it lets
>> the driver opt in, where it should only say what each pool holds.
>>
>> How about this instead. Both sides use the rx_roles bits from patch 5,
>> HEAD and PAYLOAD:
>>
>> - The provider says which buffers it backs, next to rx_page_size in
>> pp_memory_provider_params: devmem and io_uring zcrx PAYLOAD, AF_XDP
>> HEAD, or HEAD|PAYLOAD with multi-buffer.
>> - The driver says what each pool holds, in rx_roles. A pool without
>> roles but with PP_FLAG_ALLOW_UNREADABLE_NETMEM holds PAYLOAD, so
>> existing drivers don't change.
>> - page_pool_create() gives the provider to a pool whose roles are all
>> backed; the others get host pages. No opt-in.
>> - After ndo_queue_mem_alloc(), the core checks that the new pools that
>> took the provider cover everything it backs, or fails the restart.
>>
>> With fbnic's header and payload pools:
>>
>> devmem: header pool host, payload pool devmem
>> AF_XDP single-buffer: header pool UMEM, payload pool host
>> AF_XDP multi-buffer: one shared pool, UMEM
>>
>> A shared pool on a devmem queue doesn't take devmem, since devmem
>> doesn't back HEAD, so the check fails and the driver has to split.
>> That's the "headers in host memory" rule, without the driver asking
>> which provider it is.
>>
>> Wdyt?
>>
>
> Hmm, I think maybe the exsiting knobs PP_FLAG_ALLOW_UNREADABLE +
> CAP_READABLE covers your needs.
>
> The pp says whether the memory it provides is readable or not via
> CAP_READABLE (or !CAP_UNREADABLE).
>
> The driver says whether it will allow unreadable memory for this pool.
> For header pools, it doesn't specfiy PP_FLAG_ALLOW_UNREADABLE and for
> paylaod pools it specifies ALLOW_UNREADABLE.
>
> The change I think you need to make is: today we disallow any
> providers if the caller did not specify ALLOW_UNREADABLE (because we
> assume mp's are unreadable). After your series we only need to
> disallow memory providers if the mp is unreadable and the caller said
> it's not fine with unreadable.
I'm all in on making v1 simpler -- I'll try this path! So, an unreadable
provider still goes only to pools that set ALLOW_UNREADABLE. A readable
provider goes to all RX pools of the queue. The roles and
PP_FLAG_MEM_PROVIDER are gone.
One addition: a driver must opt in with a readable_mp flag in its
queue ops. Without it, its pools don't look for a provider. The reason
is that today only ALLOW_UNREADABLE pools must set queue_idx and be
created under the instance lock. Other pools may not, so they could
pick up the wrong queue's provider. Maybe there's a better way?
Thanks!
Björn
next prev parent reply other threads:[~2026-11-17 17:06 UTC|newest]
Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-10 10:20 [PATCH net-next v1 00/11] xsk: Zero copy through page-pool memory providers Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 01/11] net: Get skb fragment addresses through netmem Björn Töpel
2026-11-12 8:32 ` saeed bishara
2026-11-12 13:57 ` Björn Töpel
2026-11-13 5:17 ` Mina Almasry
2026-11-13 9:08 ` Björn Töpel
2026-11-15 21:13 ` Jakub Kicinski
2026-11-16 14:31 ` Björn Töpel
2026-11-18 0:32 ` Jakub Kicinski
2026-11-10 10:20 ` [PATCH net-next v1 02/11] net: Add memory provider capabilities Björn Töpel
2026-11-10 22:19 ` Stanislav Fomichev
2026-11-11 12:41 ` Björn Töpel
2026-11-11 23:01 ` Stanislav Fomichev
2026-11-12 14:01 ` Björn Töpel
2026-11-13 0:21 ` Stanislav Fomichev
2026-11-15 18:47 ` Björn Töpel
2026-11-11 10:22 ` sashiko-bot
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-13 5:30 ` Mina Almasry
2026-11-15 18:46 ` Björn Töpel
2026-11-16 21:01 ` Mina Almasry
2026-11-17 17:06 ` Björn Töpel [this message]
2026-11-15 21:18 ` Jakub Kicinski
2026-11-16 14:20 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 03/11] xdp: Copy buffers with restricted lifetime Björn Töpel
2026-11-10 22:32 ` Stanislav Fomichev
2026-11-11 13:27 ` Björn Töpel
2026-11-11 23:05 ` Stanislav Fomichev
2026-11-11 10:22 ` sashiko-bot
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:02 ` Björn Töpel
2026-11-13 18:54 ` Mina Almasry
2026-11-15 18:52 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 04/11] xsk: Allocate UMEM chunks through page_pool Björn Töpel
2026-11-10 15:08 ` Magnus Karlsson
2026-11-11 13:51 ` Björn Töpel
2026-11-10 23:02 ` Stanislav Fomichev
2026-11-11 13:59 ` Björn Töpel
2026-11-11 23:09 ` Stanislav Fomichev
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-13 19:13 ` Mina Almasry
2026-11-15 18:55 ` Björn Töpel
2026-11-16 21:21 ` Mina Almasry
2026-11-17 17:15 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 05/11] xsk: Install UMEM page pool providers in the core Björn Töpel
2026-11-10 23:50 ` Stanislav Fomichev
2026-11-11 14:01 ` Björn Töpel
2026-11-11 8:59 ` Magnus Karlsson
2026-11-11 19:26 ` Björn Töpel
2026-11-12 8:08 ` Magnus Karlsson
2026-11-11 10:22 ` sashiko-bot
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:05 ` Björn Töpel
2026-11-12 11:53 ` Maciej Fijalkowski
2026-11-12 14:48 ` Björn Töpel
2026-11-12 15:43 ` Maciej Fijalkowski
2026-11-12 19:51 ` Björn Töpel
2026-11-13 19:17 ` Mina Almasry
2026-11-15 18:58 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 06/11] page_pool: Coordinate refill with memory providers Björn Töpel
2026-11-10 23:51 ` Stanislav Fomichev
2026-11-11 19:37 ` Björn Töpel
2026-11-11 23:13 ` Stanislav Fomichev
2026-11-12 14:53 ` Björn Töpel
2026-11-11 10:22 ` sashiko-bot
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:17 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 07/11] xsk: Deliver provider chunks without copying Björn Töpel
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:23 ` Björn Töpel
2026-11-11 23:20 ` Stanislav Fomichev
2026-11-12 15:10 ` Björn Töpel
2026-11-12 12:17 ` Maciej Fijalkowski
2026-11-12 19:57 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 08/11] eth: fbnic: Support XDP redirect Björn Töpel
2026-11-11 10:22 ` sashiko-bot
2026-11-10 10:20 ` [PATCH net-next v1 09/11] eth: fbnic: Use netmem for RX headers and XDP_TX Björn Töpel
2026-11-13 19:27 ` Mina Almasry
2026-11-10 10:20 ` [PATCH net-next v1 10/11] eth: fbnic: Support RX buffers it cannot split Björn Töpel
2026-11-11 10:22 ` sashiko-bot
2026-11-11 21:45 ` Dimitri Daskalakis
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:12 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 11/11] eth: fbnic: Support AF_XDP page pool zero copy Björn Töpel
2026-11-11 10:22 ` sashiko-bot
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:25 ` Björn Töpel
2026-11-12 11:37 ` Maciej Fijalkowski
2026-11-12 20:03 ` Björn Töpel
2026-11-13 11:40 ` Björn Töpel
2026-11-13 16:28 ` Maciej Fijalkowski
2026-11-15 18:59 ` Björn Töpel
2026-11-16 18:48 ` Dimitri Daskalakis
2026-11-17 17:14 ` Björn Töpel
2026-11-18 0:35 ` [PATCH net-next v1 00/11] xsk: Zero copy through page-pool memory providers Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87zev71j6b.fsf@all.your.base.are.belong.to.us \
--to=bjorn@kernel.org \
--cc=aleksander.lobakin@intel.com \
--cc=alexanderduyck@fb.com \
--cc=almasrymina@google.com \
--cc=andrew+netdev@lunn.ch \
--cc=andrii@kernel.org \
--cc=asml.silence@gmail.com \
--cc=ast@kernel.org \
--cc=axboe@kernel.dk \
--cc=bestswngs@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dw@davidwei.uk \
--cc=eddyz87@gmail.com \
--cc=edumazet@kernel.org \
--cc=emil@etsalapatis.com \
--cc=hawk@kernel.org \
--cc=horms@kernel.org \
--cc=ihor.solodrai@linux.dev \
--cc=ilias.apalodimas@linaro.org \
--cc=io-uring@vger.kernel.org \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=kernel-team@meta.com \
--cc=kuba@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maciej.fijalkowski@intel.com \
--cc=magnus.karlsson@intel.com \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
--cc=mike.marciniszyn@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=razor@blackwall.org \
--cc=rdunlap@infradead.org \
--cc=sdf@fomichev.me \
--cc=skhan@linuxfoundation.org \
--cc=song@kernel.org \
--cc=stuclar@meta.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.