From: "Björn Töpel" <bjorn@kernel.org>
To: Mina Almasry <almasrymina@google.com>
Cc: Magnus Karlsson <magnus.karlsson@intel.com>,
Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
Stanislav Fomichev <sdf@fomichev.me>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Simon Horman <horms@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Randy Dunlap <rdunlap@infradead.org>,
Alexander Duyck <alexanderduyck@fb.com>,
kernel-team@meta.com, Andrew Lunn <andrew+netdev@lunn.ch>,
Jesper Dangaard Brouer <hawk@kernel.org>,
Ilias Apalodimas <ilias.apalodimas@linaro.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
John Fastabend <john.fastabend@gmail.com>,
Pavel Begunkov <asml.silence@gmail.com>,
Jens Axboe <axboe@kernel.dk>, Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Song Liu <song@kernel.org>,
Yonghong Song <yonghong.song@linux.dev>,
Jiri Olsa <jolsa@kernel.org>,
Emil Tsalapatis <emil@etsalapatis.com>,
Ihor Solodrai <ihor.solodrai@linux.dev>,
netdev@vger.kernel.org, bpf@vger.kernel.org,
io-uring@vger.kernel.org,
"Mike Marciniszyn (Meta)" <mike.marciniszyn@gmail.com>,
Weiming Shi <bestswngs@gmail.com>,
Nikolay Aleksandrov <razor@blackwall.org>,
David Wei <dw@davidwei.uk>,
Alexander Lobakin <aleksander.lobakin@intel.com>,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
Stuart Clark <stuclar@meta.com>
Subject: Re: [PATCH net-next v1 02/11] net: Add memory provider capabilities
Date: Tue, 17 Nov 2026 18:06:52 +0100 [thread overview]
Message-ID: <87zev71j6b.fsf@all.your.base.are.belong.to.us> (raw)
In-Reply-To: <CAHS8izNVDU8+JzeuwDBmF24C-XqkO+ox01qnTvK665Y8pyWNLA@mail.gmail.com>
Mina Almasry <almasrymina@google.com> writes:
> On Sun, Nov 15, 2026 at 10:47 AM Björn Töpel <bjorn@kernel.org> wrote:
>>
>> Mina Almasry <almasrymina@google.com> writes:
>>
>> > On Tue, Nov 10, 2026 at 2:21 AM Björn Töpel <bjorn@kernel.org> wrote:
>> >>
>> >> The core treats every memory provider as unreadable: installing one
>> >> needs header split and no XDP program, and any page pool that allows
>> >> unreadable netmem takes it. An AF_XDP UMEM provider is readable, needs
>> >> XDP, and hands out whole buffers.
>> >>
>> >> Describe providers by MP_CAP_READABLE and MP_CAP_FRAG; devmem and
>> >> io_uring keep their behavior with MP_CAP_FRAG. A page pool takes a
>> >> readable provider only with the new PP_FLAG_MEM_PROVIDER:
>> >
>> > Why do we need this? I thought whatever creates the pool actually
>> > shouldn't get to pick whether the pool picks up the memory provider. I
>> > thought if core has configured the rxq for a memory provider then the
>> > pool is forced to take it.
>>
>> Thanks for looking at series, Mina. ...this needed some thinking.
>>
>> It isn't forced today either: only pools with
>> PP_FLAG_ALLOW_UNREADABLE_NETMEM take the queue's provider. Drivers set
>> it on the pool that holds payloads, and keep headers in host memory: in
>> a separate pool (fbnic, bnxt), or outside the page pool (gve, mlx5).
>> That works because every provider so far backs payloads only.
>>
>> UMEM doesn't. The socket gets whole packets, so UMEM has to back the
>> header buffers, and with multi-buffer the payloads too. The flag was my
>> way to get it there, but you're right that it's the wrong knob: it lets
>> the driver opt in, where it should only say what each pool holds.
>>
>> How about this instead. Both sides use the rx_roles bits from patch 5,
>> HEAD and PAYLOAD:
>>
>> - The provider says which buffers it backs, next to rx_page_size in
>> pp_memory_provider_params: devmem and io_uring zcrx PAYLOAD, AF_XDP
>> HEAD, or HEAD|PAYLOAD with multi-buffer.
>> - The driver says what each pool holds, in rx_roles. A pool without
>> roles but with PP_FLAG_ALLOW_UNREADABLE_NETMEM holds PAYLOAD, so
>> existing drivers don't change.
>> - page_pool_create() gives the provider to a pool whose roles are all
>> backed; the others get host pages. No opt-in.
>> - After ndo_queue_mem_alloc(), the core checks that the new pools that
>> took the provider cover everything it backs, or fails the restart.
>>
>> With fbnic's header and payload pools:
>>
>> devmem: header pool host, payload pool devmem
>> AF_XDP single-buffer: header pool UMEM, payload pool host
>> AF_XDP multi-buffer: one shared pool, UMEM
>>
>> A shared pool on a devmem queue doesn't take devmem, since devmem
>> doesn't back HEAD, so the check fails and the driver has to split.
>> That's the "headers in host memory" rule, without the driver asking
>> which provider it is.
>>
>> Wdyt?
>>
>
> Hmm, I think maybe the exsiting knobs PP_FLAG_ALLOW_UNREADABLE +
> CAP_READABLE covers your needs.
>
> The pp says whether the memory it provides is readable or not via
> CAP_READABLE (or !CAP_UNREADABLE).
>
> The driver says whether it will allow unreadable memory for this pool.
> For header pools, it doesn't specfiy PP_FLAG_ALLOW_UNREADABLE and for
> paylaod pools it specifies ALLOW_UNREADABLE.
>
> The change I think you need to make is: today we disallow any
> providers if the caller did not specify ALLOW_UNREADABLE (because we
> assume mp's are unreadable). After your series we only need to
> disallow memory providers if the mp is unreadable and the caller said
> it's not fine with unreadable.
I'm all in on making v1 simpler -- I'll try this path! So, an unreadable
provider still goes only to pools that set ALLOW_UNREADABLE. A readable
provider goes to all RX pools of the queue. The roles and
PP_FLAG_MEM_PROVIDER are gone.
One addition: a driver must opt in with a readable_mp flag in its
queue ops. Without it, its pools don't look for a provider. The reason
is that today only ALLOW_UNREADABLE pools must set queue_idx and be
created under the instance lock. Other pools may not, so they could
pick up the wrong queue's provider. Maybe there's a better way?
Thanks!
Björn
next prev parent reply other threads:[~2026-11-17 17:06 UTC|newest]
Thread overview: 86+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-10 10:20 [PATCH net-next v1 00/11] xsk: Zero copy through page-pool memory providers Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 01/11] net: Get skb fragment addresses through netmem Björn Töpel
2026-11-12 8:32 ` saeed bishara
2026-11-12 13:57 ` Björn Töpel
2026-11-13 5:17 ` Mina Almasry
2026-11-13 9:08 ` Björn Töpel
2026-11-15 21:13 ` Jakub Kicinski
2026-11-16 14:31 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 02/11] net: Add memory provider capabilities Björn Töpel
2026-11-10 22:19 ` Stanislav Fomichev
2026-11-11 12:41 ` Björn Töpel
2026-11-11 23:01 ` Stanislav Fomichev
2026-11-12 14:01 ` Björn Töpel
2026-11-13 0:21 ` Stanislav Fomichev
2026-11-15 18:47 ` Björn Töpel
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-13 5:30 ` Mina Almasry
2026-11-15 18:46 ` Björn Töpel
2026-11-16 21:01 ` Mina Almasry
2026-11-17 17:06 ` Björn Töpel [this message]
2026-11-15 21:18 ` Jakub Kicinski
2026-11-16 14:20 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 03/11] xdp: Copy buffers with restricted lifetime Björn Töpel
2026-11-10 22:32 ` Stanislav Fomichev
2026-11-11 13:27 ` Björn Töpel
2026-11-11 23:05 ` Stanislav Fomichev
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:02 ` Björn Töpel
2026-11-13 18:54 ` Mina Almasry
2026-11-15 18:52 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 04/11] xsk: Allocate UMEM chunks through page_pool Björn Töpel
2026-11-10 15:08 ` Magnus Karlsson
2026-11-11 13:51 ` Björn Töpel
2026-11-10 23:02 ` Stanislav Fomichev
2026-11-11 13:59 ` Björn Töpel
2026-11-11 23:09 ` Stanislav Fomichev
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-13 19:13 ` Mina Almasry
2026-11-15 18:55 ` Björn Töpel
2026-11-16 21:21 ` Mina Almasry
2026-11-17 17:15 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 05/11] xsk: Install UMEM page pool providers in the core Björn Töpel
2026-11-10 23:50 ` Stanislav Fomichev
2026-11-11 14:01 ` Björn Töpel
2026-11-11 8:59 ` Magnus Karlsson
2026-11-11 19:26 ` Björn Töpel
2026-11-12 8:08 ` Magnus Karlsson
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:05 ` Björn Töpel
2026-11-12 11:53 ` Maciej Fijalkowski
2026-11-12 14:48 ` Björn Töpel
2026-11-12 15:43 ` Maciej Fijalkowski
2026-11-12 19:51 ` Björn Töpel
2026-11-13 19:17 ` Mina Almasry
2026-11-15 18:58 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 06/11] page_pool: Coordinate refill with memory providers Björn Töpel
2026-11-10 23:51 ` Stanislav Fomichev
2026-11-11 19:37 ` Björn Töpel
2026-11-11 23:13 ` Stanislav Fomichev
2026-11-12 14:53 ` Björn Töpel
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:17 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 07/11] xsk: Deliver provider chunks without copying Björn Töpel
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:23 ` Björn Töpel
2026-11-11 23:20 ` Stanislav Fomichev
2026-11-12 15:10 ` Björn Töpel
2026-11-12 12:17 ` Maciej Fijalkowski
2026-11-12 19:57 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 08/11] eth: fbnic: Support XDP redirect Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 09/11] eth: fbnic: Use netmem for RX headers and XDP_TX Björn Töpel
2026-11-13 19:27 ` Mina Almasry
2026-11-10 10:20 ` [PATCH net-next v1 10/11] eth: fbnic: Support RX buffers it cannot split Björn Töpel
2026-11-11 21:45 ` Dimitri Daskalakis
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:12 ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 11/11] eth: fbnic: Support AF_XDP page pool zero copy Björn Töpel
2026-11-11 22:41 ` netdev-bot+sashiko
2026-11-15 19:25 ` Björn Töpel
2026-11-12 11:37 ` Maciej Fijalkowski
2026-11-12 20:03 ` Björn Töpel
2026-11-13 11:40 ` Björn Töpel
2026-11-13 16:28 ` Maciej Fijalkowski
2026-11-15 18:59 ` Björn Töpel
2026-11-16 18:48 ` Dimitri Daskalakis
2026-11-17 17:14 ` Björn Töpel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87zev71j6b.fsf@all.your.base.are.belong.to.us \
--to=bjorn@kernel.org \
--cc=aleksander.lobakin@intel.com \
--cc=alexanderduyck@fb.com \
--cc=almasrymina@google.com \
--cc=andrew+netdev@lunn.ch \
--cc=andrii@kernel.org \
--cc=asml.silence@gmail.com \
--cc=ast@kernel.org \
--cc=axboe@kernel.dk \
--cc=bestswngs@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dw@davidwei.uk \
--cc=eddyz87@gmail.com \
--cc=edumazet@kernel.org \
--cc=emil@etsalapatis.com \
--cc=hawk@kernel.org \
--cc=horms@kernel.org \
--cc=ihor.solodrai@linux.dev \
--cc=ilias.apalodimas@linaro.org \
--cc=io-uring@vger.kernel.org \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=kernel-team@meta.com \
--cc=kuba@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maciej.fijalkowski@intel.com \
--cc=magnus.karlsson@intel.com \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
--cc=mike.marciniszyn@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=razor@blackwall.org \
--cc=rdunlap@infradead.org \
--cc=sdf@fomichev.me \
--cc=skhan@linuxfoundation.org \
--cc=song@kernel.org \
--cc=stuclar@meta.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox