From: Bobby Eshleman <bobbyeshleman@gmail.com>
To: "David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>,
Paolo Abeni <pabeni@redhat.com>, Simon Horman <horms@kernel.org>,
Kuniyuki Iwashima <kuniyu@google.com>,
Willem de Bruijn <willemb@google.com>,
Neal Cardwell <ncardwell@google.com>,
David Ahern <dsahern@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
Jonathan Corbet <corbet@lwn.net>,
Andrew Lunn <andrew+netdev@lunn.ch>,
Shuah Khan <shuah@kernel.org>,
Donald Hunter <donald.hunter@gmail.com>,
Mina Almasry <almasrymina@google.com>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-arch@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kselftest@vger.kernel.org,
Stanislav Fomichev <sdf@fomichev.me>,
Bobby Eshleman <bobbyeshleman@meta.com>
Subject: [PATCH net-next v7 4/5] net: devmem: document NETDEV_A_DMABUF_AUTORELEASE netlink attribute
Date: Wed, 19 Nov 2025 19:37:11 -0800 [thread overview]
Message-ID: <20251119-scratch-bobbyeshleman-devmem-tcp-token-upstream-v7-4-1abc8467354c@meta.com> (raw)
In-Reply-To: <20251119-scratch-bobbyeshleman-devmem-tcp-token-upstream-v7-0-1abc8467354c@meta.com>
From: Bobby Eshleman <bobbyeshleman@meta.com>
Update devmem.rst documentation to describe the autorelease netlink
attribute used during RX dmabuf binding.
The autorelease attribute is specified at bind-time via the netlink API
(NETDEV_CMD_BIND_RX) and controls what happens to outstanding tokens
when the socket closes.
Document the two token release modes (automatic vs manual), how to
configure the binding for autorelease, the perf benefits, new caveats
and restrictions, and the way the mode is enforced system-wide.
Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
---
Changes in v7:
- Document netlink instead of sockopt
- Mention system-wide locked to one mode
---
Documentation/networking/devmem.rst | 70 +++++++++++++++++++++++++++++++++++++
1 file changed, 70 insertions(+)
diff --git a/Documentation/networking/devmem.rst b/Documentation/networking/devmem.rst
index a6cd7236bfbd..67c63bc5a7ae 100644
--- a/Documentation/networking/devmem.rst
+++ b/Documentation/networking/devmem.rst
@@ -235,6 +235,76 @@ can be less than the tokens provided by the user in case of:
(a) an internal kernel leak bug.
(b) the user passed more than 1024 frags.
+
+Autorelease Control
+~~~~~~~~~~~~~~~~~~~
+
+The autorelease mode controls what happens to outstanding tokens (tokens not
+released via SO_DEVMEM_DONTNEED) when the socket closes. Autorelease is
+configured per-binding at binding creation time via the netlink API::
+
+ struct netdev_bind_rx_req *req;
+ struct netdev_bind_rx_rsp *rsp;
+ struct ynl_sock *ys;
+ struct ynl_error yerr;
+
+ ys = ynl_sock_create(&ynl_netdev_family, &yerr);
+
+ req = netdev_bind_rx_req_alloc();
+ netdev_bind_rx_req_set_ifindex(req, ifindex);
+ netdev_bind_rx_req_set_fd(req, dmabuf_fd);
+ netdev_bind_rx_req_set_autorelease(req, 0); /* 0 = manual, 1 = auto */
+ __netdev_bind_rx_req_set_queues(req, queues, n_queues);
+
+ rsp = netdev_bind_rx(ys, req);
+
+ dmabuf_id = rsp->id;
+
+When autorelease is disabled (0):
+
+- Outstanding tokens are NOT released when the socket closes
+- Outstanding tokens are only released when the dmabuf is unbound
+- Provides better performance by eliminating xarray overhead (~13% CPU reduction)
+- Kernel tracks tokens via atomic reference counters in net_iov structures
+
+When autorelease is enabled (1):
+
+- Outstanding tokens are automatically released when the socket closes
+- Backwards compatible behavior
+- Kernel tracks tokens in an xarray per socket
+
+The default is autorelease disabled.
+
+Important: In both modes, applications should call SO_DEVMEM_DONTNEED to
+return tokens as soon as they are done processing. The autorelease setting only
+affects what happens to tokens that are still outstanding when close() is called.
+
+The mode is enforced system-wide. Once a binding is created with a specific
+autorelease mode, all subsequent bindings system-wide must use the same mode.
+
+
+Performance Considerations
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Disabling autorelease provides approximately ~13% CPU utilization improvement
+in RX workloads. That said, applications must ensure all tokens are released
+via SO_DEVMEM_DONTNEED before closing the socket, otherwise the backing pages
+will remain pinned until the dmabuf is unbound.
+
+
+Caveats
+~~~~~~~
+
+- Once a system-wide autorelease mode is selected (via the first binding),
+ all subsequent bindings must use the same mode. Attempts to create bindings
+ with a different mode will be rejected with -EINVAL.
+
+- Applications using manual release mode (autorelease=0) must ensure all tokens
+ are returned via SO_DEVMEM_DONTNEED before socket close to avoid resource
+ leaks during the lifetime of the dmabuf binding. Tokens not released before
+ close() will only be freed when the dmabuf is unbound.
+
+
TX Interface
============
--
2.47.3
next prev parent reply other threads:[~2025-11-20 3:37 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-20 3:37 [PATCH net-next v7 0/5] net: devmem: improve cpu cost of RX token management Bobby Eshleman
2025-11-20 3:37 ` [PATCH net-next v7 1/5] net: devmem: rename tx_vec to vec in dmabuf binding Bobby Eshleman
2025-11-20 3:37 ` [PATCH net-next v7 2/5] net: devmem: refactor sock_devmem_dontneed for autorelease split Bobby Eshleman
2025-11-20 3:37 ` [PATCH net-next v7 3/5] net: devmem: implement autorelease token management Bobby Eshleman
2025-11-20 12:19 ` Paolo Abeni
2025-11-20 15:31 ` Bobby Eshleman
2025-11-21 16:39 ` Stanislav Fomichev
2025-11-21 17:58 ` [PATCH net-next v7 3/5] net: devmem: implement autorelease token managementy Bobby Eshleman
2025-11-26 10:11 ` [PATCH net-next v7 3/5] net: devmem: implement autorelease token management Simon Horman
2025-11-20 3:37 ` Bobby Eshleman [this message]
2025-11-20 3:37 ` [PATCH net-next v7 5/5] selftests: drv-net: devmem: add autorelease tests Bobby Eshleman
2025-11-26 10:02 ` Simon Horman
2025-12-09 19:52 ` [PATCH net-next v7 0/5] net: devmem: improve cpu cost of RX token management Bobby Eshleman
2025-12-09 19:55 ` Mina Almasry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20251119-scratch-bobbyeshleman-devmem-tcp-token-upstream-v7-4-1abc8467354c@meta.com \
--to=bobbyeshleman@gmail.com \
--cc=almasrymina@google.com \
--cc=andrew+netdev@lunn.ch \
--cc=arnd@arndb.de \
--cc=bobbyeshleman@meta.com \
--cc=corbet@lwn.net \
--cc=davem@davemloft.net \
--cc=donald.hunter@gmail.com \
--cc=dsahern@kernel.org \
--cc=edumazet@google.com \
--cc=horms@kernel.org \
--cc=kuba@kernel.org \
--cc=kuniyu@google.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=ncardwell@google.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=sdf@fomichev.me \
--cc=shuah@kernel.org \
--cc=willemb@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).