From: Jakub Kicinski <kuba@kernel.org>
To: Andy Lutomirski <luto@kernel.org>
Cc: "Mina Almasry" <almasrymina@google.com>,
linux-kernel@vger.kernel.org, linux-media@vger.kernel.org,
dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org,
netdev@vger.kernel.org, linux-arch@vger.kernel.org,
linux-kselftest@vger.kernel.org,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Christian König" <christian.koenig@amd.com>,
"David S. Miller" <davem@davemloft.net>,
"Eric Dumazet" <edumazet@google.com>,
"Paolo Abeni" <pabeni@redhat.com>,
"Jesper Dangaard Brouer" <hawk@kernel.org>,
"Ilias Apalodimas" <ilias.apalodimas@linaro.org>,
"Arnd Bergmann" <arnd@arndb.de>,
"David Ahern" <dsahern@kernel.org>,
"Willem de Bruijn" <willemdebruijn.kernel@gmail.com>,
"Shuah Khan" <shuah@kernel.org>,
jgg@ziepe.ca
Subject: Re: [RFC PATCH 00/10] Device Memory TCP
Date: Tue, 18 Jul 2023 10:32:34 -0700 [thread overview]
Message-ID: <20230718103234.711d7e4f@kernel.org> (raw)
In-Reply-To: <12393cd2-4b09-4956-fff0-93ef3929ee37@kernel.org>
On Sun, 16 Jul 2023 19:41:28 -0700 Andy Lutomirski wrote:
> I'm wondering if a more capable if somewhat higher latency model could
> work where the NIC stores received packets in its own device memory.
> Then userspace (or the kernel or a driver or whatever) could initiate a
> separate DMA from the NIC to the final target *after* reading the
> headers. Can the hardware support this?
No, no, that's impossible. SW response times are in 100s of usec (at
best) which at 200Gbps already means megabytes of data _per-queue_.
Way more than the amount of buffer NICs will have.
The Rx application can bind to a IP addr + Port and then install
a one-sided-3-tuple (dst IP+proto+port) rule in the HW. Worst case
a full 5-tuple per flow.
Most NICs support OvS offloads with 100s of thousands of flows.
The steering should be bread and butter.
It does require splitting flows into separate data and control channels,
but it's the right trade-off - complexity should be on the SW side.
next prev parent reply other threads:[~2023-07-18 17:32 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-10 22:32 [RFC PATCH 00/10] Device Memory TCP Mina Almasry
2023-07-10 22:32 ` [RFC PATCH 01/10] dma-buf: add support for paged attachment mappings Mina Almasry
2023-07-11 7:59 ` Christian König
2023-07-11 11:44 ` Mina Almasry
2023-07-11 12:13 ` Christian König
2023-07-10 22:32 ` [RFC PATCH 02/10] dma-buf: add support for NET_RX pages Mina Almasry
2023-07-10 22:32 ` [RFC PATCH 03/10] dma-buf: add support for NET_TX pages Mina Almasry
2023-07-10 22:32 ` [RFC PATCH 04/10] net: add support for skbs with unreadable frags Mina Almasry
2023-07-10 22:32 ` [RFC PATCH 05/10] tcp: implement recvmsg() RX path for devmem TCP Mina Almasry
2023-07-10 22:32 ` [RFC PATCH 06/10] net: add SO_DEVMEM_DONTNEED setsockopt to release RX pages Mina Almasry
2023-07-16 23:57 ` Andy Lutomirski
2023-07-17 2:06 ` Mina Almasry
2023-07-10 22:32 ` [RFC PATCH 07/10] tcp: implement sendmsg() TX path for for devmem tcp Mina Almasry
2023-07-10 22:32 ` [RFC PATCH 08/10] selftests: add ncdevmem, netcat for devmem TCP Mina Almasry
2023-07-10 22:33 ` [RFC PATCH 09/10] memory-provider: updates core provider API " Mina Almasry
2023-07-10 22:33 ` [RFC PATCH 10/10] memory-provider: add dmabuf devmem provider Mina Almasry
2023-07-17 2:41 ` [RFC PATCH 00/10] Device Memory TCP Andy Lutomirski
2023-07-18 17:32 ` Jakub Kicinski [this message]
2023-07-18 17:36 ` Mina Almasry
2023-07-18 18:06 ` Jason Gunthorpe
2023-07-18 18:15 ` Jakub Kicinski
2023-07-18 18:20 ` David Ahern
2023-07-18 18:29 ` Jakub Kicinski
2023-07-18 22:35 ` David Ahern
2023-07-18 22:45 ` Jakub Kicinski
2023-07-19 15:10 ` Mina Almasry
2023-07-19 17:57 ` Stephen Hemminger
2023-07-19 23:24 ` Jason Gunthorpe
2023-07-27 11:40 ` [Linaro-mm-sig] " Christian König
2023-07-19 20:36 ` Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230718103234.711d7e4f@kernel.org \
--to=kuba@kernel.org \
--cc=almasrymina@google.com \
--cc=arnd@arndb.de \
--cc=christian.koenig@amd.com \
--cc=davem@davemloft.net \
--cc=dri-devel@lists.freedesktop.org \
--cc=dsahern@kernel.org \
--cc=edumazet@google.com \
--cc=hawk@kernel.org \
--cc=ilias.apalodimas@linaro.org \
--cc=jgg@ziepe.ca \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=luto@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=shuah@kernel.org \
--cc=sumit.semwal@linaro.org \
--cc=willemdebruijn.kernel@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).