From: Cosmin Ratiu <cratiu@nvidia.com>
To: Dragos Tatulea <dtatulea@nvidia.com>,
"stfomichev@gmail.com" <stfomichev@gmail.com>,
"almasrymina@google.com" <almasrymina@google.com>
Cc: "andrew+netdev@lunn.ch" <andrew+netdev@lunn.ch>,
"hawk@kernel.org" <hawk@kernel.org>,
"leon@kernel.org" <leon@kernel.org>,
"davem@davemloft.net" <davem@davemloft.net>,
"john.fastabend@gmail.com" <john.fastabend@gmail.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"edumazet@google.com" <edumazet@google.com>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"richardcochran@gmail.com" <richardcochran@gmail.com>,
"pabeni@redhat.com" <pabeni@redhat.com>,
"ast@kernel.org" <ast@kernel.org>,
"kuba@kernel.org" <kuba@kernel.org>,
"daniel@iogearbox.net" <daniel@iogearbox.net>,
Tariq Toukan <tariqt@nvidia.com>,
Saeed Mahameed <saeedm@nvidia.com>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
Mark Bloch <mbloch@nvidia.com>,
"bpf@vger.kernel.org" <bpf@vger.kernel.org>,
Moshe Shemesh <moshe@nvidia.com>, Gal Pressman <gal@nvidia.com>
Subject: Re: [PATCH net-next V2 00/11] net/mlx5e: Add support for devmem and io_uring TCP zero-copy
Date: Fri, 6 Jun 2025 09:00:23 +0000 [thread overview]
Message-ID: <b51a92bb8f4b46e9370b93d68a0a994e9c7289d1.camel@nvidia.com> (raw)
In-Reply-To: <phuigc2himixvyaxydukgupqy2oxpobj6qo4m4hb6vsr5qenfd@7q4ct2c5gjdq>
On Thu, 2025-05-29 at 11:11 +0000, Dragos Tatulea wrote:
>
> AFAIU from Cosmin the condition happened on initial queue fill when
> there are no buffers to be released for the current WQE.
The issue happens when there isn't enough memory in the pool to
completely fill the rx ring with descriptors, and then rx eventually
fully stops once the posted descriptors get exhausted, because the ring
refill logic will actually only release ring memory back to the pool
from ring_tail, when ring_head == ring_tail (for cache efficiency).
This means if the ring cannot be completely filled, memory never gets
released because ring_head != ring_tail.
The easy workaround is to have a pool with enough memory to let the rx
ring completely fill up. I suspect in real life this is easily the
case, but in the contrived ncdevmem test with udmabuf memory of 128 MB
and artificially high ring size & MTU this corner case was hit.
As Dragos said, we will look into this after the code has been posted.
Cosmin.
next prev parent reply other threads:[~2025-06-06 9:00 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-05-22 21:41 [PATCH net-next V2 00/11] net/mlx5e: Add support for devmem and io_uring TCP zero-copy Tariq Toukan
2025-05-22 21:41 ` [PATCH net-next V2 01/11] net: Kconfig NET_DEVMEM selects GENERIC_ALLOCATOR Tariq Toukan
2025-05-22 23:07 ` Mina Almasry
2025-05-22 21:41 ` [PATCH net-next V2 02/11] net: Add skb_can_coalesce for netmem Tariq Toukan
2025-05-22 23:09 ` Mina Almasry
2025-05-25 13:03 ` Dragos Tatulea
2025-05-25 17:44 ` Mina Almasry
2025-05-28 9:13 ` Dragos Tatulea
2025-05-22 21:41 ` [PATCH net-next V2 03/11] net/mlx5e: SHAMPO: Reorganize mlx5_rq_shampo_alloc Tariq Toukan
2025-05-22 21:41 ` [PATCH net-next V2 04/11] net/mlx5e: SHAMPO: Remove redundant params Tariq Toukan
2025-05-22 21:41 ` [PATCH net-next V2 05/11] net/mlx5e: SHAMPO: Improve hw gro capability checking Tariq Toukan
2025-05-22 21:41 ` [PATCH net-next V2 06/11] net/mlx5e: SHAMPO: Separate pool for headers Tariq Toukan
2025-05-22 22:30 ` Jakub Kicinski
2025-05-22 23:08 ` Saeed Mahameed
2025-05-22 23:24 ` Mina Almasry
2025-05-22 23:43 ` Saeed Mahameed
2025-05-27 15:29 ` Jakub Kicinski
2025-05-27 15:53 ` Dragos Tatulea
2025-05-22 21:41 ` [PATCH net-next V2 07/11] net/mlx5e: SHAMPO: Headers page pool stats Tariq Toukan
2025-05-22 22:31 ` Jakub Kicinski
2025-05-22 22:58 ` Saeed Mahameed
2025-06-06 10:43 ` Cosmin Ratiu
2025-06-08 10:09 ` Tariq Toukan
2025-06-09 15:20 ` Jakub Kicinski
2025-05-22 21:41 ` [PATCH net-next V2 08/11] net/mlx5e: Convert over to netmem Tariq Toukan
2025-05-22 23:18 ` Mina Almasry
2025-05-22 23:54 ` Saeed Mahameed
2025-05-23 17:58 ` Mina Almasry
2025-05-23 19:22 ` Saeed Mahameed
2025-05-22 21:41 ` [PATCH net-next V2 09/11] net/mlx5e: Add support for UNREADABLE netmem page pools Tariq Toukan
2025-05-22 23:26 ` Mina Almasry
2025-05-22 23:56 ` Saeed Mahameed
2025-05-22 21:41 ` [PATCH net-next V2 10/11] net/mlx5e: Implement queue mgmt ops and single channel swap Tariq Toukan
2025-05-22 21:41 ` [PATCH net-next V2 11/11] net/mlx5e: Support ethtool tcp-data-split settings Tariq Toukan
2025-05-22 22:55 ` Jakub Kicinski
2025-05-22 23:19 ` Saeed Mahameed
2025-05-23 16:17 ` Cosmin Ratiu
2025-05-23 19:35 ` saeed
2025-05-27 16:10 ` Jakub Kicinski
2025-05-28 5:10 ` Gal Pressman
2025-05-29 0:12 ` Jakub Kicinski
2025-05-27 16:05 ` [PATCH net-next V2 00/11] net/mlx5e: Add support for devmem and io_uring TCP zero-copy Stanislav Fomichev
2025-05-28 9:17 ` Dragos Tatulea
2025-05-28 15:45 ` Stanislav Fomichev
2025-05-28 22:59 ` Mina Almasry
2025-05-28 23:04 ` Stanislav Fomichev
2025-05-29 11:11 ` Dragos Tatulea
2025-06-06 9:00 ` Cosmin Ratiu [this message]
2025-05-28 0:31 ` Jakub Kicinski
2025-05-28 1:20 ` patchwork-bot+netdevbpf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b51a92bb8f4b46e9370b93d68a0a994e9c7289d1.camel@nvidia.com \
--to=cratiu@nvidia.com \
--cc=almasrymina@google.com \
--cc=andrew+netdev@lunn.ch \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dtatulea@nvidia.com \
--cc=edumazet@google.com \
--cc=gal@nvidia.com \
--cc=hawk@kernel.org \
--cc=john.fastabend@gmail.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=moshe@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=richardcochran@gmail.com \
--cc=saeedm@nvidia.com \
--cc=stfomichev@gmail.com \
--cc=tariqt@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).