From: Stanislav Fomichev <sdf.kernel@gmail.com>
To: Dragos Tatulea <dtatulea@nvidia.com>
Cc: Mina Almasry <almasrymina@google.com>,
Tariq Toukan <tariqt@nvidia.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>,
netdev@vger.kernel.org, Paolo Abeni <pabeni@redhat.com>,
Bobby Eshleman <bobbyeshleman@meta.com>,
Carolina Jubran <cjubran@nvidia.com>,
Cosmin Ratiu <cratiu@nvidia.com>,
Daniel Borkmann <daniel@iogearbox.net>,
David Wei <dw@davidwei.uk>, Erfan Sharafzadeh <erfansh@meta.com>,
Gal Pressman <gal@nvidia.com>,
Jacob Keller <Jacob.e.keller@intel.com>,
Kaifeng Wang <kaifengw@google.com>,
Leon Romanovsky <leon@kernel.org>,
open list <linux-kernel@vger.kernel.org>,
linux-rdma@vger.kernel.org, Mark Bloch <mbloch@nvidia.com>,
Matt Fleming <mfleming@cloudflare.com>,
Nikolay Aleksandrov <razor@blackwall.org>,
Saeed Mahameed <saeedm@nvidia.com>,
Simon Horman <horms@kernel.org>,
Stanislav Fomichev <sdf@fomichev.me>,
William Tu <witu@nvidia.com>
Subject: Re: [PATCH net-next V2 00/13] net/mlx5e: Add netdev support for data direct
Date: Fri, 20 Nov 2026 09:14:49 -0800 [thread overview]
Message-ID: <awB_rJxrIDQ3u-73@devvm16445.scu0.facebook.com> (raw)
In-Reply-To: <0a1be802-1c53-46ff-a08c-614c3a2d6f41@nvidia.com>
On 11/20, Dragos Tatulea wrote:
> On 20.11.26 17:38, Mina Almasry wrote:
> > On Fri, Nov 20, 2026 at 6:39 AM Dragos Tatulea <dtatulea@nvidia.com> wrote:
> >>
> >> Hi all,
> >>
> >> On 10.11.26 19:38, Tariq Toukan wrote:
> >> [...]
> >>>
> >>> .../ethernet/mellanox/mlx5/core/data_direct.c | 9 +-
> >>> drivers/net/ethernet/mellanox/mlx5/core/en.h | 14 +
> >>> .../ethernet/mellanox/mlx5/core/en/params.c | 18 +-
> >>> .../ethernet/mellanox/mlx5/core/en/params.h | 2 +
> >>> .../mellanox/mlx5/core/en/reporter_rx.c | 1 +
> >>> .../mellanox/mlx5/core/en/reporter_tx.c | 1 +
> >>> .../ethernet/mellanox/mlx5/core/en_common.c | 38 ++-
> >>> .../ethernet/mellanox/mlx5/core/en_ethtool.c | 67 +++++
> >>> .../net/ethernet/mellanox/mlx5/core/en_main.c | 253 +++++++++++++++++-
> >>> .../net/ethernet/mellanox/mlx5/core/en_rx.c | 33 +--
> >>> .../net/ethernet/mellanox/mlx5/core/en_tx.c | 4 +-
> >>> .../net/ethernet/mellanox/mlx5/core/main.c | 1 +
> >>> include/linux/mlx5/driver.h | 1 +
> >>> include/net/netdev_queues.h | 1 +
> >>> include/net/netmem.h | 24 ++
> >>> net/core/devmem.c | 85 ++++++
> >>> net/core/devmem.h | 6 +
> >>> net/core/netdev_queues.c | 51 ++++
> >> While working on V3 I realized that keeping the core part in
> >> this series is not the best approach. Initially it made sense
> >> because the change was small.
> >>
> >> I think I should re-spin it in a separate series with netdevsim
> >> based selftests for the new APIs. I would base it on Mina's devmem
> >> leased queues unregister [1].
> >>
> >> Any reason not to?
> >>
> >
> > I honestly wanted to comment that you should base it on the leased
> > queues series from me but didn't want to be annoying, especially
> > because I made you wait for the previous series.
> >
> I realized that quite a few changes were needed after your series
> so I decided to base it on it.
>
> > My series creates binding->bind_dev, and binding->map_dev, and core
> > will (should) handle things correctly across both devices. This seems
> > perfect for you, because you have a bind device and a separate map
> > device and their lifetime is different.
> >
> In my series I am actually skipping leased rx queues in
> net_devmem_unbind_dma_dev() because we need some kind of notification
> mechanism + netkit implementation. I think this should be added later
> if there is a usecase. For now mlx5 simply doesn't support the combo.
>
> One note on net_devmem_unbind_dmabuf(). Posting it here as I don't
> find the right place to mention in your series:
>
> @@ -234,8 +234,17 @@ void net_devmem_unbind_dmabuf(struct net_devmem_dmabuf_binding *binding)
> busy = true;
> }
>
> - if (binding->direction == DMA_FROM_DEVICE && !busy)
> + if (binding->direction == DMA_FROM_DEVICE && !busy) {
> + struct net_device *map_dev = NULL;
> + netdevice_tracker dev_tracker;
> +
> + /* Leased queue: attachment readers hold map_dev's lock. */
> + if (READ_ONCE(binding->map_dev) != binding->bind_dev)
> + map_dev = net_devmem_lock_dev(&binding->map_dev,
> + &dev_tracker);
> net_devmem_dmabuf_binding_unmap(binding);
> + net_devmem_unlock_dev(map_dev, &dev_tracker);
> + }
> WRITE_ONCE(binding->bind_dev, NULL);
> }
>
> map_dev devmem lock is taken to prevent use after free in
> netdev_has_dmabuf_binding() when called in parallel with
> net_devmem_unbind_dmabuf(). As this can happen in my series I added it
> there.
>
> Waiting for feedback from Stan/Jakub/Paolo for a few more hours and
> will post the series tomorrow.
More and more workloads are moving into ns/netkit setup on our side,
so it would be nice to have netkit support as well. We can start with
what you have so you're not blocked for too long, but we do want the
netkit portion as well :-)
prev parent reply other threads:[~2026-11-20 17:14 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-10 18:38 [PATCH net-next V2 00/13] net/mlx5e: Add netdev support for data direct Tariq Toukan
2026-11-10 18:38 ` [PATCH net-next V2 01/13] net/mlx5: Log the data direct to PF device mapping Tariq Toukan
2026-11-10 18:38 ` [PATCH net-next V2 02/13] net/mlx5: Detach auxiliary devices on attach failure during load Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 13:58 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 03/13] net/mlx5e: Register supported netdevs as data direct users Tariq Toukan
2026-11-10 18:38 ` [PATCH net-next V2 04/13] net/mlx5e: Pre-calculate UMR padding and entry size Tariq Toukan
2026-11-10 18:38 ` [PATCH net-next V2 05/13] net/mlx5e: Add data direct ethtool private flag Tariq Toukan
2026-11-10 18:38 ` [PATCH net-next V2 06/13] net/mlx5e: Add data direct RX infrastructure Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 14:28 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 07/13] net/mlx5e: SHAMPO, Use page_pool DMA sync helper for SKB frags Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 14:36 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 08/13] net/mlx5e: Add data direct TX infrastructure Tariq Toukan
2026-11-10 18:38 ` [PATCH net-next V2 09/13] net/mlx5e: Use the correct DMA dev when data_direct is enabled Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 14:43 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 10/13] net: devmem: add netdev_has_dmabuf_binding() helper Tariq Toukan
2026-11-11 21:46 ` Stanislav Fomichev
2026-11-13 15:00 ` Dragos Tatulea
2026-11-15 7:59 ` Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 15:02 ` Dragos Tatulea
2026-11-15 18:17 ` Jakub Kicinski
2026-11-17 11:20 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 11/13] net: add netdev_unbind_dma_dev() helper Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-14 9:54 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 12/13] net/mlx5e: Disable data direct on data direct device unbind Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-14 9:54 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 13/13] net/mlx5e: Enable the data direct netdev feature Tariq Toukan
2026-11-20 14:39 ` [PATCH net-next V2 00/13] net/mlx5e: Add netdev support for data direct Dragos Tatulea
2026-11-20 16:38 ` Mina Almasry
2026-11-20 17:03 ` Dragos Tatulea
2026-11-20 17:14 ` Stanislav Fomichev [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=awB_rJxrIDQ3u-73@devvm16445.scu0.facebook.com \
--to=sdf.kernel@gmail.com \
--cc=Jacob.e.keller@intel.com \
--cc=almasrymina@google.com \
--cc=andrew+netdev@lunn.ch \
--cc=bobbyeshleman@meta.com \
--cc=cjubran@nvidia.com \
--cc=cratiu@nvidia.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dtatulea@nvidia.com \
--cc=dw@davidwei.uk \
--cc=edumazet@kernel.org \
--cc=erfansh@meta.com \
--cc=gal@nvidia.com \
--cc=horms@kernel.org \
--cc=kaifengw@google.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=mfleming@cloudflare.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=razor@blackwall.org \
--cc=saeedm@nvidia.com \
--cc=sdf@fomichev.me \
--cc=tariqt@nvidia.com \
--cc=witu@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox