From: Dragos Tatulea <dtatulea@nvidia.com>
To: netdev-bot+sashiko@kernel.org, tariqt@nvidia.com
Cc: andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@kernel.org,
kuba@kernel.org, netdev@vger.kernel.org, pabeni@redhat.com,
bobbyeshleman@meta.com, cjubran@nvidia.com, cratiu@nvidia.com,
daniel@iogearbox.net, dw@davidwei.uk, erfansh@meta.com,
gal@nvidia.com, Jacob.e.keller@intel.com, kaifengw@google.com,
leon@kernel.org, linux-kernel@vger.kernel.org,
linux-rdma@vger.kernel.org, mbloch@nvidia.com,
mfleming@cloudflare.com, almasrymina@google.com,
razor@blackwall.org, saeedm@nvidia.com, horms@kernel.org,
sdf@fomichev.me, sdf.kernel@gmail.com, witu@nvidia.com
Subject: Re: [PATCH net-next V2 09/13] net/mlx5e: Use the correct DMA dev when data_direct is enabled
Date: Fri, 13 Nov 2026 16:43:48 +0200 [thread overview]
Message-ID: <251bb36e-3cb3-419f-b47f-30d464bd4239@nvidia.com> (raw)
In-Reply-To: <179447641026.2199510.5931919267235772242@kernel.org>
On 12.11.26 10:40, netdev-bot+sashiko@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found more severe
> issue(s) in other patches of this series. This mail is only an FYI, in case
> you want to address the nit picks below when you respin.
>
> Low priority / nit picks for your consideration:
>
> - [Low] This patch makes the data-direct PCI device (dd_dev->device) the
> DMA device for the page_pool of data-direct RQs (mlx5e_alloc_rq,…
> - [Low] mlx5e_get_dma_dev() (en_main.c:965-976) reads
> mdev->data_direct->dev with READ_ONCE() through…
>
> --
>
> net/mlx5e: Use the correct DMA dev when data_direct is enabled
>
> This commit adds mlx5e_get_dma_dev(). When data_direct is enabled it
> returns the data direct device instead of the PF PCI device. The helper
> is used for the RQ page_pool DMA device and for .ndo_queue_get_dma_dev,
> and the latter now refuses to return a device in data-direct mode when
> RX queues are leased.
>
>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> index 5b99099e4ed0c..ebd41f24330d0 100644
>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> @@ -962,6 +962,19 @@ static void mlx5e_rq_free_shampo(struct mlx5e_rq *rq)
>> kvfree(shampo);
>> }
>>
>> +static struct device *mlx5e_get_dma_dev(struct mlx5_core_dev *mdev,
>> + bool data_direct)
>> +{
>> + struct mlx5_data_direct_dev *dd_dev;
>> +
>> + if (!data_direct)
>> + return mlx5_core_dma_dev(mdev);
>> +
>> + dd_dev = mlx5_data_direct_get_dev(mdev->data_direct);
>> +
>> + return dd_dev ? dd_dev->device : NULL;
>
> [Severity: Low]
> Is anything keeping dd_dev alive between the READ_ONCE() in
> mlx5_data_direct_get_dev() and the dd_dev->device dereference here?
>
> The unbind side clears the pointer, runs the notifier chain, and then
> frees the device:
>
> mlx5_data_direct_remove()
> mlx5_data_direct_dev_unreg(dev)
> ...
> mlx5_data_direct_do_unbind()
> WRITE_ONCE(reg->dd.dev, NULL);
> blocking_notifier_call_chain(®->users,
> MLX5_DATA_DIRECT_UNBIND, NULL);
> pci_disable_device(pdev);
> kfree(dev->vuid);
> kfree(dev);
>
> At this commit, mlx5e_data_direct_event() only logs on
> MLX5_DATA_DIRECT_UNBIND. It takes none of the locks the readers hold,
> which is priv->state_lock in mlx5e_queue_get_dma_dev() and in the channel
> open path that reaches mlx5e_alloc_rq().
>
> If a reader has already loaded a non-NULL dd_dev, can it dereference it
> after kfree(dev)? The returned struct device pointer is also used
> without taking a reference.
>
> This does not look reachable at this commit. set_pflag_data_direct()
> still returns -EOPNOTSUPP, so data_direct is never true here.
>
> Later in the series, "net/mlx5e: Disable data direct on data direct
> device unbind" makes the notifier take rtnl_lock(), netdev_lock() and
> priv->state_lock before it returns. That serializes it against these
> readers before kfree(dev) runs.
>
You said it yourself. So nothing to change in this patch.
>> +}
>> +
>> static int mlx5e_alloc_rq(struct mlx5e_params *params,
>> struct mlx5e_rq_param *rq_param,
>> struct mlx5e_rq_opt_param *rqo,
>> @@ -1105,7 +1118,11 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params,
>> pp_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV;
>> pp_params.pool_size = pool_size;
>> pp_params.nid = node;
>> - pp_params.dev = rq->pdev;
>> + pp_params.dev = mlx5e_get_dma_dev(mdev, data_direct);
>
> [Severity: Low]
> With this change, the page_pool of a data direct RQ is created against
> dd_dev->device. mlx5e_queue_get_dma_dev() also hands that device to
> memory providers (devmem, io_uring zcrx), which map their buffers
> against it.
>
> What tears these down when the data direct device is unbound? At this
> commit, this is the only mlx5e consumer of MLX5_DATA_DIRECT_UNBIND:
>
> mlx5e_data_direct_event() {
> ...
> if (action == MLX5_DATA_DIRECT_UNBIND)
> mlx5_core_dbg(priv->mdev, "data direct device unbound\n");
>
> return NOTIFY_OK;
> }
>
> It doesn't close the data direct RQs, destroy their page pools, or
> unbind the memory provider mappings. mlx5_data_direct_remove() then
> runs pci_disable_device() and kfree(dev) right after the notifier
> returns.
>
> Could the page_pool and dmabuf mappings outlive the device they were
> mapped against?
>
> This also doesn't appear reachable at this commit, since
> set_pflag_data_direct() in en_ethtool.c still returns -EOPNOTSUPP.
>
> Later in the series, "net/mlx5e: Disable data direct on data direct
> device unbind" rewrites mlx5e_data_direct_event(). The new version calls
> netif_close() and netdev_unbind_dma_dev(), then clears
> MLX5E_PFLAG_DATA_DIRECT. The flag only becomes settable later, in
> "net/mlx5e: Enable the data direct netdev feature".
>
Same.
Thanks,
Dragos
next prev parent reply other threads:[~2026-11-13 14:44 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-10 18:38 [PATCH net-next V2 00/13] net/mlx5e: Add netdev support for data direct Tariq Toukan
2026-11-10 18:38 ` [PATCH net-next V2 01/13] net/mlx5: Log the data direct to PF device mapping Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-10 18:38 ` [PATCH net-next V2 02/13] net/mlx5: Detach auxiliary devices on attach failure during load Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 13:58 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 03/13] net/mlx5e: Register supported netdevs as data direct users Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-10 18:38 ` [PATCH net-next V2 04/13] net/mlx5e: Pre-calculate UMR padding and entry size Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-10 18:38 ` [PATCH net-next V2 05/13] net/mlx5e: Add data direct ethtool private flag Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-10 18:38 ` [PATCH net-next V2 06/13] net/mlx5e: Add data direct RX infrastructure Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 14:28 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 07/13] net/mlx5e: SHAMPO, Use page_pool DMA sync helper for SKB frags Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 14:36 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 08/13] net/mlx5e: Add data direct TX infrastructure Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-10 18:38 ` [PATCH net-next V2 09/13] net/mlx5e: Use the correct DMA dev when data_direct is enabled Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 14:43 ` Dragos Tatulea [this message]
2026-11-10 18:38 ` [PATCH net-next V2 10/13] net: devmem: add netdev_has_dmabuf_binding() helper Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-11 21:46 ` Stanislav Fomichev
2026-11-13 15:00 ` Dragos Tatulea
2026-11-15 7:59 ` Tariq Toukan
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-13 15:02 ` Dragos Tatulea
2026-11-15 18:17 ` Jakub Kicinski
2026-11-17 11:20 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 11/13] net: add netdev_unbind_dma_dev() helper Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-14 9:54 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 12/13] net/mlx5e: Disable data direct on data direct device unbind Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
2026-11-12 9:40 ` netdev-bot+sashiko
2026-11-14 9:54 ` Dragos Tatulea
2026-11-10 18:38 ` [PATCH net-next V2 13/13] net/mlx5e: Enable the data direct netdev feature Tariq Toukan
2026-11-11 18:40 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=251bb36e-3cb3-419f-b47f-30d464bd4239@nvidia.com \
--to=dtatulea@nvidia.com \
--cc=Jacob.e.keller@intel.com \
--cc=almasrymina@google.com \
--cc=andrew+netdev@lunn.ch \
--cc=bobbyeshleman@meta.com \
--cc=cjubran@nvidia.com \
--cc=cratiu@nvidia.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dw@davidwei.uk \
--cc=edumazet@kernel.org \
--cc=erfansh@meta.com \
--cc=gal@nvidia.com \
--cc=horms@kernel.org \
--cc=kaifengw@google.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=mfleming@cloudflare.com \
--cc=netdev-bot+sashiko@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=razor@blackwall.org \
--cc=saeedm@nvidia.com \
--cc=sdf.kernel@gmail.com \
--cc=sdf@fomichev.me \
--cc=tariqt@nvidia.com \
--cc=witu@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.