Re: [PATCH net-next 08/10] net/mlx5e: Recreate netdev channels on data direct device unbind

[Dragos Tatulea <dtatulea@nvidia.com>]

On 10.10.26 01:56, Mina Almasry wrote:
> On Thu, Oct 8, 2026 at 6:28 AM Tariq Toukan <tariqt@nvidia.com> wrote:
>> From: Dragos Tatulea <dtatulea@nvidia.com>
>>
>> Note that this does not tear down dmabuf bindings attached to the data
>> direct device; the recreated channels will still pick up the binding via
>> rxq->mp_params and post DMA addresses from the data direct IOMMU domain
>> to the PF, causing IOMMU faults. A devmem revoke hook is needed to close
>> those bindings before the switch.
> 
> (Note: LLM-assisted review comment below.)
> 
> Rather than shipping this known IOMMU fault window on MLX5_DATA_DIRECT_UNBIND,
> could we add the core detach helper in patch 09/10 and invoke it here under
> netdev_lock(priv->netdev) before mlx5e_safe_switch_params()?
> 
> Specifically, reopening channels on the PF without detaching active bindings
> leaves three issues:
> 1. RX: reopened queues pick up stale rxq->mp_params and post data_direct IOVAs
>    onto ch->pdev.
> 2. TX: active TX bindings (net_devmem_dmabuf_bindings) and in-flight or
>    retransmitted skbs still hold data_direct IOVAs while mlx5e_xmit() switches
>    to sq->mkey_be (ch->pdev).
> 3. DMA unmap ordering: binding->sgt / binding->attachment stay mapped to
>    &dd_dev->pdev->dev until userspace closes all sockets, after
>    mlx5_data_direct_remove() has already returned.
> 
> If mlx5e_data_direct_event() calls a core helper such as
> netdev_unbind_dmabuf_dma_dev(priv->netdev, &dd_dev->pdev->dev) before
> mlx5e_safe_switch_params():
> - netif_mp_close_rxq() clears rxq->mp_params so reopened channels use host
>   page_pool pages on ch->pdev.
> - Clearing WRITE_ONCE(binding->dev, NULL) + synchronize_net() causes existing
>   validate_xmit_unreadable_skb() (READ_ONCE(binding->dev) != dev) to drop any
>   in-flight/retransmitted TX skbs automatically with zero driver fast-path
>   changes.
> - dma_buf_unmap_attachment_unlocked() + dma_buf_detach() complete synchronously
>   before mlx5_data_direct_remove() finishes.
> 
My plan was to address this after this series is accepted.

Thanks,
Dragos