Re: [PATCH net-next 06/10] net/mlx5e: Add data direct TX infrastructure

[sashiko-bot@kernel.org]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] net/mlx5e: GPU unbind causes DMA corruption for active devmem TX
--

commit 2beff8c1365001300351d8ed26d31fd617a37400
Author: Dragos Tatulea <dtatulea@nvidia.com>

net/mlx5e: Add data direct TX infrastructure

This commit introduces the necessary infrastructure for peer-to-peer DMA
between the NIC and another device (e.g., GPU) on the TX path. It creates
a crossing mkey linked to the SQ and adds a new SQ state flag to indicate
when Data Direct is enabled.

> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> index 14479da98f427..62ffb41a64dfe 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> @@ -182,6 +182,7 @@ mlx5e_txwqe_build_dsegs(struct mlx5e_txqsq *sq, struct sk_buff *skb,
>  			unsigned char *skb_data, u16 headlen,
>  			struct mlx5_wqe_data_seg *dseg)
>  {
> +	bool dd = test_bit(MLX5E_SQ_STATE_DATA_DIRECT, &sq->state);
>  	dma_addr_t dma_addr = 0;
>  	u8 num_dma          = 0;
>  	int i;

[ ... ]

> @@ -211,7 +212,8 @@ mlx5e_txwqe_build_dsegs(struct mlx5e_txqsq *sq, struct sk_buff *skb,
>  			goto dma_unmap_wqe_err;
>  
>  		dseg->addr       = cpu_to_be64(dma_addr);
> -		dseg->lkey       = sq->mkey_be;
> +		dseg->lkey       = dd && skb_frag_is_net_iov(frag) ?
> +				   sq->crossing_mkey_be : sq->mkey_be;

[Severity: High]
If a peer GPU device unbinds while a Data Direct devmem DMABUF is mapped
for TX, the MLX5E_SQ_STATE_DATA_DIRECT flag is forcefully cleared.

Since the asynchronous unbind bypasses the devmem socket binding safety
checks, the application can continue transmitting net_iov fragments that
still contain GPU IOVAs.

When those fragments arrive here, dd is now false, so the code falls
back to using the NIC's local sq->mkey_be for those GPU IOVAs.

Could this cause the NIC to read incorrect local memory or trigger host
IOMMU protection faults? How should active net_iov transmissions be explicitly
fenced or dropped when the queue state reverts?

>  		dseg->byte_count = cpu_to_be32(fsz);
>  
>  		mlx5e_dma_push_netmem(sq, skb_frag_netmem(frag), dma_addr, fsz);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261008132815.654147-1-tariqt@nvidia.com?part=6