Re: [PATCH net-next v6 12/12] net/mlx5e: Add TX support for netmems

[Stanislav Fomichev <stfomichev@gmail.com>]

On 06/16, Mark Bloch wrote:
> From: Dragos Tatulea <dtatulea@nvidia.com>
> 
> Declare netmem TX support in netdev.
> 
> As required, use the netmem aware dma unmapping APIs
> for unmapping netmems in tx completion path.
> 
> Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
> Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
> Reviewed-by: Mina Almasry <almasrymina@google.com>
> Signed-off-by: Mark Bloch <mbloch@nvidia.com>
> ---
>  drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h | 3 ++-
>  drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 ++
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
> index e837c21d3d21..6501252359b0 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h
> @@ -362,7 +362,8 @@ mlx5e_tx_dma_unmap(struct device *pdev, struct mlx5e_sq_dma *dma)
>  		dma_unmap_single(pdev, dma->addr, dma->size, DMA_TO_DEVICE);
>  		break;
>  	case MLX5E_DMA_MAP_PAGE:
> -		dma_unmap_page(pdev, dma->addr, dma->size, DMA_TO_DEVICE);
> +		netmem_dma_unmap_page_attrs(pdev, dma->addr, dma->size,
> +					    DMA_TO_DEVICE, 0);

For this to work, the dma->addr needs to be 0, so the callers of the
dma_map() need to be adjusted as well, or am I missing something?
There is netmem_dma_unmap_addr_set to handle that, but I don't see
anybody calling it. Do we need to add the following (untested)?

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
index 55a8629f0792..fb6465210aed 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
@@ -210,7 +210,9 @@ mlx5e_txwqe_build_dsegs(struct mlx5e_txqsq *sq, struct sk_buff *skb,
 		if (unlikely(dma_mapping_error(sq->pdev, dma_addr)))
 			goto dma_unmap_wqe_err;
 
-		dseg->addr       = cpu_to_be64(dma_addr);
+		dseg->addr = 0;
+		if (!netmem_is_net_iov(skb_frag_netmem(frag)))
+			dseg->addr = cpu_to_be64(dma_addr);
 		dseg->lkey       = sq->mkey_be;
 		dseg->byte_count = cpu_to_be32(fsz);