Re: mlx5e:zero-prefix corruption during TCP DMA-BUF RX - help requested

[Dragos Tatulea <dtatulea@nvidia.com>]

Hi Jiabin,

On Thu, Oct 08, 2026 at 02:48:53PM +0800, jiabin deng wrote:
> Hi Dragos,
> 
> Thank you for following up and suggesting the CPU-buffer comparison. We have
> now completed both ordinary CPU TCP and CPU-backed DMA-BUF receive controls,
> with no payload corruption in either 400-iteration run.
> 
> Please find attached a standalone GPU-memory reproducer with a README covering
> the required environment, build instructions, two-host execution, and result
> collection. It includes a Python TCP sender, a C++ receiver using the CUDA
> Driver API and Linux device-memory TCP interfaces, and scripts for SHAMPO
> header tracing and offline byte comparisons.
>
Thank you for the reproducer. Unfortunately I didn't get the right HW to
reproduce this.

Pasting from your previous email:
> One possible lead is an interaction between SHAMPO end eadding and
> Relaxed Ordering writes. In this version, rq_end_pad_mode() disables
> end padding only when LRO is enabled and write RO is supported.
> SHAMPO therefore selects ALIGN; at the same time, write RO is enabled
> on the payload MKey when the device advertises write RO support.
> 
> Commit e2351e517068 ("net/mlx5e: RX, Avoid possible data corruption
> when relaxed ordering and LRO combined") describes a related issue
> with consecutively placed LRO payloads. We would like to understand
> whether SHAMPO might also need to account for a similar situation.
> We do not yet have results from padding/RO diagnostic experiments.
Initially I dismissed this commit because it seemed unrelated. But now
I see that you analysis was probably spot on.

So disabling alignment padding mode is definetly a very useful experiment.
It would go in the lines of:

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
index 70a855c6d19c..794d8be88f0c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
@@ -911,10 +911,11 @@ static void mlx5e_build_rx_cq_param(struct mlx5_core_dev *mdev,
 
 static u8 rq_end_pad_mode(struct mlx5_core_dev *mdev, struct mlx5e_params *params)
 {
-       bool lro_en = params->packet_merge.type == MLX5E_PACKET_MERGE_LRO;
+       bool no_pad = params->packet_merge.type == MLX5E_PACKET_MERGE_LRO ||
+                     params->packet_merge.type == MLX5E_PACKET_MERGE_SHAMPO;
        bool ro = MLX5_CAP_GEN(mdev, relaxed_ordering_write);
 
-       return ro && lro_en ?
+       return ro && no_pad ?
                MLX5_WQ_END_PAD_MODE_NONE : MLX5_WQ_END_PAD_MODE_ALIGN;
 }

Could you try this please and let us know if you still encounter the
issue? In the meantime I am checking about this with other people.

Thanks,
Dragos