Re: [PATCH net-next] net/mlx5e: TX, Utilize WQ fragments edge for multi-packet WQEs

[Simon Horman <horms@kernel.org>]

On Wed, Mar 19, 2025 at 03:42:26PM +0200, Tariq Toukan wrote:
> For simplicity reasons, the driver avoids crossing work queue fragment
> boundaries within the same TX WQE (Work-Queue Element). Until today, as
> the number of packets in a TX MPWQE (Multi-Packet WQE) descriptor is not
> known in advance, the driver pre-prepared contiguous memory for the
> largest possible WQE. For this, when getting too close to the fragment
> edge, having no room for the largest WQE possible, the driver was
> filling the fragment remainder with NOP descriptors, aligning the next
> descriptor to the beginning of the next fragment.
> 
> Generating and handling these NOPs wastes resources, like: CPU cycles,
> work-queue entries fetched to the device, and PCI bandwidth.
> 
> In this patch, we replace this NOPs filling mechanism in the TX MPWQE
> flow. Instead, we utilize the remaining entries of the fragment with a
> TX MPWQE. If this room turns out to be too small, we simply open an
> additional descriptor starting at the beginning of the next fragment.
> 
> Performance benchmark:
> uperf test, single server against 3 clients.
> TCP multi-stream, bidir, traffic profile "2x350B read, 1400B write".
> Bottleneck is in inbound PCI bandwidth (device POV).
> 
> +---------------+------------+------------+--------+
> |               | Before     | After      |        |
> +---------------+------------+------------+--------+
> | BW            | 117.4 Gbps | 121.1 Gbps | +3.1%  |
> +---------------+------------+------------+--------+
> | tx_packets    | 15 M/sec   | 15.5 M/sec | +3.3%  |
> +---------------+------------+------------+--------+
> | tx_nops       | 3  M/sec   | 0          | -100%  |
> +---------------+------------+------------+--------+
> 
> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
> Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>

Reviewed-by: Simon Horman <horms@kernel.org>