Re: [PATCH net-next 10/10] net/mlx5e: Use the 'num_doorbells' devlink param

[Stanislav Fomichev <stfomichev@gmail.com>]

On 09/10, Tariq Toukan wrote:
> From: Cosmin Ratiu <cratiu@nvidia.com>
> 
> Use the new devlink param to control how many doorbells mlx5e devices
> allocate and use. The maximum number of doorbells configurable is capped
> to the maximum number of channels. This only applies to the Ethernet
> part, the RDMA devices using mlx5 manage their own doorbells.
> 
> Signed-off-by: Cosmin Ratiu <cratiu@nvidia.com>
> Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
> ---
>  Documentation/networking/devlink/mlx5.rst     |  8 ++++++
>  .../net/ethernet/mellanox/mlx5/core/devlink.c | 26 +++++++++++++++++++
>  .../ethernet/mellanox/mlx5/core/en_common.c   | 15 ++++++++++-
>  3 files changed, 48 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/networking/devlink/mlx5.rst b/Documentation/networking/devlink/mlx5.rst
> index 60cc9fedf1ef..0650462b3eae 100644
> --- a/Documentation/networking/devlink/mlx5.rst
> +++ b/Documentation/networking/devlink/mlx5.rst
> @@ -45,6 +45,14 @@ Parameters
>       - The range is between 1 and a device-specific max.
>       - Applies to each physical function (PF) independently, if the device
>         supports it. Otherwise, it applies symmetrically to all PFs.
> +   * - ``num_doorbells``
> +     - driverinit
> +     - This controls the number of channel doorbells used by the netdev. In all
> +       cases, an additional doorbell is allocated and used for non-channel
> +       communication (e.g. for PTP, HWS, etc.). Supported values are:
> +       - 0: No channel-specific doorbells, use the global one for everything.
> +       - [1, max_num_channels]: Spread netdev channels equally across these
> +         doorbells.

Do you have any guidance on this number? Why would the user want
`num_doorbells < num_doorbells` vs `num_doorbells == num_channels`?

IOW, why not allocate the same number of doorbells as the number of
channels and do it unconditionally without devlink param? Are extra
doorbells causing any overhead in the non-contended case?