From: Jason Gunthorpe <jgg@ziepe.ca>
To: Stanislav Fomichev <stfomichev@gmail.com>
Cc: Tariq Toukan <tariqt@nvidia.com>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Jiri Pirko <jiri@resnulli.us>, Jonathan Corbet <corbet@lwn.net>,
Leon Romanovsky <leon@kernel.org>,
Saeed Mahameed <saeedm@nvidia.com>,
Mark Bloch <mbloch@nvidia.com>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Jesper Dangaard Brouer <hawk@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
netdev@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
bpf@vger.kernel.org, Gal Pressman <gal@nvidia.com>,
Cosmin Ratiu <cratiu@nvidia.com>,
Dragos Tatulea <dtatulea@nvidia.com>,
Jiri Pirko <jiri@nvidia.com>
Subject: Re: [PATCH net-next 10/10] net/mlx5e: Use the 'num_doorbells' devlink param
Date: Wed, 10 Sep 2025 13:23:10 -0300 [thread overview]
Message-ID: <20250910162310.GF882933@ziepe.ca> (raw)
In-Reply-To: <aMGkaDoZpmOWUA_L@mini-arch>
On Wed, Sep 10, 2025 at 09:16:40AM -0700, Stanislav Fomichev wrote:
> > + * - ``num_doorbells``
> > + - driverinit
> > + - This controls the number of channel doorbells used by the netdev. In all
> > + cases, an additional doorbell is allocated and used for non-channel
> > + communication (e.g. for PTP, HWS, etc.). Supported values are:
> > + - 0: No channel-specific doorbells, use the global one for everything.
> > + - [1, max_num_channels]: Spread netdev channels equally across these
> > + doorbells.
>
> Do you have any guidance on this number? Why would the user want
> `num_doorbells < num_doorbells` vs `num_doorbells == num_channels`?
I expect it to be common that most deployment should continue to use
the historical value of num_doorbells = 0.
Certain systems with troubled CPUs will need to increase this, I don't
know if we yet fully understand what values these CPUs will need.
Nor do I think I'm permitted to say what CPUs are troubled :\
> IOW, why not allocate the same number of doorbells as the number of
> channels and do it unconditionally without devlink param? Are extra
> doorbells causing any overhead in the non-contended case?
It has a cost that should be minimized to not harm the current
majority of users.
Jason
next prev parent reply other threads:[~2025-09-10 16:23 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-10 10:24 [PATCH net-next 00/10] net/mlx5e: Use multiple doorbells Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 01/10] net/mlx5: Fix typo of MLX5_EQ_DOORBEL_OFFSET Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 02/10] net/mlx5: Remove unused 'offset' field from mlx5_sq_bfreg Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 03/10] net/mlx5e: Remove unused 'xsk' param of mlx5e_build_xdpsq_param Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 04/10] net/mlx5: Store the global doorbell in mlx5_priv Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 05/10] net/mlx5e: Prepare for using multiple TX doorbells Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 06/10] net/mlx5e: Prepare for using different CQ doorbells Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 07/10] net/mlx5e: Use multiple TX doorbells Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 08/10] net/mlx5e: Use multiple CQ doorbells Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 09/10] devlink: Add a 'num_doorbells' driverinit param Tariq Toukan
2025-09-10 10:24 ` [PATCH net-next 10/10] net/mlx5e: Use the 'num_doorbells' devlink param Tariq Toukan
2025-09-10 16:16 ` Stanislav Fomichev
2025-09-10 16:23 ` Jason Gunthorpe [this message]
2025-09-16 8:32 ` Cosmin Ratiu
2025-09-10 17:23 ` Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250910162310.GF882933@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=andrew+netdev@lunn.ch \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=cratiu@nvidia.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dtatulea@nvidia.com \
--cc=edumazet@google.com \
--cc=gal@nvidia.com \
--cc=hawk@kernel.org \
--cc=jiri@nvidia.com \
--cc=jiri@resnulli.us \
--cc=john.fastabend@gmail.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=saeedm@nvidia.com \
--cc=stfomichev@gmail.com \
--cc=tariqt@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.