From: Ido Schimmel <idosch@nvidia.com>
To: netdev@vger.kernel.org
Cc: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com,
edumazet@google.com, petrm@nvidia.com, mlxsw@nvidia.com,
Ido Schimmel <idosch@nvidia.com>
Subject: [PATCH net-next 0/4] mlxsw: Remove size limitations on egress descriptor buffer
Date: Mon, 2 May 2022 11:49:22 +0300 [thread overview]
Message-ID: <20220502084926.365268-1-idosch@nvidia.com> (raw)
Petr says:
Spectrum machines have two resources related to keeping packets in an
internal buffer: bytes (allocated in cell-sized units) for packet payload,
and descriptors, for keeping headers. Currently, mlxsw only configures the
bytes part of the resource management.
Spectrum switches permit a full parallel configuration for the descriptor
resources, including port-pool and port-TC-pool quotas. By default, these
are all configured to use pool 14, with an infinite quota. The ingress pool
14 is then infinite in size.
However, egress pool 14 has finite size by default. The size is chip
dependent, but always much lower than what the chip actually permits. As a
result, we can easily construct workloads that exhaust the configured
descriptor limit.
Going forward, mlxsw will have to fix this issue properly by maintaining
descriptor buffer sizes, TC bindings, and quotas that match the
architecture recommendation. Short term, fix the issue by configuring the
egress descriptor pool to be infinite in size as well. This will maintain
the same configuration philosophy, but will unlock all chip resources to be
usable.
In this patchset, patch #1 first adds the "desc" field into the pool
configuration register. Then in patch #2, the new field is used to
configure both ingress and egress pool 14 as infinite.
In patches #3 and #4, add a selftest that verifies that a large burst
can be absorbed by the shared buffer. This test specifically exercises a
scenario where descriptor buffer is the limiting factor and the test
fails without the above patches.
Petr Machata (4):
mlxsw: reg: Add "desc" field to SBPR
mlxsw: Configure descriptor buffers
selftests: forwarding: lib: Add start_traffic_pktsize() helpers
selftests: mlxsw: Add a test for soaking up a burst of traffic
drivers/net/ethernet/mellanox/mlxsw/reg.h | 6 +
.../mellanox/mlxsw/spectrum_buffers.c | 26 +
.../selftests/drivers/net/mlxsw/qos_burst.sh | 480 ++++++++++++++++++
tools/testing/selftests/net/forwarding/lib.sh | 21 +-
4 files changed, 530 insertions(+), 3 deletions(-)
create mode 100755 tools/testing/selftests/drivers/net/mlxsw/qos_burst.sh
--
2.35.1
next reply other threads:[~2022-05-02 8:50 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-02 8:49 Ido Schimmel [this message]
2022-05-02 8:49 ` [PATCH net-next 1/4] mlxsw: reg: Add "desc" field to SBPR Ido Schimmel
2022-05-02 8:49 ` [PATCH net-next 2/4] mlxsw: Configure descriptor buffers Ido Schimmel
2022-05-02 8:49 ` [PATCH net-next 3/4] selftests: forwarding: lib: Add start_traffic_pktsize() helpers Ido Schimmel
2022-05-02 8:49 ` [PATCH net-next 4/4] selftests: mlxsw: Add a test for soaking up a burst of traffic Ido Schimmel
2022-05-03 10:20 ` [PATCH net-next 0/4] mlxsw: Remove size limitations on egress descriptor buffer patchwork-bot+netdevbpf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220502084926.365268-1-idosch@nvidia.com \
--to=idosch@nvidia.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=mlxsw@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=petrm@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).