From: Long Li <longli@microsoft.com>
To: Long Li <longli@microsoft.com>,
Konstantin Taranov <kotaranov@microsoft.com>,
Jakub Kicinski <kuba@kernel.org>,
"David S . Miller" <davem@davemloft.net>,
Paolo Abeni <pabeni@redhat.com>,
Eric Dumazet <edumazet@google.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
Jason Gunthorpe <jgg@ziepe.ca>, Leon Romanovsky <leon@kernel.org>,
Haiyang Zhang <haiyangz@microsoft.com>,
"K . Y . Srinivasan" <kys@microsoft.com>,
Wei Liu <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
shradhagupta@linux.microsoft.com, Simon Horman <horms@kernel.org>,
ernis@linux.microsoft.com, stephen@networkplumber.org
Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org,
linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH net-next v2 12/13] net: mana: keep the surviving queues when the channel count is reduced
Date: Wed, 12 Aug 2026 22:04:17 -0700 [thread overview]
Message-ID: <20260813050418.2906468-13-longli@microsoft.com> (raw)
In-Reply-To: <20260813050418.2906468-1-longli@microsoft.com>
A channel-count reduction rebuilds every queue it keeps, even though none
of their properties change. Shrinking 16 channels to 8 creates 8 TX and 8
RX queues and then destroys all 16 of each:
set_channels 16 -> 8 created SQ=8 RQ=8 | destroyed SQ=16 RQ=16
A queue is derived from the ring sizes, the MTU/priv-flag/XDP buffer
layout and its EQ, none of which depend on the count, and queue i is
bound to EQ i at any count. Carry the survivors over and retire only the
tail: a reduction allocates two pointer arrays and a steering table
instead of a full queue set, the peak stays at one set rather than two,
and the queues that stay keep their page pools, posted RX buffers and
NAPI state.
A ring resize, MTU change, priv-flag toggle or XDP attach changes a
property of every queue, so those still rebuild.
Signed-off-by: Long Li <longli@microsoft.com>
---
drivers/net/ethernet/microsoft/mana/mana_en.c | 127 ++++++++++++++++++
.../ethernet/microsoft/mana/mana_ethtool.c | 33 +++++
include/net/mana/mana.h | 4 +
3 files changed, 164 insertions(+)
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 6e77b59cfcf907f7e584625273972bf185501d20..c2b8ac67963fb6134a21b682a1b9ee373b1d0b7c 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -4186,6 +4186,133 @@ void mana_qset_scratch_free(struct mana_port_context *scratch)
kvfree(scratch);
}
+/* Carve the live set into a kept prefix [0, @new_count) in @out_new and a
+ * tail to retire in @out_tail. @apc is untouched, on failure too.
+ *
+ * Queue i is built from the ring sizes and the buffer layout and keeps EQ i
+ * at any count, so a reduction carries the survivors over and destroys only
+ * the tail. Allocates two pointer arrays and a steering table, nothing else:
+ * the queues that stay keep their page pools, posted buffers and NAPI.
+ */
+int mana_split_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_tail)
+{
+ unsigned int old_count = apc->num_queues;
+ struct mana_tx_qp **new_tx, **tail_tx;
+ struct mana_rxq **new_rx, **tail_rx;
+ unsigned int tail_count;
+ bool indir_lost;
+ unsigned int i;
+ int err;
+
+ ASSERT_RTNL();
+
+ if (WARN_ON(new_count == 0 || new_count >= old_count))
+ return -EINVAL;
+ if (WARN_ON(!apc->tx_qp || !apc->rxqs))
+ return -EINVAL;
+
+ tail_count = old_count - new_count;
+
+ /* Build the smaller set's steering table separately: mana_config_rss()
+ * would otherwise index the shorter rxqs[] with entries still referring
+ * to retired queues.
+ */
+ scratch->num_queues = new_count;
+ err = mana_rss_table_alloc(scratch);
+ if (err)
+ return err;
+
+ if (mana_rss_table_keep(apc, new_count, &indir_lost))
+ memcpy(scratch->indir_table, apc->indir_table,
+ apc->indir_table_sz * sizeof(*apc->indir_table));
+ else
+ mana_rss_table_init(scratch);
+
+ new_tx = kzalloc_objs(struct mana_tx_qp *, new_count);
+ new_rx = kzalloc_objs(struct mana_rxq *, new_count);
+ tail_tx = kzalloc_objs(struct mana_tx_qp *, tail_count);
+ tail_rx = kzalloc_objs(struct mana_rxq *, tail_count);
+ if (!new_tx || !new_rx || !tail_tx || !tail_rx) {
+ err = -ENOMEM;
+ goto free_arrays;
+ }
+
+ for (i = 0; i < new_count; i++) {
+ new_tx[i] = apc->tx_qp[i];
+ new_rx[i] = apc->rxqs[i];
+ }
+ for (i = 0; i < tail_count; i++) {
+ tail_tx[i] = apc->tx_qp[new_count + i];
+ tail_rx[i] = apc->rxqs[new_count + i];
+ }
+
+ /* The kept prefix, with the new steering table. */
+ out_new->tx_qp = new_tx;
+ out_new->rxqs = new_rx;
+ out_new->indir_table = scratch->indir_table;
+ out_new->indir_table_sz = scratch->indir_table_sz;
+ out_new->rxobj_table = scratch->rxobj_table;
+ out_new->default_rxobj = apc->rxqs[0]->rxobj;
+ out_new->num_queues = new_count;
+ out_new->rx_queue_size = apc->rx_queue_size;
+ out_new->tx_queue_size = apc->tx_queue_size;
+ out_new->priv_flags = apc->priv_flags;
+ out_new->mtu = apc->configured_mtu;
+ out_new->bpf_prog = apc->bpf_prog;
+ out_new->rxfh_indir_lost = indir_lost;
+
+ /* Ownership of the table moved to @out_new. */
+ scratch->indir_table = NULL;
+ scratch->rxobj_table = NULL;
+
+ /* The tail. It owns no steering table; bpf_prog is carried so that
+ * retiring it drops exactly the tail's per-queue program references
+ * and leaves the kept ones alone.
+ */
+ memset(out_tail, 0, sizeof(*out_tail));
+ out_tail->tx_qp = tail_tx;
+ out_tail->rxqs = tail_rx;
+ out_tail->default_rxobj = INVALID_MANA_HANDLE;
+ out_tail->num_queues = tail_count;
+ out_tail->rx_queue_size = apc->rx_queue_size;
+ out_tail->tx_queue_size = apc->tx_queue_size;
+ out_tail->priv_flags = apc->priv_flags;
+ out_tail->mtu = apc->configured_mtu;
+ out_tail->bpf_prog = apc->bpf_prog;
+
+ return 0;
+
+free_arrays:
+ kfree(new_tx);
+ kfree(new_rx);
+ kfree(tail_tx);
+ kfree(tail_rx);
+ mana_cleanup_indir_table(scratch);
+ return err;
+}
+
+/**
+ * mana_discard_split - drop the containers built by mana_split_qset()
+ * @newq: set that was never published
+ * @tailq: matching tail
+ *
+ * Frees the pointer arrays and the steering table only: the queues they refer
+ * to are still owned by the live port context.
+ */
+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq)
+{
+ kfree(newq->tx_qp);
+ kfree(newq->rxqs);
+ kfree(newq->indir_table);
+ kfree(newq->rxobj_table);
+ kfree(tailq->tx_qp);
+ kfree(tailq->rxqs);
+ memset(newq, 0, sizeof(*newq));
+ memset(tailq, 0, sizeof(*tailq));
+}
+
/* Rebuild the queues at the current count in @scratch, for callers changing a
* per-queue property; a count change goes through mana_split_qset() or
* mana_grow_qset(), so this never has to add an EQ. The installed set keeps
diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
index 024119dd4e353e33d11ccc883b4fe09a99434a26..415422aa68672accd03c161732fa1e96dd5562ba 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
@@ -722,6 +722,39 @@ static int mana_set_channels(struct net_device *ndev,
goto clear_flag;
}
+ /* A reduction keeps its queues configured identically, so carry them
+ * over and retire only the tail: no DMA ring, no hardware WQ object,
+ * and no old+new peak.
+ */
+ if (new_count < apc->num_queues) {
+ struct mana_qset tailq;
+
+ err = mana_split_qset(apc, scratch, new_count, &newq, &tailq);
+ if (err)
+ goto free_scratch; /* current qset untouched */
+
+ err = mana_publish_qset(apc, &newq, &oldq);
+ if (err) {
+ /* The old set is live again; drop only the containers
+ * built above, never the queues they point at.
+ */
+ mana_discard_split(&newq, &tailq);
+ goto free_scratch;
+ }
+
+ /* @oldq holds the original arrays and steering table. Every
+ * queue they referenced is now owned by either the published
+ * set or the tail, so only the containers are freed here.
+ */
+ kfree(oldq.tx_qp);
+ kfree(oldq.rxqs);
+ kfree(oldq.indir_table);
+ kfree(oldq.rxobj_table);
+
+ mana_free_qset(apc, scratch, &tailq);
+ goto free_scratch;
+ }
+
err = mana_alloc_qset(apc, scratch, new_count, apc->rx_queue_size,
apc->tx_queue_size, apc->priv_flags,
apc->configured_mtu, apc->bpf_prog, &newq);
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index 0d7375adeb5366bbbb99a7faa931a8830f71ef10..cd41a135815710162b1e5fd3edafa119ae12d1dd 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -760,6 +760,10 @@ int mana_alloc_qset(struct mana_port_context *apc,
unsigned int rx_queue_size, unsigned int tx_queue_size,
u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,
struct mana_qset *out);
+int mana_split_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_tail);
+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq);
int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,
struct mana_qset *out_old);
void mana_publish_close_if_needed(struct mana_port_context *apc);
--
2.43.0
next prev parent reply other threads:[~2026-08-13 5:05 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-13 5:04 [PATCH net-next v2 00/13] net: mana: reconfigure by replacing the queue set Long Li
2026-08-13 5:04 ` [PATCH net-next v2 01/13] net: mana: add queue-set allocation and teardown helpers Long Li
2026-08-13 5:04 ` [PATCH net-next v2 02/13] net: mana: swap queue sets in mana_set_channels Long Li
2026-08-13 5:04 ` [PATCH net-next v2 03/13] net: mana: swap queue sets in mana_set_ringparam Long Li
2026-08-13 5:04 ` [PATCH net-next v2 04/13] net: mana: swap queue sets in mana_set_priv_flags Long Li
2026-08-13 5:04 ` [PATCH net-next v2 05/13] net: mana: swap queue sets in mana_change_mtu Long Li
2026-08-13 5:04 ` [PATCH net-next v2 06/13] net: mana: swap queue sets in mana_xdp_set Long Li
2026-08-13 5:04 ` [PATCH net-next v2 07/13] net: mana: do not bail out of mana_detach on dealloc failure Long Li
2026-08-13 5:04 ` [PATCH net-next v2 08/13] net: mana: keep per-queue statistics in the port context Long Li
2026-08-13 5:04 ` [PATCH net-next v2 09/13] net: mana: share the EQ pool across a queue-set swap Long Li
2026-08-13 5:04 ` [PATCH net-next v2 10/13] net: mana: release EQs left idle by a channel-count reduction Long Li
2026-08-13 5:04 ` [PATCH net-next v2 11/13] net: mana: keep a user-configured RSS table across a queue rebuild Long Li
2026-08-13 5:04 ` Long Li [this message]
2026-08-13 5:04 ` [PATCH net-next v2 13/13] net: mana: keep the existing queues when the channel count is raised Long Li
-- strict thread matches above, loose matches on Subject: below --
2026-08-11 6:35 [PATCH net-next v2 01/13] net: mana: add queue-set allocation and teardown helpers Long Li
2026-08-11 6:35 ` [PATCH net-next v2 12/13] net: mana: keep the surviving queues when the channel count is reduced Long Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260813050418.2906468-13-longli@microsoft.com \
--to=longli@microsoft.com \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=decui@microsoft.com \
--cc=edumazet@google.com \
--cc=ernis@linux.microsoft.com \
--cc=haiyangz@microsoft.com \
--cc=horms@kernel.org \
--cc=jgg@ziepe.ca \
--cc=kotaranov@microsoft.com \
--cc=kuba@kernel.org \
--cc=kys@microsoft.com \
--cc=leon@kernel.org \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=shradhagupta@linux.microsoft.com \
--cc=stephen@networkplumber.org \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox