Netdev List
 help / color / mirror / Atom feed
* [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload
@ 2026-11-18  9:30 Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 1/6] net: mvneta: lift the per-queue TX rate limits Josef Schlehofer
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Josef Schlehofer @ 2026-11-18  9:30 UTC (permalink / raw)
  To: netdev
  Cc: marcin.s.wojtas, andrew+netdev, davem, edumazet, kuba, pabeni,
	maxime.chevallier, w, linux-kernel

The NETA TX arbiter is left at its reset default, fixed priority.
Under load a busy TX queue starves the others. OpenWrt has worked
around this since 2017 by sending all TX traffic of its 32-bit boards
through queue 0 [1].

Patch 6 serves the TX queues in weighted round robin, as mvpp2 does.
With an mqprio offload the fixed priority stays, so a class that fills
the link still starves the lower classes. With the default priority
map this includes ARP, which is in class 0.

Patches 1 to 5 fix older bugs in the mqprio offload, found while
working on patch 6. All of them need an admin to set up an mqprio
offload, and the oops fixed by patch 3 also needs mvneta.txq_number
below rxq_number (both default to 8). v4 therefore targets net-next.
If patches 1 to 5 should go to net instead, I can send them as a
separate series.

With patch 2, replacing an offloaded mqprio qdisc under a new handle
loses the traffic classes of the new qdisc. Patch 2 explains why.

Tested on a Turris Omnia (Armada 385) as net f56daaadfeca plus this
series, which applies unchanged to net-next. Without an offload four
TCP streams got 23.5 Mbit/s each on a 100 Mbit/s link and 232 Mbit/s
each on a 1 Gbit/s link. With an offload the higher class got 94 and
926 Mbit/s. Only the Armada 385 was tested, the version 1 registers
for the Armada 370 and XP come from their manuals, and suspend and
resume were not tested.

Changes in v4:
- Retargeted from net to net-next.
- Patch 5 replaces "unwind a failed mqprio setup" and checks the whole
  setup before writing, so a rejected replace keeps the hardware setup
  of the attached qdisc (Sashiko). Reviewed-by dropped.
- Patch 3 is new and checks the TX queue range without a shaper too.
- Patches 3 and 4 come before patch 5, so they apply without it.
- Patch 6 also writes the version 1 arbiter registers and keeps the
  fixed priority when a setup is rejected.
- Patches 1 and 2 keep their code and their Reviewed-by tags.
- v3: https://lore.kernel.org/netdev/20261104224518.66487-1-pepe.schlehofer@gmail.com/
- v2: https://lore.kernel.org/netdev/20261103125021.70845-1-pepe.schlehofer@gmail.com/
- v1: https://lore.kernel.org/netdev/20261102112816.12322-1-pepe.schlehofer@gmail.com/

[1] https://github.com/openwrt/openwrt/blob/a5e96c837ead549820867bb83c1ad471e39f5ace/target/linux/mvebu/patches-6.18/700-mvneta-tx-queue-workaround.patch

Josef Schlehofer (6):
  net: mvneta: lift the per-queue TX rate limits
  net: mvneta: fix the removal of the mqprio offload
  net: mvneta: check the TX queue range of an mqprio offload
  net: mvneta: keep the TX rate limits across open and MTU change
  net: mvneta: validate an mqprio setup before applying it
  net: mvneta: serve the TX queues in weighted round robin

 drivers/net/ethernet/marvell/mvneta.c | 184 +++++++++++++++++++-------
 1 file changed, 137 insertions(+), 47 deletions(-)


base-commit: 7ccc5e05043b1bb84326b19a2b05938d8f7a2ae6
-- 
2.54.0 (Apple Git-157)


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH net-next v4 1/6] net: mvneta: lift the per-queue TX rate limits
  2026-11-18  9:30 [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload Josef Schlehofer
@ 2026-11-18  9:30 ` Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 2/6] net: mvneta: fix the removal of the mqprio offload Josef Schlehofer
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Josef Schlehofer @ 2026-11-18  9:30 UTC (permalink / raw)
  To: netdev
  Cc: marcin.s.wojtas, andrew+netdev, davem, edumazet, kuba, pabeni,
	maxime.chevallier, w, linux-kernel

After the mqprio shaper is turned off, the per-queue rate limits stay
in effect. mvneta_disable_per_queue_rate_limit() only changes the mode
bits in MVNETA_TXQ_CMD1_REG, but the token buckets limit the TX queues
in that mode as well, and their refill still holds the old rate.

Refill the token buckets of all TX queues at full rate when the rate
limit is disabled. Deleting the qdisc reaches this code only with the
next patch.

Triggered during code development on a Turris Omnia (Armada 385) with
a 100 Mbit/s link:

  tc qdisc add dev eth2 root handle 100: mqprio num_tc 2 \
    map 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 queues 4@0 4@4 hw 1 \
    mode channel shaper bw_rlimit max_rate 10Mbit 10Mbit
  tc qdisc del dev eth2 root
  tc qdisc add dev eth2 root handle 100: mqprio num_tc 2 \
    map 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 queues 4@0 4@4 hw 1

Each TX queue then stayed at 10 Mbit/s. With this patch and the next
one the link reaches 94 Mbit/s, and 941 Mbit/s on a 1 Gbit/s link. No
kernel selftest covers the mvneta mqprio offload.

Fixes: 2551dc9e398c ("net: mvneta: Add TC traffic shaping offload")
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Reviewed-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
---
v4: Commit message only, commands and discovery statement added.
---
 drivers/net/ethernet/marvell/mvneta.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 915c4c35364a..c0abc843b1c1 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -5204,9 +5204,16 @@ static int mvneta_enable_per_queue_rate_limit(struct mvneta_port *pp)
 static void mvneta_disable_per_queue_rate_limit(struct mvneta_port *pp)
 {
 	u32 val = mvreg_read(pp, MVNETA_TXQ_CMD1_REG);
+	int queue;
 
 	val |= (MVNETA_TXQ_CMD1_BW_LIM_SEL_V1 | MVNETA_TXQ_CMD1_BW_LIM_EN);
 	mvreg_write(pp, MVNETA_TXQ_CMD1_REG, val);
+
+	/* The token buckets still limit the queues, refill them at full rate */
+	val = (1 << MVNETA_TXQ_BUCKET_REFILL_PERIOD_SHIFT) |
+	      MVNETA_TXQ_BUCKET_REFILL_VALUE_MAX;
+	for (queue = 0; queue < txq_number; queue++)
+		mvreg_write(pp, MVNETA_TXQ_BUCKET_REFILL_REG(queue), val);
 }
 
 static int mvneta_setup_queue_rates(struct mvneta_port *pp, int queue,
-- 
2.54.0 (Apple Git-157)


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH net-next v4 2/6] net: mvneta: fix the removal of the mqprio offload
  2026-11-18  9:30 [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 1/6] net: mvneta: lift the per-queue TX rate limits Josef Schlehofer
@ 2026-11-18  9:30 ` Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 3/6] net: mvneta: check the TX queue range of an " Josef Schlehofer
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Josef Schlehofer @ 2026-11-18  9:30 UTC (permalink / raw)
  To: netdev
  Cc: marcin.s.wojtas, andrew+netdev, davem, edumazet, kuba, pabeni,
	maxime.chevallier, w, linux-kernel

Deleting an offloaded mqprio qdisc leaves its traffic classes and
per-queue rate limits in place:

  tc qdisc add dev eth2 root handle 100: mqprio num_tc 2 \
    map 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 queues 4@0 4@4 hw 1 \
    mode channel shaper bw_rlimit max_rate 10Mbit 10Mbit
  tc qdisc del dev eth2 root

mqprio_disable_offload() passes an all-zero configuration, so qopt.hw
is 0 and mvneta_setup_mqprio() returns before it reaches the teardown
code. Check qopt.hw only when traffic classes are set up.

With this fix, replacing an offloaded mqprio qdisc under a new handle
by another qdisc with traffic classes loses them. The replacement is
set up before the old qdisc is destroyed, and the teardown call does
not say which qdisc it belongs to. The old qdisc's teardown therefore
clears the traffic classes of the new one, and for a new offload also
its rate limits and RX queue mapping. tc then shows the new qdisc
without traffic classes. The core does the same when the old qdisc is
a software mqprio. A taprio qdisc left without traffic classes sends
nothing.

Triggered during code development. Reading mvneta_setup_mqprio()
showed the early return, and the commands above confirmed it on a
Turris Omnia (Armada 385) with a 100 Mbit/s link. Only TX queues 0-3
were used and the link carried 19 Mbit/s. With this patch and the
previous one the queues above 3 are used again and the link reaches
94 Mbit/s. No kernel selftest covers the mvneta mqprio offload.

Fixes: e7ca75fe6662 ("net: mvneta: Don't force-set the offloading flag")
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Reviewed-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
---
v4: Commit message only. Commands and discovery statement added, and
the replace paragraph also names the rate limits and the RX queue
mapping (Sashiko).
---
 drivers/net/ethernet/marvell/mvneta.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index c0abc843b1c1..06b5453543a1 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -5251,11 +5251,12 @@ static int mvneta_setup_mqprio(struct net_device *dev,
 	int rxq, txq, tc, ret;
 	u8 num_tc;
 
-	if (mqprio->qopt.hw != TC_MQPRIO_HW_OFFLOAD_TCS)
-		return 0;
-
 	num_tc = mqprio->qopt.num_tc;
 
+	/* On removal the qdisc passes an all-zero qopt, with hw 0 */
+	if (num_tc && mqprio->qopt.hw != TC_MQPRIO_HW_OFFLOAD_TCS)
+		return 0;
+
 	if (num_tc > rxq_number)
 		return -EINVAL;
 
-- 
2.54.0 (Apple Git-157)


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH net-next v4 3/6] net: mvneta: check the TX queue range of an mqprio offload
  2026-11-18  9:30 [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 1/6] net: mvneta: lift the per-queue TX rate limits Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 2/6] net: mvneta: fix the removal of the mqprio offload Josef Schlehofer
@ 2026-11-18  9:30 ` Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 4/6] net: mvneta: keep the TX rate limits across open and MTU change Josef Schlehofer
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Josef Schlehofer @ 2026-11-18  9:30 UTC (permalink / raw)
  To: netdev
  Cc: marcin.s.wojtas, andrew+netdev, davem, edumazet, kuba, pabeni,
	maxime.chevallier, w, linux-kernel

An mqprio offload without a shaper can map a traffic class to TX
queues that do not exist. Reading the class statistics then reads past
the TX queue array of the device, also as an unprivileged user:

  # kernel command line: mvneta.txq_number=4
  tc qdisc add dev eth2 root handle 100: mqprio num_tc 2 \
    map 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 queues 4@0 4@4 hw 1
  tc -s class show dev eth2

mvneta_setup_mqprio() checks the queue ranges against the number of RX
queues, and against the number of TX queues only when a shaper is set.
With fewer TX than RX queues a range can pass the RX check and end
beyond the last TX queue. Check every queue of a class against both
numbers.

Found by an AI code review agent while working on this series, and
confirmed on a Turris Omnia (Armada 385) booted with
mvneta.txq_number=4. Without this patch the setup was accepted, and
the class dump as uid 65534 caused a kernel oops in
mqprio_dump_class_stats(). With this patch the range is rejected with
-EINVAL, so the class dump stays within the TX queue array. Tested
with the series applied up to this patch and with the whole series
applied. No kernel selftest covers the mvneta mqprio offload.

Fixes: 4906887a8ae5 ("net: mvneta: Implement mqprio support")
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
---
v4: New patch. Answering TC_QUERY_CAPS with validate_queue_counts would
let the core check the TX range, but it would also reject classes
without queues and overlapping ranges, which the driver accepts today,
and it does not check the RX queues.
---
 drivers/net/ethernet/marvell/mvneta.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 06b5453543a1..76dfbc484269 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -5277,7 +5277,8 @@ static int mvneta_setup_mqprio(struct net_device *dev,
 		for (rxq = mqprio->qopt.offset[tc];
 		     rxq < mqprio->qopt.count[tc] + mqprio->qopt.offset[tc];
 		     rxq++) {
-			if (rxq >= rxq_number)
+			/* The range sets the TX queues of the class as well */
+			if (rxq >= rxq_number || rxq >= txq_number)
 				return -EINVAL;
 
 			mvneta_map_vlan_prio_to_rxq(pp, tc, rxq);
-- 
2.54.0 (Apple Git-157)


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH net-next v4 4/6] net: mvneta: keep the TX rate limits across open and MTU change
  2026-11-18  9:30 [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload Josef Schlehofer
                   ` (2 preceding siblings ...)
  2026-11-18  9:30 ` [PATCH net-next v4 3/6] net: mvneta: check the TX queue range of an " Josef Schlehofer
@ 2026-11-18  9:30 ` Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 5/6] net: mvneta: validate an mqprio setup before applying it Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 6/6] net: mvneta: serve the TX queues in weighted round robin Josef Schlehofer
  5 siblings, 0 replies; 7+ messages in thread
From: Josef Schlehofer @ 2026-11-18  9:30 UTC (permalink / raw)
  To: netdev
  Cc: marcin.s.wojtas, andrew+netdev, davem, edumazet, kuba, pabeni,
	maxime.chevallier, w, linux-kernel

After the interface is brought up again or its MTU is changed, every
per-queue rate limit set by mqprio runs at 10.23 Mbit/s, whatever rate
was configured. tc still shows the configured rates.

mvneta_txq_hw_init() writes the token configuration of the legacy rate
limiter for each TX queue. On the Armada 385 this also sets the refill
value of the queue's token bucket to 1023, which is 10.23 Mbit/s at
the refill period that mqprio uses.

Remember the refill of each rate limited queue and program the rate
limits again when the port is started. Resume also starts the port
with mvneta_start_dev().

Found by an AI code review agent while working on this series, and
confirmed on a Turris Omnia (Armada 385) with 1 Mbit/s limits on a
100 Mbit/s link. After down and up or an MTU change each busy TX queue
carried about 10 Mbit/s. With this patch it is back at 1.0 Mbit/s
after a burst of about one second, and the limits also survive
ethtool -r. Resume was not tested. No kernel selftest covers the
mvneta mqprio offload.

Fixes: 2551dc9e398c ("net: mvneta: Add TC traffic shaping offload")
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
---
v4: Moved before the validation patch, so it applies without it.
Discovery statement added.
---
 drivers/net/ethernet/marvell/mvneta.c | 31 ++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 76dfbc484269..58453c69be36 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -683,6 +683,9 @@ struct mvneta_tx_queue {
 
 	u32 done_pkts_coal;
 
+	/* Bucket refill of the mqprio rate limit, 0 when not limited */
+	u32 bucket_refill;
+
 	/* Virtual address of the TX DMA descriptors array */
 	struct mvneta_tx_desc *descs;
 
@@ -3752,6 +3755,8 @@ static int mvneta_config_interface(struct mvneta_port *pp,
 	return ret;
 }
 
+static void mvneta_restore_per_queue_rate_limit(struct mvneta_port *pp);
+
 static void mvneta_start_dev(struct mvneta_port *pp)
 {
 	int cpu;
@@ -3760,6 +3765,7 @@ static void mvneta_start_dev(struct mvneta_port *pp)
 
 	mvneta_max_rx_size_set(pp, pp->pkt_size);
 	mvneta_txq_max_tx_size_set(pp, pp->pkt_size);
+	mvneta_restore_per_queue_rate_limit(pp);
 
 	/* start the Rx/Tx activity */
 	mvneta_port_enable(pp);
@@ -5212,8 +5218,10 @@ static void mvneta_disable_per_queue_rate_limit(struct mvneta_port *pp)
 	/* The token buckets still limit the queues, refill them at full rate */
 	val = (1 << MVNETA_TXQ_BUCKET_REFILL_PERIOD_SHIFT) |
 	      MVNETA_TXQ_BUCKET_REFILL_VALUE_MAX;
-	for (queue = 0; queue < txq_number; queue++)
+	for (queue = 0; queue < txq_number; queue++) {
+		pp->txqs[queue].bucket_refill = 0;
 		mvreg_write(pp, MVNETA_TXQ_BUCKET_REFILL_REG(queue), val);
+	}
 }
 
 static int mvneta_setup_queue_rates(struct mvneta_port *pp, int queue,
@@ -5239,11 +5247,32 @@ static int mvneta_setup_queue_rates(struct mvneta_port *pp, int queue,
 	val |= (MVNETA_TXQ_BUCKET_REFILL_PERIOD <<
 		MVNETA_TXQ_BUCKET_REFILL_PERIOD_SHIFT);
 
+	pp->txqs[queue].bucket_refill = val;
 	mvreg_write(pp, MVNETA_TXQ_BUCKET_REFILL_REG(queue), val);
 
 	return 0;
 }
 
+/* The TX queue setup overwrites the rate limits, program them again */
+static void mvneta_restore_per_queue_rate_limit(struct mvneta_port *pp)
+{
+	int queue;
+
+	for (queue = 0; queue < txq_number; queue++) {
+		if (pp->txqs[queue].bucket_refill)
+			break;
+	}
+
+	if (queue == txq_number || mvneta_enable_per_queue_rate_limit(pp))
+		return;
+
+	for (; queue < txq_number; queue++) {
+		if (pp->txqs[queue].bucket_refill)
+			mvreg_write(pp, MVNETA_TXQ_BUCKET_REFILL_REG(queue),
+				    pp->txqs[queue].bucket_refill);
+	}
+}
+
 static int mvneta_setup_mqprio(struct net_device *dev,
 			       struct tc_mqprio_qopt_offload *mqprio)
 {
-- 
2.54.0 (Apple Git-157)


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH net-next v4 5/6] net: mvneta: validate an mqprio setup before applying it
  2026-11-18  9:30 [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload Josef Schlehofer
                   ` (3 preceding siblings ...)
  2026-11-18  9:30 ` [PATCH net-next v4 4/6] net: mvneta: keep the TX rate limits across open and MTU change Josef Schlehofer
@ 2026-11-18  9:30 ` Josef Schlehofer
  2026-11-18  9:30 ` [PATCH net-next v4 6/6] net: mvneta: serve the TX queues in weighted round robin Josef Schlehofer
  5 siblings, 0 replies; 7+ messages in thread
From: Josef Schlehofer @ 2026-11-18  9:30 UTC (permalink / raw)
  To: netdev
  Cc: marcin.s.wojtas, andrew+netdev, davem, edumazet, kuba, pabeni,
	maxime.chevallier, w, linux-kernel

A failed mqprio offload setup leaves part of its configuration in the
hardware. When one max_rate is not a multiple of 10 kbit/s, the
command fails, but the RX queue mapping, the rate limit mode and the
rates of TX queues 0-3 are already written:

  tc qdisc add dev eth2 root handle 100: mqprio num_tc 2 \
    map 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 queues 4@0 4@4 hw 1 \
    mode channel shaper bw_rlimit max_rate 10Mbit 10001kbit

mvneta_setup_mqprio() checks the queue ranges and rates while it
programs them, and the qdisc then fails to attach without calling the
driver again.

Check the whole configuration first and write to the hardware only
when it is valid, so a setup that the driver rejects changes nothing.
A class without queues is still accepted. This also covers a rejected
replace under a new handle, which runs while the old offload stays
attached:

  tc qdisc add dev eth2 root handle 100: mqprio num_tc 2 \
    map 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 queues 4@0 4@4 hw 1 \
    mode channel shaper bw_rlimit max_rate 10Mbit 10Mbit
  tc qdisc replace dev eth2 root handle 200: mqprio num_tc 2 \
    map 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 queues 4@0 4@4 hw 1 \
    mode channel shaper bw_rlimit max_rate 10Mbit 10001kbit

The hardware setup of the old qdisc is kept. The rejected qdisc never
enabled its offload, so its mqprio_destroy() still calls
netdev_set_num_tc(dev, 0). Until the old qdisc is deleted, tc shows it
without traffic classes and the stack spreads the flows over all TX
queues.

Found by an AI code review agent while working on this series, and
confirmed on a Turris Omnia (Armada 385). Without this patch, after
the first command failed, TX queues 0-3 stayed limited to 10 Mbit/s
and the VLAN priority map stayed set. With this patch both failed
commands leave every register unchanged, and after the failed replace
the 10 Mbit/s limits of 100: stay in effect. No kernel selftest covers
the mvneta mqprio offload.

Fixes: e9f7099d0730 ("net: mvneta: Allow having more than one queue per TC")
Fixes: 2551dc9e398c ("net: mvneta: Add TC traffic shaping offload")
Link: https://lore.kernel.org/netdev/20261022144814.66366-1-shawnq1@163.com/
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
---
v4: Replaces "unwind a failed mqprio setup". The unwind also cleared
the hardware setup of an attached qdisc when its replacement failed,
as Sashiko pointed out on v3 [1]. Maxime's Reviewed-by is dropped
because the approach changed.

The rate conversion moves to mvneta_txq_refill_val() unchanged. A
follow-up will tighten its range check for very large rates.

[1] https://lore.kernel.org/netdev/179418382346.994134.3211728825005845989@kernel.org/
---
 drivers/net/ethernet/marvell/mvneta.c | 90 +++++++++++++--------------
 1 file changed, 44 insertions(+), 46 deletions(-)

diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 58453c69be36..89ade2ed7ce4 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -5224,11 +5224,10 @@ static void mvneta_disable_per_queue_rate_limit(struct mvneta_port *pp)
 	}
 }
 
-static int mvneta_setup_queue_rates(struct mvneta_port *pp, int queue,
-				    u64 min_rate, u64 max_rate)
+/* Convert a rate limit to the token bucket refill of a TX queue */
+static int mvneta_txq_refill_val(u64 min_rate, u64 max_rate, u32 *val)
 {
 	u32 refill_val, rem;
-	u32 val = 0;
 
 	/* Convert to from Bps to bps */
 	max_rate *= 8;
@@ -5243,12 +5242,8 @@ static int mvneta_setup_queue_rates(struct mvneta_port *pp, int queue,
 	    refill_val > MVNETA_TXQ_BUCKET_REFILL_VALUE_MAX)
 		return -EINVAL;
 
-	val = refill_val;
-	val |= (MVNETA_TXQ_BUCKET_REFILL_PERIOD <<
-		MVNETA_TXQ_BUCKET_REFILL_PERIOD_SHIFT);
-
-	pp->txqs[queue].bucket_refill = val;
-	mvreg_write(pp, MVNETA_TXQ_BUCKET_REFILL_REG(queue), val);
+	*val = refill_val | (MVNETA_TXQ_BUCKET_REFILL_PERIOD <<
+			     MVNETA_TXQ_BUCKET_REFILL_PERIOD_SHIFT);
 
 	return 0;
 }
@@ -5276,8 +5271,10 @@ static void mvneta_restore_per_queue_rate_limit(struct mvneta_port *pp)
 static int mvneta_setup_mqprio(struct net_device *dev,
 			       struct tc_mqprio_qopt_offload *mqprio)
 {
+	bool shaper = mqprio->shaper == TC_MQPRIO_SHAPER_BW_RATE;
 	struct mvneta_port *pp = netdev_priv(dev);
-	int rxq, txq, tc, ret;
+	u32 refill[TC_QOPT_MAX_QUEUE];
+	int queue, last, tc, ret;
 	u8 num_tc;
 
 	num_tc = mqprio->qopt.num_tc;
@@ -5286,58 +5283,59 @@ static int mvneta_setup_mqprio(struct net_device *dev,
 	if (num_tc && mqprio->qopt.hw != TC_MQPRIO_HW_OFFLOAD_TCS)
 		return 0;
 
-	if (num_tc > rxq_number)
-		return -EINVAL;
-
-	mvneta_clear_rx_prio_map(pp);
-
 	if (!num_tc) {
+		mvneta_clear_rx_prio_map(pp);
 		mvneta_disable_per_queue_rate_limit(pp);
 		netdev_reset_tc(dev);
 		return 0;
 	}
 
-	netdev_set_num_tc(dev, mqprio->qopt.num_tc);
+	/* Check everything first, a rejected setup must not change anything */
+	if (num_tc > rxq_number || (shaper && num_tc > txq_number))
+		return -EINVAL;
 
-	for (tc = 0; tc < mqprio->qopt.num_tc; tc++) {
-		netdev_set_tc_queue(dev, tc, mqprio->qopt.count[tc],
-				    mqprio->qopt.offset[tc]);
+	for (tc = 0; tc < num_tc; tc++) {
+		if (!mqprio->qopt.count[tc])
+			continue;
 
-		for (rxq = mqprio->qopt.offset[tc];
-		     rxq < mqprio->qopt.count[tc] + mqprio->qopt.offset[tc];
-		     rxq++) {
-			/* The range sets the TX queues of the class as well */
-			if (rxq >= rxq_number || rxq >= txq_number)
-				return -EINVAL;
+		last = mqprio->qopt.offset[tc] + mqprio->qopt.count[tc];
+		if (last > rxq_number || last > txq_number)
+			return -EINVAL;
 
-			mvneta_map_vlan_prio_to_rxq(pp, tc, rxq);
-		}
+		if (!shaper)
+			continue;
+
+		ret = mvneta_txq_refill_val(mqprio->min_rate[tc],
+					    mqprio->max_rate[tc], &refill[tc]);
+		if (ret)
+			return ret;
 	}
 
-	if (mqprio->shaper != TC_MQPRIO_SHAPER_BW_RATE) {
+	/* This can only fail before it writes to the hardware */
+	if (shaper) {
+		ret = mvneta_enable_per_queue_rate_limit(pp);
+		if (ret)
+			return ret;
+	} else {
 		mvneta_disable_per_queue_rate_limit(pp);
-		return 0;
 	}
 
-	if (mqprio->qopt.num_tc > txq_number)
-		return -EINVAL;
+	mvneta_clear_rx_prio_map(pp);
+	netdev_set_num_tc(dev, num_tc);
 
-	ret = mvneta_enable_per_queue_rate_limit(pp);
-	if (ret)
-		return ret;
+	for (tc = 0; tc < num_tc; tc++) {
+		netdev_set_tc_queue(dev, tc, mqprio->qopt.count[tc],
+				    mqprio->qopt.offset[tc]);
+
+		last = mqprio->qopt.offset[tc] + mqprio->qopt.count[tc];
+		for (queue = mqprio->qopt.offset[tc]; queue < last; queue++) {
+			mvneta_map_vlan_prio_to_rxq(pp, tc, queue);
+			if (!shaper)
+				continue;
 
-	for (tc = 0; tc < mqprio->qopt.num_tc; tc++) {
-		for (txq = mqprio->qopt.offset[tc];
-		     txq < mqprio->qopt.count[tc] + mqprio->qopt.offset[tc];
-		     txq++) {
-			if (txq >= txq_number)
-				return -EINVAL;
-
-			ret = mvneta_setup_queue_rates(pp, txq,
-						       mqprio->min_rate[tc],
-						       mqprio->max_rate[tc]);
-			if (ret)
-				return ret;
+			pp->txqs[queue].bucket_refill = refill[tc];
+			mvreg_write(pp, MVNETA_TXQ_BUCKET_REFILL_REG(queue),
+				    refill[tc]);
 		}
 	}
 
-- 
2.54.0 (Apple Git-157)


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH net-next v4 6/6] net: mvneta: serve the TX queues in weighted round robin
  2026-11-18  9:30 [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload Josef Schlehofer
                   ` (4 preceding siblings ...)
  2026-11-18  9:30 ` [PATCH net-next v4 5/6] net: mvneta: validate an mqprio setup before applying it Josef Schlehofer
@ 2026-11-18  9:30 ` Josef Schlehofer
  5 siblings, 0 replies; 7+ messages in thread
From: Josef Schlehofer @ 2026-11-18  9:30 UTC (permalink / raw)
  To: netdev
  Cc: marcin.s.wojtas, andrew+netdev, davem, edumazet, kuba, pabeni,
	maxime.chevallier, w, linux-kernel

Under load, some TX queues get no service at all. With four TCP
streams on a 100 Mbit/s link, up to three streams sent nothing for 20
seconds. The TX queue arbiter is left at its reset default, which
serves all queues in fixed priority, and the stack spreads the flows
over the queues.

Give the TX queues the same WRR weight, the smallest one that is
larger than the largest packet, and clear the fixed priority. The
Armada 370 and XP manuals list separate registers for the version 1
arbiter, which the driver runs unless a shaper is set, so write both
register sets. On the Armada 385 they are the same registers, as
writing one set and reading the other shows.

Keep the fixed priority while an mqprio offload is set up, as its
traffic classes may rely on it, and clear it when the offload is
removed. A rejected setup leaves it as it was, and after a rejected
replace it applies to all flows until the qdisc is deleted. A class
that fills the link starves the lower classes, and as the fixed
priority is set per queue, the queues of one class still starve each
other.

Observed on machines in real use. OpenWrt users reported it as a
multi-flow lockout on a Linksys WRT1200AC, and OpenWrt has worked
around it since 2017 by sending all TX traffic of its 32-bit boards
through queue 0. Reproduced on a Turris Omnia (Armada 385). With this
patch each busy TX queue gets the same share and no stream stalls,
4 x 23.5 Mbit/s on a 100 Mbit/s link and 4 x 232 Mbit/s on a 1 Gbit/s
link. With an mqprio offload a higher queue still has priority over a
lower one. Only the Armada 385 was tested. No kernel selftest covers
the TX queue arbiter.

Fixes: ee40a116ebf1 ("net: mvneta: fix improper tx queue usage in mvneta_tx()")
Link: https://github.com/openwrt/openwrt/issues/5411
Suggested-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
---
v4: The fixed priority and the weights are also written to the version
1 registers. A setup that the driver rejects no longer clears the fixed
priority. Discovery statement added.
---
 drivers/net/ethernet/marvell/mvneta.c | 56 ++++++++++++++++++++++++++-
 1 file changed, 55 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 89ade2ed7ce4..35409695d3d7 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -118,6 +118,7 @@
 #define      MVNETA_HSGMII_SERDES_PROTO		 0x1107
 #define MVNETA_TYPE_PRIO                         0x24bc
 #define      MVNETA_FORCE_UNI                    BIT(21)
+#define MVETH_TXQ_FIXED_PRIO_REG                 0x24dc
 #define MVNETA_TXQ_CMD_1                         0x24e4
 #define MVNETA_TXQ_CMD                           0x2448
 #define      MVNETA_TXQ_DISABLE_SHIFT            8
@@ -196,6 +197,7 @@
 #define      MVNETA_RXQ_ENABLE_MASK              0x000000ff
 #define MVETH_TXQ_TOKEN_COUNT_REG(q)             (0x2700 + ((q) << 4))
 #define MVETH_TXQ_TOKEN_CFG_REG(q)               (0x2704 + ((q) << 4))
+#define MVETH_TXQ_WRR_ARB_REG(q)                 (0x2708 + ((q) << 4))
 #define MVNETA_GMAC_CTRL_0                       0x2c00
 #define      MVNETA_GMAC_MAX_RX_SIZE_SHIFT       2
 #define      MVNETA_GMAC_MAX_RX_SIZE_MASK        0x7ffc
@@ -253,6 +255,8 @@
 #define MVNETA_TXQ_CMD1_REG			 0x3e00
 #define      MVNETA_TXQ_CMD1_BW_LIM_SEL_V1	 BIT(3)
 #define      MVNETA_TXQ_CMD1_BW_LIM_EN		 BIT(0)
+#define MVNETA_TXQ_FIXED_PRIO_REG		 0x3e04
+#define      MVNETA_TXQ_FIXED_PRIO_ALL		 0xff
 #define MVNETA_REFILL_NUM_CLK_REG		 0x3e08
 #define      MVNETA_REFILL_MAX_NUM_CLK		 0x0000ffff
 #define MVNETA_TX_MTU                            0x3e0c
@@ -264,6 +268,8 @@
 #define      MVNETA_TXQ_BUCKET_REFILL_VALUE_MAX	 0x0007ffff
 #define MVNETA_TXQ_TOKEN_SIZE_REG(q)             (0x3e40 + ((q) << 2))
 #define      MVNETA_TXQ_TOKEN_SIZE_MAX           0x7fffffff
+#define MVNETA_TXQ_WRR_ARB_REG(q)		 (0x3e80 + ((q) << 2))
+#define      MVNETA_TXQ_WRR_WEIGHT_MASK		 0xff
 
 /* The values of the bucket refill base period and refill period are taken from
  * the reference manual, and adds up to a base resolution of 10Kbps. This allows
@@ -518,6 +524,8 @@ struct mvneta_port {
 	 */
 	spinlock_t lock;
 	bool is_stopped;
+	/* An mqprio offload keeps the TX queues in fixed priority */
+	bool txq_fixed_prio;
 
 	u32 cause_rx_tx;
 	struct napi_struct napi;
@@ -1632,6 +1640,39 @@ static void mvneta_txq_max_tx_size_set(struct mvneta_port *pp, int max_tx_size)
 	}
 }
 
+static void mvneta_txq_wrr_weight(struct mvneta_port *pp, u32 reg, u32 weight)
+{
+	u32 val = mvreg_read(pp, reg);
+
+	val &= ~MVNETA_TXQ_WRR_WEIGHT_MASK;
+	mvreg_write(pp, reg, val | weight);
+}
+
+/* With the default fixed priority, a busy queue starves the others.
+ * The Armada 370 and XP manuals list separate registers for the
+ * version 1 arbiter, which runs without a shaper, so set both.
+ */
+static void mvneta_txq_wrr_set(struct mvneta_port *pp, int max_tx_size)
+{
+	/* In units of 256 bytes, must exceed the largest packet */
+	u32 weight = max_tx_size / 256 + 1;
+	int queue;
+
+	for (queue = 0; queue < txq_number; queue++) {
+		mvneta_txq_wrr_weight(pp, MVNETA_TXQ_WRR_ARB_REG(queue), weight);
+		mvneta_txq_wrr_weight(pp, MVETH_TXQ_WRR_ARB_REG(queue), weight);
+	}
+}
+
+/* Only the traffic classes of an mqprio offload keep the fixed priority */
+static void mvneta_txq_fixed_prio_set(struct mvneta_port *pp)
+{
+	u32 val = pp->txq_fixed_prio ? MVNETA_TXQ_FIXED_PRIO_ALL : 0;
+
+	mvreg_write(pp, MVNETA_TXQ_FIXED_PRIO_REG, val);
+	mvreg_write(pp, MVETH_TXQ_FIXED_PRIO_REG, val);
+}
+
 /* Set unicast address */
 static void mvneta_set_ucast_addr(struct mvneta_port *pp, u8 last_nibble,
 				  int queue)
@@ -3765,6 +3806,8 @@ static void mvneta_start_dev(struct mvneta_port *pp)
 
 	mvneta_max_rx_size_set(pp, pp->pkt_size);
 	mvneta_txq_max_tx_size_set(pp, pp->pkt_size);
+	mvneta_txq_wrr_set(pp, pp->pkt_size);
+	mvneta_txq_fixed_prio_set(pp);
 	mvneta_restore_per_queue_rate_limit(pp);
 
 	/* start the Rx/Tx activity */
@@ -5345,9 +5388,20 @@ static int mvneta_setup_mqprio(struct net_device *dev,
 static int mvneta_setup_tc(struct net_device *dev, enum tc_setup_type type,
 			   void *type_data)
 {
+	struct mvneta_port *pp = netdev_priv(dev);
+	struct tc_mqprio_qopt_offload *mqprio;
+	int ret;
+
 	switch (type) {
 	case TC_SETUP_QDISC_MQPRIO:
-		return mvneta_setup_mqprio(dev, type_data);
+		mqprio = type_data;
+		ret = mvneta_setup_mqprio(dev, mqprio);
+		if (ret)
+			return ret;
+
+		pp->txq_fixed_prio = mqprio->qopt.num_tc;
+		mvneta_txq_fixed_prio_set(pp);
+		return 0;
 	default:
 		return -EOPNOTSUPP;
 	}
-- 
2.54.0 (Apple Git-157)


^ permalink raw reply related	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-11-18  9:30 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18  9:30 [PATCH net-next v4 0/6] net: mvneta: fix the TX queue arbitration and the mqprio offload Josef Schlehofer
2026-11-18  9:30 ` [PATCH net-next v4 1/6] net: mvneta: lift the per-queue TX rate limits Josef Schlehofer
2026-11-18  9:30 ` [PATCH net-next v4 2/6] net: mvneta: fix the removal of the mqprio offload Josef Schlehofer
2026-11-18  9:30 ` [PATCH net-next v4 3/6] net: mvneta: check the TX queue range of an " Josef Schlehofer
2026-11-18  9:30 ` [PATCH net-next v4 4/6] net: mvneta: keep the TX rate limits across open and MTU change Josef Schlehofer
2026-11-18  9:30 ` [PATCH net-next v4 5/6] net: mvneta: validate an mqprio setup before applying it Josef Schlehofer
2026-11-18  9:30 ` [PATCH net-next v4 6/6] net: mvneta: serve the TX queues in weighted round robin Josef Schlehofer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox