[PATCH] app/testpmd: warn on shared rxq switch mismatch

[PATCH] app/testpmd: warn on shared rxq switch mismatch

[a.schollmeyer]

From: Adrian Schollmeyer <a.schollmeyer@syseleven.de>

Shared Rx queues do not work with every combination of ports. Besides
requiring the corresponding device capability, shared Rx queues also
require all ports of one share group and queue ID have the same switch
domain and Rx domain. When these fields do not match, shared Rx queues
are not properly set up and queue sharing may fail silently. This can
happen even in some less intuitive cases like multiple VFs of one
physical NIC.

To make debugging issues with shared Rx queue configuration easier, this
commit introduces simple checks and warning messages for all members of
a share_group and share_qid to warn whenever there is a mismatch in the
switch and Rx domain.

Signed-off-by: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
---
 app/test-pmd/testpmd.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 1fe41d852a..6e42930730 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2675,6 +2675,48 @@ rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 	uint32_t prev_hdrs = 0;
 	int ret;
 
+	if (rx_conf->share_group > 0) {
+		/* Additional warnings for bad Rx queue sharing */
+
+		const uint16_t dom_id = ports[port_id].dev_info.switch_info.domain_id;
+		const uint16_t rx_dom = ports[port_id].dev_info.switch_info.rx_domain;
+
+		uint16_t pid;
+		RTE_ETH_FOREACH_DEV(pid) {
+			struct rte_port *o_port = &ports[pid];
+			for (uint16_t q = 0; q < nb_rxq; ++q) {
+				struct port_rxqueue *rxq = &o_port->rxq[q];
+				if (rxq->conf.share_group != rx_conf->share_group ||
+						rxq->conf.share_qid != rx_conf->share_qid)
+					continue;
+
+				const uint16_t o_dom_id = o_port->dev_info.switch_info.domain_id;
+				const uint16_t o_rx_dom = o_port->dev_info.switch_info.rx_domain;
+
+				if (o_dom_id != dom_id) {
+					fprintf(stderr, "Bad shared rxq config: "
+						"switch domain mismatch in share group/qid %u/%u: "
+						"%u (port %u) vs. %u (port %u)\n",
+						(unsigned int)rxq->conf.share_group,
+						(unsigned int)rxq->conf.share_qid,
+						(unsigned int)dom_id,
+						(unsigned int)port_id,
+						(unsigned int)o_dom_id,
+						(unsigned int)pid);
+				} else if (o_rx_dom != rx_dom) {
+					fprintf(stderr, "Bad shared rxq config: "
+						"switch rx domain mismatch in share group/qid %u/%u: "
+						"%u (port %u) vs. %u (port %u)\n",
+						(unsigned int)rxq->conf.share_group,
+						(unsigned int)rxq->conf.share_qid,
+						(unsigned int)rx_dom,
+						(unsigned int)port_id,
+						(unsigned int)o_rx_dom,
+						(unsigned int)pid);
+				}
+			}
+		}
+	}
 
 	if ((rx_pkt_nb_segs > 1) &&
 	    (rx_conf->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
-- 
2.34.1

Re: [PATCH] app/testpmd: warn on shared rxq switch mismatch

[Stephen Hemminger]

On Tue, 25 Nov 2025 11:22:07 +0100
a.schollmeyer@syseleven.de wrote:

> +				if (o_dom_id != dom_id) {
> +					fprintf(stderr, "Bad shared rxq config: "
> +						"switch domain mismatch in share group/qid %u/%u: "
> +						"%u (port %u) vs. %u (port %u)\n",
> +						(unsigned int)rxq->conf.share_group,
> +						(unsigned int)rxq->conf.share_qid,
> +						(unsigned int)dom_id,
> +						(unsigned int)port_id,
> +						(unsigned int)o_dom_id,
> +						(unsigned int)pid);
> +				} else if (o_rx_dom != rx_dom) {
> +					fprintf(stderr, "Bad shared rxq config: "
> +						"switch rx domain mismatch in share group/qid %u/%u: "
> +						"%u (port %u) vs. %u (port %u)\n",
> +						(unsigned int)rxq->conf.share_group,
> +						(unsigned int)rxq->conf.share_qid,
> +						(unsigned int)rx_dom,
> +						(unsigned int)port_id,
> +						(unsigned int)o_rx_dom,
> +						(unsigned int)pid);
> +				}

Same message repeated twice, change the if logic.
Also, why so many casts? Those should all be unsigned.

[PATCH v2] app/testpmd: warn on shared rxq switch mismatch

[a.schollmeyer]

From: Adrian Schollmeyer <a.schollmeyer@syseleven.de>

Shared Rx queues do not work with every combination of ports. Besides
requiring the corresponding device capability, shared Rx queues also
require all ports of one share group and queue ID have the same switch
domain and Rx domain. When these fields do not match, shared Rx queues
are not properly set up and queue sharing may fail silently. This can
happen even in some less intuitive cases like multiple VFs of one
physical NIC.

To help with debugging issues with shared Rx queue configuration when
using testpmd, this commit introduces simple checks and warning messages
for all members of a share_group and share_qid to warn whenever there is
a mismatch in the switch and Rx domain.

Signed-off-by: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
---
v2 changes:

  * remove unnecessary casts
  * refactor message printing logic to remove duplicate fprintf()

 app/test-pmd/testpmd.c | 46 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 1fe41d852a..33b99756f7 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2675,6 +2675,52 @@ rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 	uint32_t prev_hdrs = 0;
 	int ret;
 
+	if (rx_conf->share_group > 0) {
+		/* Additional warnings for bad Rx queue sharing */
+
+		const uint16_t dom_id = ports[port_id].dev_info.switch_info.domain_id;
+		const uint16_t rx_dom = ports[port_id].dev_info.switch_info.rx_domain;
+
+		uint16_t pid;
+		RTE_ETH_FOREACH_DEV(pid) {
+			struct rte_port *o_port = &ports[pid];
+			for (uint16_t q = 0; q < nb_rxq; ++q) {
+				struct port_rxqueue *rxq = &o_port->rxq[q];
+				if (rxq->conf.share_group != rx_conf->share_group ||
+						rxq->conf.share_qid != rx_conf->share_qid)
+					continue;
+
+				const uint16_t o_dom_id = o_port->dev_info.switch_info.domain_id;
+				const uint16_t o_rx_dom = o_port->dev_info.switch_info.rx_domain;
+				const char *mismatch = NULL;
+				uint16_t this_val = 0;
+				uint16_t other_val = 0;
+
+				if (o_dom_id != dom_id) {
+					mismatch = "switch domain";
+					this_val = dom_id;
+					other_val = o_dom_id;
+				} else if (o_rx_dom != rx_dom) {
+					mismatch = "rx domain";
+					this_val = rx_dom;
+					other_val = o_rx_dom;
+				}
+
+				if (mismatch) {
+					fprintf(stderr, "Bad shared rxq config: "
+						"%s mismatch in share group/qid %u/%u: "
+						"%u (port %u) vs. %u (port %u)\n",
+						mismatch,
+						rxq->conf.share_group,
+						rxq->conf.share_qid,
+						this_val,
+						port_id,
+						other_val,
+						pid);
+				}
+			}
+		}
+	}
 
 	if ((rx_pkt_nb_segs > 1) &&
 	    (rx_conf->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
-- 
2.34.1

Re: [PATCH v2] app/testpmd: warn on shared rxq switch mismatch

[Stephen Hemminger]

On Wed, 26 Nov 2025 11:40:09 +0100
a.schollmeyer@syseleven.de wrote:

> +				if (mismatch) {
> +					fprintf(stderr, "Bad shared rxq config: "
> +						"%s mismatch in share group/qid %u/%u: "

It is better to not split formats across lines.
Can you shorten it, and put on oneline.
Despite what checkpatch may say, you can make that a longish line like 200 characters.
Also, maybe break from the loop on first mismatch? and return an error?
Then message can be outside the loop.

[PATCH v3] app/testpmd: fail on shared rxq switch mismatch

[a.schollmeyer]

From: Adrian Schollmeyer <a.schollmeyer@syseleven.de>

Shared Rx queues do not work with every combination of ports. Besides
requiring the corresponding device capability, shared Rx queues also
require all ports of one share group and queue ID have the same switch
domain and Rx domain. When these fields do not match, shared Rx queues
are not properly set up and queue sharing may fail silently. This can
happen even in some less intuitive cases like multiple VFs of one
physical NIC and may cause packets not to be read by the application,
depending on how the data path is implemented.

To help with debugging issues with shared Rx queue configuration when
using testpmd, this commit introduces simple checks and error messages
for members of a share_group and share_qid to fail whenever there is a
mismatch in the switch and Rx domain.

Signed-off-by: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
---
v3 changes:

  * make switch/rx domain mismatch an error with shared Rx queues
  * shorten error message
  * move error message outside of the loops
  * update commit message to reflect warning -> failure

v2 changes:

  * remove unnecessary casts
  * refactor message printing logic to remove duplicate fprintf()

 app/test-pmd/testpmd.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 1fe41d852a..6464a97b9b 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2675,6 +2675,47 @@ rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 	uint32_t prev_hdrs = 0;
 	int ret;
 
+	if (rx_conf->share_group > 0) {
+		/* Check required switch info for Rx queue sharing */
+
+		const uint16_t dom_id = ports[port_id].dev_info.switch_info.domain_id;
+		const uint16_t rx_dom = ports[port_id].dev_info.switch_info.rx_domain;
+
+		uint16_t pid;
+		const char *mismatch = NULL;
+		uint16_t mismatch_pid = (uint16_t)RTE_PORT_ALL;
+		RTE_ETH_FOREACH_DEV(pid) {
+			struct rte_port *o_port = &ports[pid];
+			const uint16_t o_dom_id = o_port->dev_info.switch_info.domain_id;
+			const uint16_t o_rx_dom = o_port->dev_info.switch_info.rx_domain;
+
+			for (uint16_t q = 0; q < nb_rxq; ++q) {
+				struct port_rxqueue *rxq = &o_port->rxq[q];
+				if (rxq->conf.share_group != rx_conf->share_group ||
+						rxq->conf.share_qid != rx_conf->share_qid)
+					continue;
+				if (o_dom_id == dom_id && o_rx_dom == rx_dom)
+					continue;
+
+				if (o_dom_id != dom_id)
+					mismatch = "switch domain";
+				else if (o_rx_dom != rx_dom)
+					mismatch = "rx domain";
+
+				mismatch_pid = pid;
+				break;
+			}
+		}
+
+		if (mismatch) {
+			fprintf(stderr,
+				"Invalid shared rxq config: %s mismatch between ports %u and %u\n",
+				mismatch,
+				port_id,
+				mismatch_pid);
+			return -EINVAL;
+		}
+	}
 
 	if ((rx_pkt_nb_segs > 1) &&
 	    (rx_conf->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
-- 
2.34.1

Re: [PATCH v3] app/testpmd: fail on shared rxq switch mismatch

[Stephen Hemminger]

On Mon,  1 Dec 2025 11:22:11 +0100
a.schollmeyer@syseleven.de wrote:

> From: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
> 
> Shared Rx queues do not work with every combination of ports. Besides
> requiring the corresponding device capability, shared Rx queues also
> require all ports of one share group and queue ID have the same switch
> domain and Rx domain. When these fields do not match, shared Rx queues
> are not properly set up and queue sharing may fail silently. This can
> happen even in some less intuitive cases like multiple VFs of one
> physical NIC and may cause packets not to be read by the application,
> depending on how the data path is implemented.
> 
> To help with debugging issues with shared Rx queue configuration when
> using testpmd, this commit introduces simple checks and error messages
> for members of a share_group and share_qid to fail whenever there is a
> mismatch in the switch and Rx domain.
> 
> Signed-off-by: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
> ---

Queued to next-net for 26.03

Re: [PATCH v3] app/testpmd: fail on shared rxq switch mismatch

[Dariusz Sosnowski]

Hi all,

On Mon, Dec 01, 2025 at 11:22:11AM +0100, a.schollmeyer@syseleven.de wrote:
> From: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
> 
> Shared Rx queues do not work with every combination of ports. Besides
> requiring the corresponding device capability, shared Rx queues also
> require all ports of one share group and queue ID have the same switch
> domain and Rx domain. When these fields do not match, shared Rx queues
> are not properly set up and queue sharing may fail silently. This can
> happen even in some less intuitive cases like multiple VFs of one
> physical NIC and may cause packets not to be read by the application,
> depending on how the data path is implemented.
> 
> To help with debugging issues with shared Rx queue configuration when
> using testpmd, this commit introduces simple checks and error messages
> for members of a share_group and share_qid to fail whenever there is a
> mismatch in the switch and Rx domain.
> 
> Signed-off-by: Adrian Schollmeyer <a.schollmeyer@syseleven.de>

Our internal regression tests have flagged issues with shared Rx queues.
Specifically, issues with domain mismatches:

	Invalid shared rxq config: switch domain mismatch ports 0 and 3

In my opinion, additional testpmd fixes for shared Rx queues or a revert is needed.
Please find the details below and let me know what you all think.

Best regards,
Dariusz Sosnowski

---

https://github.com/DPDK/dpdk/commit/8ebba91086f47c90e398d7775921e05659c0d62f

**ethdev and mlx5 driver behavior**

In DPDK API, to enable Rx queue sharing,
the application must provide 2 parameters in rte_eth_rxconf struct:

- share_group
- share_qid

2 Rx queues can be shared if and only if these queues are created on
ports which share both switch and Rx domain.
In mlx5 NICs case this means that queues can be shared only between
representors for a single embedded switch.

Both ethdev library and mlx5 driver do not explicitly require that
share group index is globally unique.
The same value of share_group index can be used in different domains.
For example, the following is allowed:

- port 1, switch/Rx domain 1, share_group index == 1
- port 2, switch/Rx domain 2, share_group index == 1

In this case, both share groups are considered unique and separate
because they are set up in different domains.

**testpmd**

--rxq-share parameter can be used in 2 ways:

- --rxq-share=N - Set of ports is divided into subsets, each with N
  ports. Each subset gets new share group index.
  Index is assigned incrementally based on port index.
- --rxq-share - Same as N=UINT32_MAX. In practice this means
  that all ports get the same share group index equal to 1.

In our internal testing, where combination of sharing Rx queues
and dynamic representor attachment was tested,
"--rxq-share" parameter was used.
These tests relied on assigning the same group index to each shared queue.
When new representors were probed, their queues were added to existing
share groups.

After this patch was applied it is no longer possible,
regardless of the --rxq-share option variant used:

- Without additional N parameter - probing 2 different PFs and
  configuring shared queues failes on mismatch.
- With additional N parameter - queues of dynamically added representors
  cannot be added to existing share groups.

**Possible solutions**

For now, I see 2 possible ways to resolve it:

1. testpmd assigns unique share group index per domain:
    - Behavior change for --rxq-share parameter.
    - Instead of statically assigning indexes based on port index,
      testpmd will internally allocate new share group index
      per unique switch and Rx domain.
    - Drop additional parameter N for --rxq-share.
2. Expand testpmd parameters for shared Rx queues:
    - Allow explicit assignment of share group index through additional
      parameters.
    - Right now, it's unclear how to handle dynamic port attachment in
      this solution.

Re: [PATCH] app/testpmd: warn on shared rxq switch mismatch

[Stephen Hemminger]

On Tue, 25 Nov 2025 11:22:07 +0100
a.schollmeyer@syseleven.de wrote:

> From: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
> 
> Shared Rx queues do not work with every combination of ports. Besides
> requiring the corresponding device capability, shared Rx queues also
> require all ports of one share group and queue ID have the same switch
> domain and Rx domain. When these fields do not match, shared Rx queues
> are not properly set up and queue sharing may fail silently. This can
> happen even in some less intuitive cases like multiple VFs of one
> physical NIC.
> 
> To make debugging issues with shared Rx queue configuration easier, this
> commit introduces simple checks and warning messages for all members of
> a share_group and share_qid to warn whenever there is a mismatch in the
> switch and Rx domain.
> 
> Signed-off-by: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
> ---

Why is this not handled at ethdev or device level?

Testpmd is just a test program. Real applications might make same mistake.

RE: [PATCH] app/testpmd: warn on shared rxq switch mismatch

[Dariusz Sosnowski]

> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Monday, March 23, 2026 5:17 PM
> To: a.schollmeyer@syseleven.de
> Cc: Aman Singh <aman.deep.singh@intel.com>; dev@dpdk.org
> Subject: Re: [PATCH] app/testpmd: warn on shared rxq switch mismatch
> 
> On Tue, 25 Nov 2025 11:22:07 +0100
> a.schollmeyer@syseleven.de wrote:
> 
> > From: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
> >
> > Shared Rx queues do not work with every combination of ports. Besides
> > requiring the corresponding device capability, shared Rx queues also
> > require all ports of one share group and queue ID have the same switch
> > domain and Rx domain. When these fields do not match, shared Rx queues
> > are not properly set up and queue sharing may fail silently. This can
> > happen even in some less intuitive cases like multiple VFs of one
> > physical NIC.

Adrian: Could you please elaborate what kind of issues have you faced with shared queue configuration?

> >
> > To make debugging issues with shared Rx queue configuration easier,
> > this commit introduces simple checks and warning messages for all
> > members of a share_group and share_qid to warn whenever there is a
> > mismatch in the switch and Rx domain.
> >
> > Signed-off-by: Adrian Schollmeyer <a.schollmeyer@syseleven.de>
> > ---
> 
> Why is this not handled at ethdev or device level?

Are you referring to share group index logic?

Based on current implementation in ethdev and mlx5 PMD,
assuring that share group indexes are unique is not needed.
Share group index is not required to be globally unique since
sharing works only within the same switch and Rx domain [1].

> 
> Testpmd is just a test program. Real applications might make same mistake.

Best regards,
Dariusz Sosnowski

---

[1]: https://inbox.dpdk.org/dev/yotjxacqrodttraqrr3r6ftut4cty66g6cjnr5ughswtatapgh@gqqkftskp3qq/

Re: [PATCH] app/testpmd: warn on shared rxq switch mismatch

[Adrian Schollmeyer]

[-- Attachment #1: Type: text/plain, Size: 3081 bytes --]

Hi,

Am Dienstag, dem 24.03.2026 um 09:04 +0000 schrieb Dariusz Sosnowski:
> 
> Adrian: Could you please elaborate what kind of issues have you faced
> with shared queue configuration?

For one, I find the documentation around shared Rx queues quite poor to
begin with. Back then, I tried to make it work on a regular ConnectX-6
NIC with multiple VFs. For lack of better understanding, I just tried
to run testpmd to check whether shared Rx queues works with this kind
of setup. Unfortunately, testpmd neither emitted a warning/error
message nor was able to set up the queues properly. Instead, everything
failed silently. It may very well have been me just using the feature
wrong, but testpmd did nothing to help me debug the issue.

Unfortunately, I am not working on this topic anymore, so I have no
recent experience with similar issues which I could provide. Also, as I
did this quite some time ago, I cannot remember all the details.

> > 
> > 
> Based on current implementation in ethdev and mlx5 PMD,
> assuring that share group indexes are unique is not needed.
> Share group index is not required to be globally unique since
> sharing works only within the same switch and Rx domain [1].

So if I understand this correctly, it is correct behavior for DPDK to
silently accept the same share_group and qid for ports on different
domains?

From a user perspective, I see two issues here:

* The rx_domain and domain_id values are obscure magic values, whose
  meaning is neither obvious nor very well documented. Therefore, in my
  opinion, users cannot be expected to "just know" what they imply and
  and where they might be different.

  My personal assumption was that they probably differ from NIC to
  NIC or possibly from PF to PF, but from what I learned while
  experimenting with that ConnectX-6 NIC, this doesn't necessarily
  seem to be the case.

* Using the same share_group for two Rx queues intuitively means that
  they are shared. Since this feature is used to combine Rx paths
  across ports, I would expect that the identifiers are valid across
  ports, not switch domains or Rx domains. In my opinion, this is
  highly unintuitive from a user's perspective and I would expect DPDK 
  to assume that the same share group ID actually means trying to use
  the same share group, no matter which ports are combined.

Therefore, I would prefer requiring uniqueness of share group IDs
across all ports. In any case, the shared Rx queue feature could
probably benefit from improved documentation.

Best regards
Adrian


-- 
Adrian Schollmeyer

SysEleven GmbH 
Boxhagener Straße 80 
10245 Berlin 

T +49 30 / 23 32 012-0
F +49 30 / 61 67 55 5-0 

https://www.syseleven.de 
https://www.linkedin.com/company/syseleven-gmbh

Current system status always at: 
https://www.syseleven-status.net/ 

Company headquarters: Berlin 
Registered court: AG Berlin Charlottenburg, HRB 108571 Berlin 
Managing directors: Andreas Hermann, Jens Ihlenfeld, Jens Plogsties,
Andreas Rückriegel



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]