[PATCH 0/6] net/idpf: fix split queue AVX2 datapath

[PATCH 0/6] net/idpf: fix split queue AVX2 datapath

[Shaiq Wani]

Fix the split queue AVX2 vectorized path to correctly handle the
virtchnl2_rx_flex_desc_adv_nic_3 completion descriptor format and
the flex scheduled Tx descriptor encoding.

The split queue completion descriptor layout is significantly
different from the single queue path — the generation bit, DD bit,
and packet length occupy different positions than the base Rx
descriptor, and the buffer queue owns the mbuf pool rather than the
completion queue.

On Rx, four issues are fixed:
  - Generation bit extraction order (mask-before-shift)
  - DD bit byte offset (byte 8 not byte 1)
  - mbuf initializer sourced from bufq2 instead of the completion queue
  - Packet type inserted into the correct AVX2 lane

On Tx, two issues are fixed:
  - Buffer size shift corrected from 34 to 48 for flex descriptors
  - Burst clamped to tx_rs_thresh with proper tx_next_rs tracking

Shaiq Wani (6):
  net/idpf: fix gen bit extraction in split queue AVX2 Rx
  net/idpf: fix DD bit byte offset in split queue AVX2 Rx
  net/idpf: fix mbuf initializer source in split queue AVX2 Rx
  net/idpf: fix ptype insert position in split queue AVX2 Rx
  net/idpf: fix split queue AVX2 Tx buffer size shift
  net/idpf: fix split queue AVX2 Tx burst and completion

 .../net/intel/idpf/idpf_common_rxtx_avx2.c    | 72 +++++++++++--------
 1 file changed, 43 insertions(+), 29 deletions(-)

-- 
2.34.1

[PATCH 1/6] net/idpf: fix gen bit extraction in split queue AVX2 Rx

[Shaiq Wani]

The generation bit in the pktlen_gen_bufq_id field of the split queue
completion descriptor (virtchnl2_rx_flex_desc_adv_nic_3) must be
extracted by masking first and then shifting, not the other way around.

With shift-then-mask, the mask is applied to already-shifted bits,
which can produce incorrect results when upper bits (packet length,
buffer queue ID) leak into the extracted value.

Change to mask-then-shift to correctly isolate the generation bit
before comparing it with the expected generation ID.

Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
---
 .../net/intel/idpf/idpf_common_rxtx_avx2.c    | 20 +++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
index db7728afad..cd10c27a30 100644
--- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
+++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
@@ -524,8 +524,8 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
 
 	/* check if there is at least one packet available */
 	head_gen = rxdp->flex_adv_nic_3_wb.pktlen_gen_bufq_id;
-	if (((head_gen >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
-		 VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) != queue->expected_gen_id)
+	if (((head_gen & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+		 VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) != queue->expected_gen_id)
 		return 0;
 
 	for (i = 0; i < nb_pkts;
@@ -599,17 +599,17 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
 		pktlen_gen3 = (uint16_t)_mm_extract_epi16(d3, 2);
 
 		valid0 = (stat0 & 1) &&
-			 (((pktlen_gen0 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
-			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
+			 (((pktlen_gen0 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
 		valid1 = (stat1 & 1) &&
-			 (((pktlen_gen1 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
-			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
+			 (((pktlen_gen1 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
 		valid2 = (stat2 & 1) &&
-			 (((pktlen_gen2 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
-			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
+			 (((pktlen_gen2 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
 		valid3 = (stat3 & 1) &&
-			 (((pktlen_gen3 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
-			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
+			 (((pktlen_gen3 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
+			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
 
 		/* count valid descriptors (holes are impossible because
 		 * descriptors are read in reverse order while the NIC
-- 
2.34.1

Re: [PATCH 1/6] net/idpf: fix gen bit extraction in split queue AVX2 Rx

[Bruce Richardson]

On Mon, May 11, 2026 at 02:39:30PM +0530, Shaiq Wani wrote:
> The generation bit in the pktlen_gen_bufq_id field of the split queue
> completion descriptor (virtchnl2_rx_flex_desc_adv_nic_3) must be
> extracted by masking first and then shifting, not the other way around.
> 
> With shift-then-mask, the mask is applied to already-shifted bits,
> which can produce incorrect results when upper bits (packet length,
> buffer queue ID) leak into the extracted value.
> 
> Change to mask-then-shift to correctly isolate the generation bit
> before comparing it with the expected generation ID.
> 
> Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
> Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  .../net/intel/idpf/idpf_common_rxtx_avx2.c    | 20 +++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> index db7728afad..cd10c27a30 100644
> --- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> +++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> @@ -524,8 +524,8 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
>  
>  	/* check if there is at least one packet available */
>  	head_gen = rxdp->flex_adv_nic_3_wb.pktlen_gen_bufq_id;
> -	if (((head_gen >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
> -		 VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) != queue->expected_gen_id)
> +	if (((head_gen & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
> +		 VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) != queue->expected_gen_id)
>  		return 0;
>  
>  	for (i = 0; i < nb_pkts;
> @@ -599,17 +599,17 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
>  		pktlen_gen3 = (uint16_t)_mm_extract_epi16(d3, 2);
>  
>  		valid0 = (stat0 & 1) &&
> -			 (((pktlen_gen0 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
> -			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
> +			 (((pktlen_gen0 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
> +			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
>  		valid1 = (stat1 & 1) &&
> -			 (((pktlen_gen1 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
> -			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
> +			 (((pktlen_gen1 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
> +			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
>  		valid2 = (stat2 & 1) &&
> -			 (((pktlen_gen2 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
> -			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
> +			 (((pktlen_gen2 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
> +			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
>  		valid3 = (stat3 & 1) &&
> -			 (((pktlen_gen3 >> VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) &
> -			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) == queue->expected_gen_id);
> +			 (((pktlen_gen3 & VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_M) >>
> +			   VIRTCHNL2_RX_FLEX_DESC_ADV_GEN_S) == queue->expected_gen_id);
>  
>  		/* count valid descriptors (holes are impossible because
>  		 * descriptors are read in reverse order while the NIC
> -- 
> 2.34.1
> 

[PATCH 2/6] net/idpf: fix DD bit byte offset in split queue AVX2 Rx

[Shaiq Wani]

The split queue completion descriptor (virtchnl2_rx_flex_desc_adv_nic_3)
has two distinct status fields: status_err0_qw0 at byte offset 1 and
status_err0_qw1 at byte offset 8.  The DD (descriptor done) bit lives
in status_err0_qw1 (byte 8), not status_err0_qw0 (byte 1).

Byte 1 (status_err0_qw0) bit 0 is the LPBK (loopback) indicator, so
reading DD from byte 1 checks the wrong field entirely.

Fix the _mm_extract_epi8 index from 1 to 8 so the code reads the DD
bit from its correct location in the writeback descriptor.

Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
---
 drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
index cd10c27a30..28d4246134 100644
--- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
+++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
@@ -587,11 +587,11 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
 		_mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, rearm2);
 		_mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, rearm3);
 
-		/* Extract DD and generation bits from the already-loaded descriptor data (d0-d3) */
-		stat0 = (uint8_t)_mm_extract_epi8(d0, 1);
-		stat1 = (uint8_t)_mm_extract_epi8(d1, 1);
-		stat2 = (uint8_t)_mm_extract_epi8(d2, 1);
-		stat3 = (uint8_t)_mm_extract_epi8(d3, 1);
+		/* Extract DD bit from status_err0_qw1 (byte 8 of descriptor) */
+		stat0 = (uint8_t)_mm_extract_epi8(d0, 8);
+		stat1 = (uint8_t)_mm_extract_epi8(d1, 8);
+		stat2 = (uint8_t)_mm_extract_epi8(d2, 8);
+		stat3 = (uint8_t)_mm_extract_epi8(d3, 8);
 
 		pktlen_gen0 = (uint16_t)_mm_extract_epi16(d0, 2);
 		pktlen_gen1 = (uint16_t)_mm_extract_epi16(d1, 2);
-- 
2.34.1

Re: [PATCH 2/6] net/idpf: fix DD bit byte offset in split queue AVX2 Rx

[Bruce Richardson]

On Mon, May 11, 2026 at 02:39:31PM +0530, Shaiq Wani wrote:
> The split queue completion descriptor (virtchnl2_rx_flex_desc_adv_nic_3)
> has two distinct status fields: status_err0_qw0 at byte offset 1 and
> status_err0_qw1 at byte offset 8.  The DD (descriptor done) bit lives
> in status_err0_qw1 (byte 8), not status_err0_qw0 (byte 1).
> 
> Byte 1 (status_err0_qw0) bit 0 is the LPBK (loopback) indicator, so
> reading DD from byte 1 checks the wrong field entirely.
> 
> Fix the _mm_extract_epi8 index from 1 to 8 so the code reads the DD
> bit from its correct location in the writeback descriptor.
> 
> Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
> Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> index cd10c27a30..28d4246134 100644
> --- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> +++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> @@ -587,11 +587,11 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
>  		_mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, rearm2);
>  		_mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, rearm3);
>  
> -		/* Extract DD and generation bits from the already-loaded descriptor data (d0-d3) */
> -		stat0 = (uint8_t)_mm_extract_epi8(d0, 1);
> -		stat1 = (uint8_t)_mm_extract_epi8(d1, 1);
> -		stat2 = (uint8_t)_mm_extract_epi8(d2, 1);
> -		stat3 = (uint8_t)_mm_extract_epi8(d3, 1);
> +		/* Extract DD bit from status_err0_qw1 (byte 8 of descriptor) */
> +		stat0 = (uint8_t)_mm_extract_epi8(d0, 8);
> +		stat1 = (uint8_t)_mm_extract_epi8(d1, 8);
> +		stat2 = (uint8_t)_mm_extract_epi8(d2, 8);
> +		stat3 = (uint8_t)_mm_extract_epi8(d3, 8);
>  
>  		pktlen_gen0 = (uint16_t)_mm_extract_epi16(d0, 2);
>  		pktlen_gen1 = (uint16_t)_mm_extract_epi16(d1, 2);
> -- 
> 2.34.1
> 

[PATCH 3/6] net/idpf: fix mbuf initializer source in split queue AVX2 Rx

[Shaiq Wani]

In split queue mode the completion queue (rxq) does not own the mbuf
pool — the buffer queue (bufq2) does.  The mbuf initializer encodes
the mempool pointer, refcount and other per-pool mbuf metadata that
is stamped into every received mbuf during rearm.

Using queue->mbuf initializer reads an uninitialised or zero value
from the completion queue, corrupting every mbuf rearm.  Use
queue->bufq2->mbuf initializer to get the correct value from the
buffer queue that actually owns the mbufs.

Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
---
 drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
index 28d4246134..d3a8e17778 100644
--- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
+++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
@@ -491,7 +491,7 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
 	struct rte_mbuf **sw_ring = &queue->bufq2->sw_ring[queue->rx_tail];
 	volatile union virtchnl2_rx_desc *rxdp =
 		(volatile union virtchnl2_rx_desc *)queue->rx_ring + queue->rx_tail;
-	const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0, queue->mbuf_initializer);
+	const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0, queue->bufq2->mbuf_initializer);
 	uint64_t head_gen;
 	uint16_t received = 0;
 	int i;
-- 
2.34.1

Re: [PATCH 3/6] net/idpf: fix mbuf initializer source in split queue AVX2 Rx

[Bruce Richardson]

On Mon, May 11, 2026 at 02:39:32PM +0530, Shaiq Wani wrote:
> In split queue mode the completion queue (rxq) does not own the mbuf
> pool — the buffer queue (bufq2) does.  The mbuf initializer encodes
> the mempool pointer, refcount and other per-pool mbuf metadata that
> is stamped into every received mbuf during rearm.
> 
> Using queue->mbuf initializer reads an uninitialised or zero value
> from the completion queue, corrupting every mbuf rearm.  Use
> queue->bufq2->mbuf initializer to get the correct value from the
> buffer queue that actually owns the mbufs.
> 
> Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
> Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> index 28d4246134..d3a8e17778 100644
> --- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> +++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> @@ -491,7 +491,7 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
>  	struct rte_mbuf **sw_ring = &queue->bufq2->sw_ring[queue->rx_tail];
>  	volatile union virtchnl2_rx_desc *rxdp =
>  		(volatile union virtchnl2_rx_desc *)queue->rx_ring + queue->rx_tail;
> -	const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0, queue->mbuf_initializer);
> +	const __m256i mbuf_init = _mm256_set_epi64x(0, 0, 0, queue->bufq2->mbuf_initializer);
>  	uint64_t head_gen;
>  	uint16_t received = 0;
>  	int i;
> -- 
> 2.34.1
> 

[PATCH 4/6] net/idpf: fix ptype insert position in split queue AVX2 Rx

[Shaiq Wani]

The __m256i register mb10 holds rearm data for two mbufs: mbuf 0 in
the low 128-bit lane (dwords 0-3) and mbuf 1 in the high 128-bit
lane (dwords 4-7).  The packet_type field sits at dword 0 within
each mbuf's rearm_data layout.

For mbuf 1 (high lane), the packet_type must be inserted at
_mm256_insert_epi32 index 4 (first dword of the high 128-bit lane).
Index 2 is the third dword of the low lane, which overwrites the
wrong mbuf's data.  The same applies to mb32 for mbuf 3.

Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
---
 drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
index d3a8e17778..e66dcc7a14 100644
--- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
+++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
@@ -570,9 +570,9 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
 		ptype2 = (uint16_t)_mm256_extract_epi16(pt32, 1);
 		ptype3 = (uint16_t)_mm256_extract_epi16(pt32, 9);
 
-		mb10 = _mm256_insert_epi32(mb10, (int)ptype_tbl[ptype1], 2);
+		mb10 = _mm256_insert_epi32(mb10, (int)ptype_tbl[ptype1], 4);
 		mb10 = _mm256_insert_epi32(mb10, (int)ptype_tbl[ptype0], 0);
-		mb32 = _mm256_insert_epi32(mb32, (int)ptype_tbl[ptype3], 2);
+		mb32 = _mm256_insert_epi32(mb32, (int)ptype_tbl[ptype3], 4);
 		mb32 = _mm256_insert_epi32(mb32, (int)ptype_tbl[ptype2], 0);
 
 		/* Build rearm data for each mbuf */
-- 
2.34.1

Re: [PATCH 4/6] net/idpf: fix ptype insert position in split queue AVX2 Rx

[Bruce Richardson]

On Mon, May 11, 2026 at 02:39:33PM +0530, Shaiq Wani wrote:
> The __m256i register mb10 holds rearm data for two mbufs: mbuf 0 in
> the low 128-bit lane (dwords 0-3) and mbuf 1 in the high 128-bit
> lane (dwords 4-7).  The packet_type field sits at dword 0 within
> each mbuf's rearm_data layout.
> 
> For mbuf 1 (high lane), the packet_type must be inserted at
> _mm256_insert_epi32 index 4 (first dword of the high 128-bit lane).
> Index 2 is the third dword of the low lane, which overwrites the
> wrong mbuf's data.  The same applies to mb32 for mbuf 3.
> 
> Fixes: 1f065f9d75ff ("net/idpf: add AVX2 Rx path for split queue config")
> Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> index d3a8e17778..e66dcc7a14 100644
> --- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> +++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> @@ -570,9 +570,9 @@ idpf_dp_splitq_recv_pkts_avx2(void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_
>  		ptype2 = (uint16_t)_mm256_extract_epi16(pt32, 1);
>  		ptype3 = (uint16_t)_mm256_extract_epi16(pt32, 9);
>  
> -		mb10 = _mm256_insert_epi32(mb10, (int)ptype_tbl[ptype1], 2);
> +		mb10 = _mm256_insert_epi32(mb10, (int)ptype_tbl[ptype1], 4);
>  		mb10 = _mm256_insert_epi32(mb10, (int)ptype_tbl[ptype0], 0);
> -		mb32 = _mm256_insert_epi32(mb32, (int)ptype_tbl[ptype3], 2);
> +		mb32 = _mm256_insert_epi32(mb32, (int)ptype_tbl[ptype3], 4);
>  		mb32 = _mm256_insert_epi32(mb32, (int)ptype_tbl[ptype2], 0);
>  
>  		/* Build rearm data for each mbuf */
> -- 
> 2.34.1
> 

[PATCH 5/6] net/idpf: fix split queue AVX2 Tx buffer size shift

[Shaiq Wani]

The flex scheduled Tx descriptor (DTYPE 0x0C) used in split queue mode
places the buffer size at bits 48-63 of QW1, requiring a left-shift
of 48.  The code incorrectly used IDPF_TXD_QW1_TX_BUF_SZ_S (34),
which is the shift for base Tx descriptors (DTYPE 0x0) used in single
queue mode.

This caused the data_len to be placed in the wrong bit position,
resulting in hardware reading an incorrect buffer size of zero.

Define IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S (48) for the flex descriptor
layout and use it in both vtx1 and vtx_avx2, consistent with the
AVX512 split queue Tx path.

Fixes: 57560a92167a ("net/idpf: add AVX2 Tx path for split queue config")
Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
---
 drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
index e66dcc7a14..7c547b5f09 100644
--- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
+++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
@@ -819,6 +819,8 @@ idpf_splitq_scan_cq_ring(struct ci_tx_queue *cq)
 	cq->tx_tail = cq_qid;
 }
 
+#define IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S  48
+
 static __rte_always_inline void
 idpf_splitq_vtx1_avx2(struct idpf_flex_tx_sched_desc *txdp,
 				struct rte_mbuf *pkt, uint64_t flags)
@@ -826,7 +828,7 @@ idpf_splitq_vtx1_avx2(struct idpf_flex_tx_sched_desc *txdp,
 	uint64_t high_qw =
 		IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE |
 		((uint64_t)flags) |
-		((uint64_t)pkt->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
+		((uint64_t)pkt->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
 
 	__m128i descriptor = _mm_set_epi64x(high_qw,
 						pkt->buf_iova + pkt->data_off);
@@ -848,13 +850,13 @@ idpf_splitq_vtx_avx2(struct idpf_flex_tx_sched_desc *txdp,
 	for (; nb_pkts >= IDPF_VPMD_DESCS_PER_LOOP; txdp += IDPF_VPMD_DESCS_PER_LOOP,
 			pkt += IDPF_VPMD_DESCS_PER_LOOP, nb_pkts -= IDPF_VPMD_DESCS_PER_LOOP) {
 		uint64_t hi_qw0 = hi_qw_tmpl |
-			((uint64_t)pkt[0]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
+			((uint64_t)pkt[0]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
 		uint64_t hi_qw1 = hi_qw_tmpl |
-			((uint64_t)pkt[1]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
+			((uint64_t)pkt[1]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
 		uint64_t hi_qw2 = hi_qw_tmpl |
-			((uint64_t)pkt[2]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
+			((uint64_t)pkt[2]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
 		uint64_t hi_qw3 = hi_qw_tmpl |
-			((uint64_t)pkt[3]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
+			((uint64_t)pkt[3]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
 
 		__m256i desc0_1 = _mm256_set_epi64x(hi_qw1,
 			pkt[1]->buf_iova + pkt[1]->data_off,
-- 
2.34.1

Re: [PATCH 5/6] net/idpf: fix split queue AVX2 Tx buffer size shift

[Bruce Richardson]

On Mon, May 11, 2026 at 02:39:34PM +0530, Shaiq Wani wrote:
> The flex scheduled Tx descriptor (DTYPE 0x0C) used in split queue mode
> places the buffer size at bits 48-63 of QW1, requiring a left-shift
> of 48.  The code incorrectly used IDPF_TXD_QW1_TX_BUF_SZ_S (34),
> which is the shift for base Tx descriptors (DTYPE 0x0) used in single
> queue mode.
> 
> This caused the data_len to be placed in the wrong bit position,
> resulting in hardware reading an incorrect buffer size of zero.
> 
> Define IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S (48) for the flex descriptor
> layout and use it in both vtx1 and vtx_avx2, consistent with the
> AVX512 split queue Tx path.
> 
> Fixes: 57560a92167a ("net/idpf: add AVX2 Tx path for split queue config")
> Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  drivers/net/intel/idpf/idpf_common_rxtx_avx2.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> index e66dcc7a14..7c547b5f09 100644
> --- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> +++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> @@ -819,6 +819,8 @@ idpf_splitq_scan_cq_ring(struct ci_tx_queue *cq)
>  	cq->tx_tail = cq_qid;
>  }
>  
> +#define IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S  48
> +
>  static __rte_always_inline void
>  idpf_splitq_vtx1_avx2(struct idpf_flex_tx_sched_desc *txdp,
>  				struct rte_mbuf *pkt, uint64_t flags)
> @@ -826,7 +828,7 @@ idpf_splitq_vtx1_avx2(struct idpf_flex_tx_sched_desc *txdp,
>  	uint64_t high_qw =
>  		IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE |
>  		((uint64_t)flags) |
> -		((uint64_t)pkt->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
> +		((uint64_t)pkt->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
>  
>  	__m128i descriptor = _mm_set_epi64x(high_qw,
>  						pkt->buf_iova + pkt->data_off);
> @@ -848,13 +850,13 @@ idpf_splitq_vtx_avx2(struct idpf_flex_tx_sched_desc *txdp,
>  	for (; nb_pkts >= IDPF_VPMD_DESCS_PER_LOOP; txdp += IDPF_VPMD_DESCS_PER_LOOP,
>  			pkt += IDPF_VPMD_DESCS_PER_LOOP, nb_pkts -= IDPF_VPMD_DESCS_PER_LOOP) {
>  		uint64_t hi_qw0 = hi_qw_tmpl |
> -			((uint64_t)pkt[0]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
> +			((uint64_t)pkt[0]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
>  		uint64_t hi_qw1 = hi_qw_tmpl |
> -			((uint64_t)pkt[1]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
> +			((uint64_t)pkt[1]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
>  		uint64_t hi_qw2 = hi_qw_tmpl |
> -			((uint64_t)pkt[2]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
> +			((uint64_t)pkt[2]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
>  		uint64_t hi_qw3 = hi_qw_tmpl |
> -			((uint64_t)pkt[3]->data_len << IDPF_TXD_QW1_TX_BUF_SZ_S);
> +			((uint64_t)pkt[3]->data_len << IDPF_TXD_FLEX_QW1_TX_BUF_SZ_S);
>  
>  		__m256i desc0_1 = _mm256_set_epi64x(hi_qw1,
>  			pkt[1]->buf_iova + pkt[1]->data_off,
> -- 
> 2.34.1
> 

[PATCH 6/6] net/idpf: fix split queue AVX2 Tx burst and completion

[Shaiq Wani]

Clamp burst size to tx_rs_thresh to prevent crossing completion
boundaries.  Update tx_next_rs on ring wrap and after each burst
so that completion tracking in idpf_splitq_scan_cq_ring works
correctly.  Fix sw_ring pointer to use base-plus-offset.

Fixes: 57560a92167a ("net/idpf: add AVX2 Tx path for split queue config")
Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
---
 .../net/intel/idpf/idpf_common_rxtx_avx2.c    | 24 ++++++++++++++-----
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
index 7c547b5f09..b6c4fdf20e 100644
--- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
+++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
@@ -884,17 +884,21 @@ idpf_splitq_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts,
 	struct ci_tx_queue *txq = (struct ci_tx_queue *)tx_queue;
 	struct idpf_flex_tx_sched_desc *txdp;
 	struct ci_tx_entry_vec *txep;
-	uint16_t n, nb_commit;
+	uint16_t n, nb_commit, tx_id;
 	uint64_t cmd_dtype = IDPF_TXD_FLEX_FLOW_CMD_EOP;
-	uint16_t tx_id = txq->tx_tail;
 
-	nb_commit = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
-	nb_pkts = nb_commit;
+	/* cross rs_thresh boundary is not allowed */
+	nb_pkts = RTE_MIN(nb_pkts, txq->tx_rs_thresh);
+
+	nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
+	nb_commit = nb_pkts;
 	if (unlikely(nb_pkts == 0))
 		return 0;
 
-	txdp = (struct idpf_flex_tx_sched_desc *)&txq->desc_ring[tx_id];
-	txep = &txq->sw_ring_vec[tx_id];
+	tx_id = txq->tx_tail;
+	txdp = &txq->desc_ring[tx_id];
+	txep = (void *)txq->sw_ring;
+	txep += tx_id;
 
 	txq->nb_tx_free = (uint16_t)(txq->nb_tx_free - nb_pkts);
 
@@ -909,10 +913,14 @@ idpf_splitq_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts,
 		idpf_splitq_vtx1_avx2(txdp, *tx_pkts++, cmd_dtype);
 
 		nb_commit = (uint16_t)(nb_commit - n);
+
 		tx_id = 0;
+		txq->tx_next_rs = (uint16_t)(txq->tx_rs_thresh - 1);
 
+		/* avoid reach the end of ring */
 		txdp = &txq->desc_ring[tx_id];
 		txep = (void *)txq->sw_ring;
+		txep += tx_id;
 	}
 
 	ci_tx_backlog_entry_vec(txep, tx_pkts, nb_commit);
@@ -920,6 +928,10 @@ idpf_splitq_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts,
 	idpf_splitq_vtx_avx2(txdp, tx_pkts, nb_commit, cmd_dtype);
 
 	tx_id = (uint16_t)(tx_id + nb_commit);
+	if (tx_id > txq->tx_next_rs)
+		txq->tx_next_rs =
+			(uint16_t)(txq->tx_next_rs + txq->tx_rs_thresh);
+
 	txq->tx_tail = tx_id;
 
 	IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
-- 
2.34.1

Re: [PATCH 6/6] net/idpf: fix split queue AVX2 Tx burst and completion

[Bruce Richardson]

On Mon, May 11, 2026 at 02:39:35PM +0530, Shaiq Wani wrote:
> Clamp burst size to tx_rs_thresh to prevent crossing completion
> boundaries.  Update tx_next_rs on ring wrap and after each burst
> so that completion tracking in idpf_splitq_scan_cq_ring works
> correctly.  Fix sw_ring pointer to use base-plus-offset.
> 
> Fixes: 57560a92167a ("net/idpf: add AVX2 Tx path for split queue config")
> Signed-off-by: Shaiq Wani <shaiq.wani@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  .../net/intel/idpf/idpf_common_rxtx_avx2.c    | 24 ++++++++++++++-----
>  1 file changed, 18 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> index 7c547b5f09..b6c4fdf20e 100644
> --- a/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> +++ b/drivers/net/intel/idpf/idpf_common_rxtx_avx2.c
> @@ -884,17 +884,21 @@ idpf_splitq_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts,
>  	struct ci_tx_queue *txq = (struct ci_tx_queue *)tx_queue;
>  	struct idpf_flex_tx_sched_desc *txdp;
>  	struct ci_tx_entry_vec *txep;
> -	uint16_t n, nb_commit;
> +	uint16_t n, nb_commit, tx_id;
>  	uint64_t cmd_dtype = IDPF_TXD_FLEX_FLOW_CMD_EOP;
> -	uint16_t tx_id = txq->tx_tail;
>  
> -	nb_commit = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
> -	nb_pkts = nb_commit;
> +	/* cross rs_thresh boundary is not allowed */
> +	nb_pkts = RTE_MIN(nb_pkts, txq->tx_rs_thresh);
> +
> +	nb_pkts = (uint16_t)RTE_MIN(txq->nb_tx_free, nb_pkts);
> +	nb_commit = nb_pkts;
>  	if (unlikely(nb_pkts == 0))
>  		return 0;
>  
> -	txdp = (struct idpf_flex_tx_sched_desc *)&txq->desc_ring[tx_id];
> -	txep = &txq->sw_ring_vec[tx_id];
> +	tx_id = txq->tx_tail;
> +	txdp = &txq->desc_ring[tx_id];
> +	txep = (void *)txq->sw_ring;
> +	txep += tx_id;
>  
>  	txq->nb_tx_free = (uint16_t)(txq->nb_tx_free - nb_pkts);
>  
> @@ -909,10 +913,14 @@ idpf_splitq_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts,
>  		idpf_splitq_vtx1_avx2(txdp, *tx_pkts++, cmd_dtype);
>  
>  		nb_commit = (uint16_t)(nb_commit - n);
> +
>  		tx_id = 0;
> +		txq->tx_next_rs = (uint16_t)(txq->tx_rs_thresh - 1);
>  
> +		/* avoid reach the end of ring */
>  		txdp = &txq->desc_ring[tx_id];
>  		txep = (void *)txq->sw_ring;
> +		txep += tx_id;
>  	}
>  
>  	ci_tx_backlog_entry_vec(txep, tx_pkts, nb_commit);
> @@ -920,6 +928,10 @@ idpf_splitq_xmit_fixed_burst_vec_avx2(void *tx_queue, struct rte_mbuf **tx_pkts,
>  	idpf_splitq_vtx_avx2(txdp, tx_pkts, nb_commit, cmd_dtype);
>  
>  	tx_id = (uint16_t)(tx_id + nb_commit);
> +	if (tx_id > txq->tx_next_rs)
> +		txq->tx_next_rs =
> +			(uint16_t)(txq->tx_next_rs + txq->tx_rs_thresh);
> +
>  	txq->tx_tail = tx_id;
>  
>  	IDPF_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
> -- 
> 2.34.1
> 

Re: [PATCH 0/6] net/idpf: fix split queue AVX2 datapath

[Bruce Richardson]

On Mon, May 11, 2026 at 02:39:29PM +0530, Shaiq Wani wrote:
> Fix the split queue AVX2 vectorized path to correctly handle the
> virtchnl2_rx_flex_desc_adv_nic_3 completion descriptor format and
> the flex scheduled Tx descriptor encoding.
> 
> The split queue completion descriptor layout is significantly
> different from the single queue path — the generation bit, DD bit,
> and packet length occupy different positions than the base Rx
> descriptor, and the buffer queue owns the mbuf pool rather than the
> completion queue.
> 
> On Rx, four issues are fixed:
>   - Generation bit extraction order (mask-before-shift)
>   - DD bit byte offset (byte 8 not byte 1)
>   - mbuf initializer sourced from bufq2 instead of the completion queue
>   - Packet type inserted into the correct AVX2 lane
> 
> On Tx, two issues are fixed:
>   - Buffer size shift corrected from 34 to 48 for flex descriptors
>   - Burst clamped to tx_rs_thresh with proper tx_next_rs tracking
> 
> Shaiq Wani (6):
>   net/idpf: fix gen bit extraction in split queue AVX2 Rx
>   net/idpf: fix DD bit byte offset in split queue AVX2 Rx
>   net/idpf: fix mbuf initializer source in split queue AVX2 Rx
>   net/idpf: fix ptype insert position in split queue AVX2 Rx
>   net/idpf: fix split queue AVX2 Tx buffer size shift
>   net/idpf: fix split queue AVX2 Tx burst and completion
> 
Series applied to dpdk-next-net-intel
Thanks,
/Bruce