* Re: [PATCH v2] net/ixgbe: fix MAC/VLAN item validation for ntuple
From: Bruce Richardson @ 2026-05-19 17:00 UTC (permalink / raw)
To: Daniil Iskhakov
Cc: Anatoly Burakov, Vladimir Medvedkin, Wei Zhao, Wenzhuo Lu, dev,
stable, Daniil Agalakov, sdl.dpdk, rrv
In-Reply-To: <20260507132117.1449179-1-dish@amicon.ru>
On Thu, May 07, 2026 at 04:21:17PM +0300, Daniil Iskhakov wrote:
> When parsing an ntuple filter, the code attempts to ensure that if the
> first item is ETH or VLAN, its spec and mask are either absent or
> contain only zero fields. The current check is:
>
> if ((item->spec || item->mask) &&
> (memcmp(spec, &null_struct, size) ||
> memcmp(mask, &null_struct, size)))
>
> This condition is logically incorrect. If item->spec points to a
> zero-filled structure and item->mask is NULL, memcmp(mask) would
> dereference a NULL pointer.
>
> The intended behavior is to reject any non-zero spec or mask.
>
> Guard each memcmp() call with a check of the corresponding pointer while
> keeping a single error path.
>
> Found by Linux Verification Center (linuxtesting.org) with SVACE.
>
> Fixes: 46ea969177f3 ("net/ixgbe: add ntuple support to flow parser")
> Cc: stable@dpdk.org
>
> Signed-off-by: Daniil Agalakov <ade@amicon.ru>
> Signed-off-by: Daniil Iskhakov <dish@amicon.ru>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Applied to dpdk-next-net-intel.
Thanks,
/Bruce
^ permalink raw reply
* Re: [PATCH v3] net/idpf: harden PTP frequency adjustment
From: Bruce Richardson @ 2026-05-19 16:30 UTC (permalink / raw)
To: Soumyadeep Hore; +Cc: manoj.kumar.subbarao, aman.deep.singh, dev, rajesh3.kumar
In-Reply-To: <20260506204134.18634-1-soumyadeep.hore@intel.com>
On Wed, May 06, 2026 at 04:41:33PM -0400, Soumyadeep Hore wrote:
> Use the direct __int128 calculation on 64-bit builds to keep the
> frequency adjustment path short and readable, while preserving the
> portable split-and-scale calculation on 32-bit builds where __int128
> may be unavailable.
>
> Both paths avoid the INT64_MIN negation issue in the original code.
>
> Coverity issue: 501832
>
Fixes: 1c6e2737fd3e ("net/idpf: support timesync")
> Signed-off-by: Soumyadeep Hore <soumyadeep.hore@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Applied to dpdk-next-net-intel,
Thanks,
/Bruce
^ permalink raw reply
* Re: [PATCH v1 1/1] net/iavf: fix large VF IRQ mapping
From: Bruce Richardson @ 2026-05-19 16:12 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev, Vladimir Medvedkin, david.marchand, mb
In-Reply-To: <c50ef6c3d4ecaa7902d325d5e8a7fd4c92c8eeb6.1778076415.git.anatoly.burakov@intel.com>
On Wed, May 06, 2026 at 03:07:03PM +0100, Anatoly Burakov wrote:
> The PF will check buffer size for being too big, and the chunk sizing code
> correctly calls that out. However, the size was actually still too big
> because `struct virtchnl_queue_vector_maps` already had one queue vector
> as part of its definition, so `chunk_sz` was too big by 1.
>
> Fixes: 292d3b781ac4 ("net/iavf: replace unnecessary hugepage memory allocations")
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Although there is discussion on-list about this patch, none of it concerns
this patch being wrong in itself, but rather future cleanup work done.
Therefore, I'm considering this patch for merge to next-net-intel shortly.
/Bruce
> drivers/net/intel/iavf/iavf_vchnl.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/intel/iavf/iavf_vchnl.c b/drivers/net/intel/iavf/iavf_vchnl.c
> index c2f340db81..dd09b0fa61 100644
> --- a/drivers/net/intel/iavf/iavf_vchnl.c
> +++ b/drivers/net/intel/iavf/iavf_vchnl.c
> @@ -1528,7 +1528,7 @@ iavf_config_irq_map_lv_chunk(struct iavf_adapter *adapter,
>
> /* for some reason PF side checks for buffer being too big, so adjust it down */
> buf_len = sizeof(struct virtchnl_queue_vector_maps) +
> - sizeof(struct virtchnl_queue_vector) * chunk_sz;
> + sizeof(struct virtchnl_queue_vector) * (chunk_sz - 1);
>
> args.ops = VIRTCHNL_OP_MAP_QUEUE_VECTOR;
> args.in_args = (u8 *)map_info;
> --
> 2.47.3
>
^ permalink raw reply
* [RFC PATCH 8/8] net/ipn3ke: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:06 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, for example, flow
configuration and so is fine using the standard libc memcpy function in
place of the DPDK-specific rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/ipn3ke/ipn3ke_ethdev.h | 1 -
drivers/net/intel/ipn3ke/ipn3ke_flow.c | 26 +++++++++----------
drivers/net/intel/ipn3ke/ipn3ke_representor.c | 8 +++---
drivers/net/intel/ipn3ke/ipn3ke_tm.c | 6 ++---
4 files changed, 20 insertions(+), 21 deletions(-)
diff --git a/drivers/net/intel/ipn3ke/ipn3ke_ethdev.h b/drivers/net/intel/ipn3ke/ipn3ke_ethdev.h
index 4b93d2649e..0105fc9056 100644
--- a/drivers/net/intel/ipn3ke/ipn3ke_ethdev.h
+++ b/drivers/net/intel/ipn3ke/ipn3ke_ethdev.h
@@ -18,7 +18,6 @@
#include <ethdev_driver.h>
#include <ethdev_vdev.h>
#include <rte_malloc.h>
-#include <rte_memcpy.h>
#include <bus_vdev_driver.h>
#include <rte_kvargs.h>
#include <rte_spinlock.h>
diff --git a/drivers/net/intel/ipn3ke/ipn3ke_flow.c b/drivers/net/intel/ipn3ke/ipn3ke_flow.c
index a2f76268b5..941d2921fb 100644
--- a/drivers/net/intel/ipn3ke/ipn3ke_flow.c
+++ b/drivers/net/intel/ipn3ke/ipn3ke_flow.c
@@ -100,7 +100,7 @@ ipn3ke_pattern_vxlan(const struct rte_flow_item patterns[],
case RTE_FLOW_ITEM_TYPE_ETH:
eth = item->spec;
- rte_memcpy(&parser->key[0],
+ memcpy(&parser->key[0],
eth->hdr.src_addr.addr_bytes,
RTE_ETHER_ADDR_LEN);
break;
@@ -108,7 +108,7 @@ ipn3ke_pattern_vxlan(const struct rte_flow_item patterns[],
case RTE_FLOW_ITEM_TYPE_VXLAN:
vxlan = item->spec;
- rte_memcpy(&parser->key[6], vxlan->hdr.vni, 3);
+ memcpy(&parser->key[6], vxlan->hdr.vni, 3);
break;
default:
@@ -164,7 +164,7 @@ ipn3ke_pattern_mac(const struct rte_flow_item patterns[],
case RTE_FLOW_ITEM_TYPE_ETH:
eth = item->spec;
- rte_memcpy(parser->key,
+ memcpy(parser->key,
eth->hdr.src_addr.addr_bytes,
RTE_ETHER_ADDR_LEN);
break;
@@ -369,13 +369,13 @@ ipn3ke_pattern_ip_tcp(const struct rte_flow_item patterns[],
case RTE_FLOW_ITEM_TYPE_IPV4:
ipv4 = item->spec;
- rte_memcpy(&parser->key[0], &ipv4->hdr.src_addr, 4);
+ memcpy(&parser->key[0], &ipv4->hdr.src_addr, 4);
break;
case RTE_FLOW_ITEM_TYPE_TCP:
tcp = item->spec;
- rte_memcpy(&parser->key[4], &tcp->hdr.src_port, 2);
+ memcpy(&parser->key[4], &tcp->hdr.src_port, 2);
break;
default:
@@ -434,13 +434,13 @@ ipn3ke_pattern_ip_udp(const struct rte_flow_item patterns[],
case RTE_FLOW_ITEM_TYPE_IPV4:
ipv4 = item->spec;
- rte_memcpy(&parser->key[0], &ipv4->hdr.src_addr, 4);
+ memcpy(&parser->key[0], &ipv4->hdr.src_addr, 4);
break;
case RTE_FLOW_ITEM_TYPE_UDP:
udp = item->spec;
- rte_memcpy(&parser->key[4], &udp->hdr.src_port, 2);
+ memcpy(&parser->key[4], &udp->hdr.src_port, 2);
break;
default:
@@ -502,19 +502,19 @@ ipn3ke_pattern_ip_nvgre(const struct rte_flow_item patterns[],
case RTE_FLOW_ITEM_TYPE_IPV4:
ipv4 = item->spec;
- rte_memcpy(&parser->key[0], &ipv4->hdr.src_addr, 4);
+ memcpy(&parser->key[0], &ipv4->hdr.src_addr, 4);
break;
case RTE_FLOW_ITEM_TYPE_UDP:
udp = item->spec;
- rte_memcpy(&parser->key[4], &udp->hdr.src_port, 2);
+ memcpy(&parser->key[4], &udp->hdr.src_port, 2);
break;
case RTE_FLOW_ITEM_TYPE_NVGRE:
nvgre = item->spec;
- rte_memcpy(&parser->key[6], nvgre->tni, 3);
+ memcpy(&parser->key[6], nvgre->tni, 3);
break;
default:
@@ -576,19 +576,19 @@ ipn3ke_pattern_vxlan_ip_udp(const struct rte_flow_item patterns[],
case RTE_FLOW_ITEM_TYPE_VXLAN:
vxlan = item->spec;
- rte_memcpy(&parser->key[0], vxlan->hdr.vni, 3);
+ memcpy(&parser->key[0], vxlan->hdr.vni, 3);
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
ipv4 = item->spec;
- rte_memcpy(&parser->key[3], &ipv4->hdr.src_addr, 4);
+ memcpy(&parser->key[3], &ipv4->hdr.src_addr, 4);
break;
case RTE_FLOW_ITEM_TYPE_UDP:
udp = item->spec;
- rte_memcpy(&parser->key[7], &udp->hdr.src_port, 2);
+ memcpy(&parser->key[7], &udp->hdr.src_port, 2);
break;
default:
diff --git a/drivers/net/intel/ipn3ke/ipn3ke_representor.c b/drivers/net/intel/ipn3ke/ipn3ke_representor.c
index cd34d08055..281c025820 100644
--- a/drivers/net/intel/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/intel/ipn3ke/ipn3ke_representor.c
@@ -137,7 +137,7 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
if (hw->retimer.mac_type == IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
/* Set mac address */
- rte_memcpy(((char *)(&val)),
+ memcpy(((char *)(&val)),
(char *)&dev->data->mac_addrs->addr_bytes[0],
sizeof(uint32_t));
(*hw->f_mac_write)(hw,
@@ -145,7 +145,7 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
IPN3KE_MAC_PRIMARY_MAC_ADDR0,
rpst->port_id,
0);
- rte_memcpy(((char *)(&val)),
+ memcpy(((char *)(&val)),
(char *)&dev->data->mac_addrs->addr_bytes[4],
sizeof(uint16_t));
(*hw->f_mac_write)(hw,
@@ -2746,13 +2746,13 @@ ipn3ke_rpst_mac_addr_set(struct rte_eth_dev *ethdev,
rte_ether_addr_copy(&mac_addr[0], &rpst->mac_addr);
/* Set mac address */
- rte_memcpy(((char *)(&val)), &mac_addr[0], sizeof(uint32_t));
+ memcpy(((char *)(&val)), &mac_addr[0], sizeof(uint32_t));
(*hw->f_mac_write)(hw,
val,
IPN3KE_MAC_PRIMARY_MAC_ADDR0,
rpst->port_id,
0);
- rte_memcpy(((char *)(&val)), &mac_addr[4], sizeof(uint16_t));
+ memcpy(((char *)(&val)), &mac_addr[4], sizeof(uint16_t));
(*hw->f_mac_write)(hw,
val,
IPN3KE_MAC_PRIMARY_MAC_ADDR0,
diff --git a/drivers/net/intel/ipn3ke/ipn3ke_tm.c b/drivers/net/intel/ipn3ke/ipn3ke_tm.c
index 04677138e1..47b7734d4f 100644
--- a/drivers/net/intel/ipn3ke/ipn3ke_tm.c
+++ b/drivers/net/intel/ipn3ke/ipn3ke_tm.c
@@ -814,7 +814,7 @@ ipn3ke_tm_shaper_profile_add(struct rte_eth_dev *dev,
rte_strerror(EINVAL));
} else {
sp->valid = 1;
- rte_memcpy(&sp->params, profile, sizeof(sp->params));
+ memcpy(&sp->params, profile, sizeof(sp->params));
}
tm->h.n_shaper_profiles++;
@@ -960,7 +960,7 @@ ipn3ke_tm_tdrop_profile_add(struct rte_eth_dev *dev,
IPN3KE_TDROP_TH2_MASK);
tp->th1 = th1;
tp->th2 = th2;
- rte_memcpy(&tp->params, profile, sizeof(tp->params));
+ memcpy(&tp->params, profile, sizeof(tp->params));
/* Add to list */
tm->h.n_tdrop_profiles++;
@@ -1308,7 +1308,7 @@ ipn3ke_tm_node_add(struct rte_eth_dev *dev,
n->tdrop_profile = ipn3ke_hw_tm_tdrop_profile_search(hw,
params->leaf.wred.wred_profile_id);
- rte_memcpy(&n->params, params, sizeof(n->params));
+ memcpy(&n->params, params, sizeof(n->params));
return 0;
}
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 7/8] net/cpfl: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:06 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, and so is fine using
the standard libc memcpy function in place of the DPDK-specific
rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/cpfl/cpfl_ethdev.c | 4 ++--
drivers/net/intel/cpfl/cpfl_vchnl.c | 6 +++---
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/net/intel/cpfl/cpfl_ethdev.c b/drivers/net/intel/cpfl/cpfl_ethdev.c
index 617b823f5a..552d1feb27 100644
--- a/drivers/net/intel/cpfl/cpfl_ethdev.c
+++ b/drivers/net/intel/cpfl/cpfl_ethdev.c
@@ -2112,7 +2112,7 @@ int vcpf_save_chunk_in_cfgq(struct cpfl_adapter_ext *adapter)
PMD_DRV_LOG(ERR, "Failed to allocate memory for add_q");
return -ENOMEM;
}
- rte_memcpy(adapter->cfgq_in.cfgq_add, add_q, struct_size);
+ memcpy(adapter->cfgq_in.cfgq_add, add_q, struct_size);
num_chunks = add_q->chunks.num_chunks;
for (u16 i = 0; i < num_chunks; i++) {
@@ -2536,7 +2536,7 @@ cpfl_adapter_ext_init(struct rte_pci_device *pci_dev, struct cpfl_adapter_ext *a
strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
- rte_memcpy(&base->caps, &req_caps, sizeof(struct virtchnl2_get_capabilities));
+ memcpy(&base->caps, &req_caps, sizeof(struct virtchnl2_get_capabilities));
ret = idpf_adapter_init(base);
if (ret != 0) {
diff --git a/drivers/net/intel/cpfl/cpfl_vchnl.c b/drivers/net/intel/cpfl/cpfl_vchnl.c
index 9c842b60df..c9d122d2c3 100644
--- a/drivers/net/intel/cpfl/cpfl_vchnl.c
+++ b/drivers/net/intel/cpfl/cpfl_vchnl.c
@@ -32,7 +32,7 @@ cpfl_cc_vport_list_get(struct cpfl_adapter_ext *adapter,
return err;
}
- rte_memcpy(response, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
+ memcpy(response, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
return 0;
}
@@ -66,7 +66,7 @@ cpfl_cc_vport_info_get(struct cpfl_adapter_ext *adapter,
return err;
}
- rte_memcpy(response, args.out_buffer, sizeof(*response));
+ memcpy(response, args.out_buffer, sizeof(*response));
return 0;
}
@@ -139,7 +139,7 @@ vcpf_add_queues(struct cpfl_adapter_ext *adapter)
return err;
}
- rte_memcpy(adapter->addq_recv_info, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
+ memcpy(adapter->addq_recv_info, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
return err;
}
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 6/8] net/idpf: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:06 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, for example, flow
configuration and so is fine using the standard libc memcpy function in
place of the DPDK-specific rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/idpf/base/idpf_osdep.h | 5 ++---
drivers/net/intel/idpf/idpf_common_device.c | 2 +-
drivers/net/intel/idpf/idpf_common_virtchnl.c | 20 +++++++++----------
drivers/net/intel/idpf/idpf_ethdev.c | 10 +++++-----
drivers/net/intel/idpf/idpf_rxtx.c | 8 ++++----
5 files changed, 22 insertions(+), 23 deletions(-)
diff --git a/drivers/net/intel/idpf/base/idpf_osdep.h b/drivers/net/intel/idpf/base/idpf_osdep.h
index 47b95d0da6..67bc33f2e6 100644
--- a/drivers/net/intel/idpf/base/idpf_osdep.h
+++ b/drivers/net/intel/idpf/base/idpf_osdep.h
@@ -14,7 +14,6 @@
#include <stdbool.h>
#include <rte_common.h>
-#include <rte_memcpy.h>
#include <rte_malloc.h>
#include <rte_memzone.h>
#include <rte_byteorder.h>
@@ -199,8 +198,8 @@ struct __rte_packed_begin idpf_virt_mem {
#define idpf_free(h, m) rte_free(m)
#define idpf_memset(a, b, c, d) memset((a), (b), (c))
-#define idpf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
-#define idpf_memdup(a, b, c, d) rte_memcpy(idpf_malloc(a, c), b, c)
+#define idpf_memcpy(a, b, c, d) memcpy((a), (b), (c))
+#define idpf_memdup(a, b, c, d) memcpy(idpf_malloc(a, c), b, c)
#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
diff --git a/drivers/net/intel/idpf/idpf_common_device.c b/drivers/net/intel/idpf/idpf_common_device.c
index 24ee00db23..f27a911977 100644
--- a/drivers/net/intel/idpf/idpf_common_device.c
+++ b/drivers/net/intel/idpf/idpf_common_device.c
@@ -496,7 +496,7 @@ idpf_vport_init(struct idpf_vport *vport,
vport->num_rx_q = vport_info->num_rx_q;
vport->num_rx_bufq = vport_info->num_rx_bufq;
vport->max_mtu = vport_info->max_mtu;
- rte_memcpy(vport->default_mac_addr,
+ memcpy(vport->default_mac_addr,
vport_info->default_mac_addr, ETH_ALEN);
vport->rss_algorithm = vport_info->rss_algorithm;
vport->rss_key_size = RTE_MIN(IDPF_RSS_KEY_LEN,
diff --git a/drivers/net/intel/idpf/idpf_common_virtchnl.c b/drivers/net/intel/idpf/idpf_common_virtchnl.c
index f8c487c537..4befd5fc83 100644
--- a/drivers/net/intel/idpf/idpf_common_virtchnl.c
+++ b/drivers/net/intel/idpf/idpf_common_virtchnl.c
@@ -141,7 +141,7 @@ idpf_read_msg_from_cp(struct idpf_adapter *adapter, uint16_t buf_len,
return result;
}
- rte_memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
+ memcpy(buf, ctlq_msg.ctx.indirect.payload->va, buf_len);
opcode = rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_opcode);
adapter->cmd_retval = rte_le_to_cpu_32(ctlq_msg.cookie.mbx.chnl_retval);
@@ -323,7 +323,7 @@ idpf_vc_caps_get(struct idpf_adapter *adapter)
return err;
}
- rte_memcpy(&adapter->caps, args.out_buffer, sizeof(struct virtchnl2_get_capabilities));
+ memcpy(&adapter->caps, args.out_buffer, sizeof(struct virtchnl2_get_capabilities));
return 0;
}
@@ -361,7 +361,7 @@ idpf_vc_vport_create(struct idpf_vport *vport,
return err;
}
- rte_memcpy(&(vport->vport_info.info), args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
+ memcpy(&(vport->vport_info.info), args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
return 0;
}
@@ -419,7 +419,7 @@ idpf_vc_queue_grps_add(struct idpf_vport *vport,
return err;
}
- rte_memcpy(p2p_queue_grps_out, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
+ memcpy(p2p_queue_grps_out, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
return 0;
}
@@ -475,7 +475,7 @@ idpf_vc_rss_key_set(struct idpf_vport *vport)
rss_key->vport_id = vport->vport_id;
rss_key->key_len = vport->rss_key_size;
- rte_memcpy(rss_key->key, vport->rss_key,
+ memcpy(rss_key->key, vport->rss_key,
sizeof(rss_key->key[0]) * vport->rss_key_size);
memset(&args, 0, sizeof(args));
@@ -528,7 +528,7 @@ int idpf_vc_rss_key_get(struct idpf_vport *vport)
return -ENOMEM;
}
}
- rte_memcpy(vport->rss_key, rss_key_ret->key, vport->rss_key_size);
+ memcpy(vport->rss_key, rss_key_ret->key, vport->rss_key_size);
} else {
DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_RSS_KEY");
}
@@ -553,7 +553,7 @@ idpf_vc_rss_lut_set(struct idpf_vport *vport)
rss_lut->vport_id = vport->vport_id;
rss_lut->lut_entries = vport->rss_lut_size;
- rte_memcpy(rss_lut->lut, vport->rss_lut,
+ memcpy(rss_lut->lut, vport->rss_lut,
sizeof(rss_lut->lut[0]) * vport->rss_lut_size);
memset(&args, 0, sizeof(args));
@@ -605,7 +605,7 @@ idpf_vc_rss_lut_get(struct idpf_vport *vport)
return -ENOMEM;
}
}
- rte_memcpy(vport->rss_lut, rss_lut_ret->lut, rss_lut_ret->lut_entries);
+ memcpy(vport->rss_lut, rss_lut_ret->lut, rss_lut_ret->lut_entries);
vport->rss_lut_size = rss_lut_ret->lut_entries;
} else {
DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_RSS_LUT");
@@ -742,7 +742,7 @@ idpf_vc_vectors_alloc(struct idpf_vport *vport, uint16_t num_vectors)
if (err != 0)
DRV_LOG(ERR, "Failed to execute command VIRTCHNL2_OP_ALLOC_VECTORS");
- rte_memcpy(vport->recv_vectors, args.out_buffer, len);
+ memcpy(vport->recv_vectors, args.out_buffer, len);
rte_free(alloc_vec);
return err;
}
@@ -1007,7 +1007,7 @@ idpf_vc_ptype_info_query(struct idpf_adapter *adapter,
if (err != 0)
DRV_LOG(ERR, "Failed to execute command of VIRTCHNL2_OP_GET_PTYPE_INFO");
- rte_memcpy(recv_ptype_info, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
+ memcpy(recv_ptype_info, args.out_buffer, IDPF_DFLT_MBX_BUF_SIZE);
return err;
}
diff --git a/drivers/net/intel/idpf/idpf_ethdev.c b/drivers/net/intel/idpf/idpf_ethdev.c
index 5e57a45775..6b3016ba77 100644
--- a/drivers/net/intel/idpf/idpf_ethdev.c
+++ b/drivers/net/intel/idpf/idpf_ethdev.c
@@ -439,7 +439,7 @@ idpf_init_rss(struct idpf_vport *vport)
vport->rss_key_size);
return -EINVAL;
} else {
- rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ memcpy(vport->rss_key, rss_conf->rss_key,
vport->rss_key_size);
}
@@ -558,7 +558,7 @@ idpf_rss_hash_update(struct rte_eth_dev *dev,
return -EINVAL;
}
- rte_memcpy(vport->rss_key, rss_conf->rss_key,
+ memcpy(vport->rss_key, rss_conf->rss_key,
vport->rss_key_size);
ret = idpf_vc_rss_key_set(vport);
if (ret != 0) {
@@ -631,7 +631,7 @@ idpf_rss_hash_conf_get(struct rte_eth_dev *dev,
if (rss_conf->rss_key_len > vport->rss_key_size)
rss_conf->rss_key_len = vport->rss_key_size;
- rte_memcpy(rss_conf->rss_key, vport->rss_key, rss_conf->rss_key_len);
+ memcpy(rss_conf->rss_key, vport->rss_key, rss_conf->rss_key_len);
return 0;
}
@@ -1363,7 +1363,7 @@ idpf_handle_virtchnl_msg(struct idpf_adapter_ext *adapter_ex)
return;
}
- rte_memcpy(adapter->mbx_resp, ctlq_msg.ctx.indirect.payload->va,
+ memcpy(adapter->mbx_resp, ctlq_msg.ctx.indirect.payload->va,
IDPF_DFLT_MBX_BUF_SIZE);
mbx_op = rte_le_to_cpu_16(ctlq_msg.opcode);
@@ -1477,7 +1477,7 @@ idpf_adapter_ext_init(struct rte_pci_device *pci_dev, struct idpf_adapter_ext *a
strncpy(adapter->name, pci_dev->device.name, PCI_PRI_STR_SIZE);
- rte_memcpy(&base->caps, &req_caps, sizeof(struct virtchnl2_get_capabilities));
+ memcpy(&base->caps, &req_caps, sizeof(struct virtchnl2_get_capabilities));
ret = idpf_adapter_init(base);
if (ret != 0) {
diff --git a/drivers/net/intel/idpf/idpf_rxtx.c b/drivers/net/intel/idpf/idpf_rxtx.c
index b316c77b62..07f3df4a80 100644
--- a/drivers/net/intel/idpf/idpf_rxtx.c
+++ b/drivers/net/intel/idpf/idpf_rxtx.c
@@ -74,7 +74,7 @@ idpf_dma_zone_reserve(struct rte_eth_dev *dev, uint16_t queue_idx,
else
ring_size = RTE_ALIGN(len * sizeof(struct ci_tx_desc),
IDPF_DMA_MEM_ALIGN);
- rte_memcpy(ring_name, "idpf Tx ring", sizeof("idpf Tx ring"));
+ memcpy(ring_name, "idpf Tx ring", sizeof("idpf Tx ring"));
break;
case VIRTCHNL2_QUEUE_TYPE_RX:
if (splitq)
@@ -83,17 +83,17 @@ idpf_dma_zone_reserve(struct rte_eth_dev *dev, uint16_t queue_idx,
else
ring_size = RTE_ALIGN(len * sizeof(struct virtchnl2_singleq_rx_buf_desc),
IDPF_DMA_MEM_ALIGN);
- rte_memcpy(ring_name, "idpf Rx ring", sizeof("idpf Rx ring"));
+ memcpy(ring_name, "idpf Rx ring", sizeof("idpf Rx ring"));
break;
case VIRTCHNL2_QUEUE_TYPE_TX_COMPLETION:
ring_size = RTE_ALIGN(len * sizeof(struct idpf_splitq_tx_compl_desc),
IDPF_DMA_MEM_ALIGN);
- rte_memcpy(ring_name, "idpf Tx compl ring", sizeof("idpf Tx compl ring"));
+ memcpy(ring_name, "idpf Tx compl ring", sizeof("idpf Tx compl ring"));
break;
case VIRTCHNL2_QUEUE_TYPE_RX_BUFFER:
ring_size = RTE_ALIGN(len * sizeof(struct virtchnl2_splitq_rx_buf_desc),
IDPF_DMA_MEM_ALIGN);
- rte_memcpy(ring_name, "idpf Rx buf ring", sizeof("idpf Rx buf ring"));
+ memcpy(ring_name, "idpf Rx buf ring", sizeof("idpf Rx buf ring"));
break;
default:
PMD_INIT_LOG(ERR, "Invalid queue type");
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 5/8] net/e1000: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:06 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, for example, flow
configuration and so is fine using the standard libc memcpy function in
place of the DPDK-specific rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/e1000/em_rxtx.c | 1 -
drivers/net/intel/e1000/igb_flow.c | 8 ++++----
drivers/net/intel/e1000/igb_pf.c | 5 ++---
drivers/net/intel/e1000/igb_rxtx.c | 1 -
4 files changed, 6 insertions(+), 9 deletions(-)
diff --git a/drivers/net/intel/e1000/em_rxtx.c b/drivers/net/intel/e1000/em_rxtx.c
index 54971fe285..5879013a1d 100644
--- a/drivers/net/intel/e1000/em_rxtx.c
+++ b/drivers/net/intel/e1000/em_rxtx.c
@@ -20,7 +20,6 @@
#include <rte_pci.h>
#include <bus_pci_driver.h>
#include <rte_memory.h>
-#include <rte_memcpy.h>
#include <rte_memzone.h>
#include <rte_launch.h>
#include <rte_eal.h>
diff --git a/drivers/net/intel/e1000/igb_flow.c b/drivers/net/intel/e1000/igb_flow.c
index ea9b290e1c..ca76bf5ae4 100644
--- a/drivers/net/intel/e1000/igb_flow.c
+++ b/drivers/net/intel/e1000/igb_flow.c
@@ -1484,7 +1484,7 @@ igb_flow_create(struct rte_eth_dev *dev,
goto out;
}
- rte_memcpy(&ntuple_filter_ptr->filter_info,
+ memcpy(&ntuple_filter_ptr->filter_info,
&ntuple_filter,
sizeof(struct rte_eth_ntuple_filter));
TAILQ_INSERT_TAIL(&igb_filter_ntuple_list,
@@ -1511,7 +1511,7 @@ igb_flow_create(struct rte_eth_dev *dev,
goto out;
}
- rte_memcpy(ðertype_filter_ptr->filter_info,
+ memcpy(ðertype_filter_ptr->filter_info,
ðertype_filter,
sizeof(struct rte_eth_ethertype_filter));
TAILQ_INSERT_TAIL(&igb_filter_ethertype_list,
@@ -1536,7 +1536,7 @@ igb_flow_create(struct rte_eth_dev *dev,
goto out;
}
- rte_memcpy(&syn_filter_ptr->filter_info,
+ memcpy(&syn_filter_ptr->filter_info,
&syn_filter,
sizeof(struct rte_eth_syn_filter));
TAILQ_INSERT_TAIL(&igb_filter_syn_list,
@@ -1562,7 +1562,7 @@ igb_flow_create(struct rte_eth_dev *dev,
goto out;
}
- rte_memcpy(&flex_filter_ptr->filter_info,
+ memcpy(&flex_filter_ptr->filter_info,
&flex_filter,
sizeof(struct igb_flex_filter));
TAILQ_INSERT_TAIL(&igb_filter_flex_list,
diff --git a/drivers/net/intel/e1000/igb_pf.c b/drivers/net/intel/e1000/igb_pf.c
index c7588ea57e..a3327e0bf0 100644
--- a/drivers/net/intel/e1000/igb_pf.c
+++ b/drivers/net/intel/e1000/igb_pf.c
@@ -17,7 +17,6 @@
#include <rte_eal.h>
#include <rte_ether.h>
#include <ethdev_driver.h>
-#include <rte_memcpy.h>
#include <rte_malloc.h>
#include <rte_random.h>
@@ -290,7 +289,7 @@ igb_vf_reset(struct rte_eth_dev *dev, uint16_t vf, uint32_t *msgbuf)
/* reply to reset with ack and vf mac address */
msgbuf[0] = E1000_VF_RESET | E1000_VT_MSGTYPE_ACK;
- rte_memcpy(new_mac, vf_mac, RTE_ETHER_ADDR_LEN);
+ memcpy(new_mac, vf_mac, RTE_ETHER_ADDR_LEN);
e1000_write_mbx(hw, msgbuf, 3, vf);
return 0;
@@ -308,7 +307,7 @@ igb_vf_set_mac_addr(struct rte_eth_dev *dev, uint32_t vf, uint32_t *msgbuf)
if (rte_is_unicast_ether_addr((struct rte_ether_addr *)new_mac)) {
if (!rte_is_zero_ether_addr((struct rte_ether_addr *)new_mac))
- rte_memcpy(vfinfo[vf].vf_mac_addresses, new_mac,
+ memcpy(vfinfo[vf].vf_mac_addresses, new_mac,
sizeof(vfinfo[vf].vf_mac_addresses));
hw->mac.ops.rar_set(hw, new_mac, rar_entry);
rah = E1000_READ_REG(hw, E1000_RAH(rar_entry));
diff --git a/drivers/net/intel/e1000/igb_rxtx.c b/drivers/net/intel/e1000/igb_rxtx.c
index 14b2858817..4fda5d57a0 100644
--- a/drivers/net/intel/e1000/igb_rxtx.c
+++ b/drivers/net/intel/e1000/igb_rxtx.c
@@ -19,7 +19,6 @@
#include <rte_debug.h>
#include <rte_pci.h>
#include <rte_memory.h>
-#include <rte_memcpy.h>
#include <rte_memzone.h>
#include <rte_launch.h>
#include <rte_eal.h>
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 4/8] net/ixgbe: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:06 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, for example, flow
configuration and so is fine using the standard libc memcpy function in
place of the DPDK-specific rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 4 ++--
drivers/net/intel/ixgbe/ixgbe_fdir.c | 6 ++---
drivers/net/intel/ixgbe/ixgbe_flow.c | 32 ++++++++++++-------------
drivers/net/intel/ixgbe/ixgbe_ipsec.c | 4 ++--
drivers/net/intel/ixgbe/ixgbe_pf.c | 5 ++--
drivers/net/intel/ixgbe/ixgbe_tm.c | 6 ++---
drivers/net/intel/ixgbe/rte_pmd_ixgbe.c | 2 +-
7 files changed, 29 insertions(+), 30 deletions(-)
diff --git a/drivers/net/intel/ixgbe/ixgbe_ethdev.c b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
index 57d929cf2c..d4e3d643f8 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ethdev.c
@@ -6805,7 +6805,7 @@ ixgbe_add_del_ntuple_filter(struct rte_eth_dev *dev,
sizeof(struct ixgbe_5tuple_filter), 0);
if (filter == NULL)
return -ENOMEM;
- rte_memcpy(&filter->filter_info,
+ memcpy(&filter->filter_info,
&filter_5tuple,
sizeof(struct ixgbe_5tuple_filter_info));
filter->queue = ntuple_filter->queue;
@@ -7867,7 +7867,7 @@ ixgbe_dev_l2_tunnel_filter_add(struct rte_eth_dev *dev,
if (!node)
return -ENOMEM;
- rte_memcpy(&node->key,
+ memcpy(&node->key,
&key,
sizeof(struct ixgbe_l2_tn_key));
node->pool = l2_tunnel->pool;
diff --git a/drivers/net/intel/ixgbe/ixgbe_fdir.c b/drivers/net/intel/ixgbe/ixgbe_fdir.c
index 0bdfbd411a..003f8240bd 100644
--- a/drivers/net/intel/ixgbe/ixgbe_fdir.c
+++ b/drivers/net/intel/ixgbe/ixgbe_fdir.c
@@ -53,7 +53,7 @@
#define IPV6_ADDR_TO_MASK(ipaddr, ipv6m) do { \
uint8_t ipv6_addr[16]; \
uint8_t i; \
- rte_memcpy(ipv6_addr, (ipaddr), sizeof(ipv6_addr));\
+ memcpy(ipv6_addr, (ipaddr), sizeof(ipv6_addr));\
(ipv6m) = 0; \
for (i = 0; i < sizeof(ipv6_addr); i++) { \
if (ipv6_addr[i] == UINT8_MAX) \
@@ -73,7 +73,7 @@
else \
ipv6_addr[i] = 0; \
} \
- rte_memcpy((ipaddr), ipv6_addr, sizeof(ipv6_addr));\
+ memcpy((ipaddr), ipv6_addr, sizeof(ipv6_addr));\
} while (0)
#define IXGBE_FDIRIP6M_INNER_MAC_SHIFT 4
@@ -1219,7 +1219,7 @@ ixgbe_fdir_filter_program(struct rte_eth_dev *dev,
0);
if (!node)
return -ENOMEM;
- rte_memcpy(&node->ixgbe_fdir,
+ memcpy(&node->ixgbe_fdir,
&rule->ixgbe_fdir,
sizeof(union ixgbe_atr_input));
node->fdirflags = fdircmd_flags;
diff --git a/drivers/net/intel/ixgbe/ixgbe_flow.c b/drivers/net/intel/ixgbe/ixgbe_flow.c
index 01cd4f9bde..8671761bc5 100644
--- a/drivers/net/intel/ixgbe/ixgbe_flow.c
+++ b/drivers/net/intel/ixgbe/ixgbe_flow.c
@@ -2024,9 +2024,9 @@ ixgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev,
if (item->spec) {
rule->b_spec = TRUE;
ipv6_spec = item->spec;
- rte_memcpy(rule->ixgbe_fdir.formatted.src_ip,
+ memcpy(rule->ixgbe_fdir.formatted.src_ip,
&ipv6_spec->hdr.src_addr, 16);
- rte_memcpy(rule->ixgbe_fdir.formatted.dst_ip,
+ memcpy(rule->ixgbe_fdir.formatted.dst_ip,
&ipv6_spec->hdr.dst_addr, 16);
}
@@ -2581,13 +2581,13 @@ ixgbe_parse_fdir_filter_tunnel(const struct rte_flow_attr *attr,
return -rte_errno;
}
- rte_memcpy(&rule->mask.tunnel_id_mask, vxlan_mask->hdr.vni,
+ memcpy(&rule->mask.tunnel_id_mask, vxlan_mask->hdr.vni,
RTE_DIM(vxlan_mask->hdr.vni));
if (item->spec) {
rule->b_spec = TRUE;
vxlan_spec = item->spec;
- rte_memcpy(((uint8_t *)
+ memcpy(((uint8_t *)
&rule->ixgbe_fdir.formatted.tni_vni),
vxlan_spec->hdr.vni, RTE_DIM(vxlan_spec->hdr.vni));
}
@@ -2658,7 +2658,7 @@ ixgbe_parse_fdir_filter_tunnel(const struct rte_flow_attr *attr,
return -rte_errno;
}
/* tni is a 24-bits bit field */
- rte_memcpy(&rule->mask.tunnel_id_mask, nvgre_mask->tni,
+ memcpy(&rule->mask.tunnel_id_mask, nvgre_mask->tni,
RTE_DIM(nvgre_mask->tni));
rule->mask.tunnel_id_mask <<= 8;
@@ -2684,7 +2684,7 @@ ixgbe_parse_fdir_filter_tunnel(const struct rte_flow_attr *attr,
return -rte_errno;
}
/* tni is a 24-bits bit field */
- rte_memcpy(&rule->ixgbe_fdir.formatted.tni_vni,
+ memcpy(&rule->ixgbe_fdir.formatted.tni_vni,
nvgre_spec->tni, RTE_DIM(nvgre_spec->tni));
}
}
@@ -3155,7 +3155,7 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
PMD_DRV_LOG(ERR, "failed to allocate memory");
goto out;
}
- rte_memcpy(&ntuple_filter_ptr->filter_info,
+ memcpy(&ntuple_filter_ptr->filter_info,
&ntuple_filter,
sizeof(struct rte_eth_ntuple_filter));
TAILQ_INSERT_TAIL(&filter_ntuple_list,
@@ -3181,7 +3181,7 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
PMD_DRV_LOG(ERR, "failed to allocate memory");
goto out;
}
- rte_memcpy(ðertype_filter_ptr->filter_info,
+ memcpy(ðertype_filter_ptr->filter_info,
ðertype_filter,
sizeof(struct rte_eth_ethertype_filter));
TAILQ_INSERT_TAIL(&filter_ethertype_list,
@@ -3205,7 +3205,7 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
PMD_DRV_LOG(ERR, "failed to allocate memory");
goto out;
}
- rte_memcpy(&syn_filter_ptr->filter_info,
+ memcpy(&syn_filter_ptr->filter_info,
&syn_filter,
sizeof(struct rte_eth_syn_filter));
TAILQ_INSERT_TAIL(&filter_syn_list,
@@ -3268,7 +3268,7 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
PMD_DRV_LOG(ERR, "failed to allocate memory");
goto out;
}
- rte_memcpy(&fdir_rule_ptr->filter_info,
+ memcpy(&fdir_rule_ptr->filter_info,
&fdir_rule,
sizeof(struct ixgbe_fdir_rule));
TAILQ_INSERT_TAIL(&filter_fdir_list,
@@ -3305,7 +3305,7 @@ ixgbe_flow_create(struct rte_eth_dev *dev,
PMD_DRV_LOG(ERR, "failed to allocate memory");
goto out;
}
- rte_memcpy(&l2_tn_filter_ptr->filter_info,
+ memcpy(&l2_tn_filter_ptr->filter_info,
&l2_tn_filter,
sizeof(struct ixgbe_l2_tunnel_conf));
TAILQ_INSERT_TAIL(&filter_l2_tunnel_list,
@@ -3447,7 +3447,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
case RTE_ETH_FILTER_NTUPLE:
ntuple_filter_ptr = (struct ixgbe_ntuple_filter_ele *)
pmd_flow->rule;
- rte_memcpy(&ntuple_filter,
+ memcpy(&ntuple_filter,
&ntuple_filter_ptr->filter_info,
sizeof(struct rte_eth_ntuple_filter));
ret = ixgbe_add_del_ntuple_filter(dev, &ntuple_filter, FALSE);
@@ -3460,7 +3460,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
case RTE_ETH_FILTER_ETHERTYPE:
ethertype_filter_ptr = (struct ixgbe_ethertype_filter_ele *)
pmd_flow->rule;
- rte_memcpy(ðertype_filter,
+ memcpy(ðertype_filter,
ðertype_filter_ptr->filter_info,
sizeof(struct rte_eth_ethertype_filter));
ret = ixgbe_add_del_ethertype_filter(dev,
@@ -3474,7 +3474,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
case RTE_ETH_FILTER_SYN:
syn_filter_ptr = (struct ixgbe_eth_syn_filter_ele *)
pmd_flow->rule;
- rte_memcpy(&syn_filter,
+ memcpy(&syn_filter,
&syn_filter_ptr->filter_info,
sizeof(struct rte_eth_syn_filter));
ret = ixgbe_syn_filter_set(dev, &syn_filter, FALSE);
@@ -3486,7 +3486,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
break;
case RTE_ETH_FILTER_FDIR:
fdir_rule_ptr = (struct ixgbe_fdir_rule_ele *)pmd_flow->rule;
- rte_memcpy(&fdir_rule,
+ memcpy(&fdir_rule,
&fdir_rule_ptr->filter_info,
sizeof(struct ixgbe_fdir_rule));
ret = ixgbe_fdir_filter_program(dev, &fdir_rule, TRUE, FALSE);
@@ -3501,7 +3501,7 @@ ixgbe_flow_destroy(struct rte_eth_dev *dev,
case RTE_ETH_FILTER_L2_TUNNEL:
l2_tn_filter_ptr = (struct ixgbe_eth_l2_tunnel_conf_ele *)
pmd_flow->rule;
- rte_memcpy(&l2_tn_filter, &l2_tn_filter_ptr->filter_info,
+ memcpy(&l2_tn_filter, &l2_tn_filter_ptr->filter_info,
sizeof(struct ixgbe_l2_tunnel_conf));
ret = ixgbe_dev_l2_tunnel_filter_del(dev, &l2_tn_filter);
if (!ret) {
diff --git a/drivers/net/intel/ixgbe/ixgbe_ipsec.c b/drivers/net/intel/ixgbe/ixgbe_ipsec.c
index fe9a96c54d..fd3d7a6689 100644
--- a/drivers/net/intel/ixgbe/ixgbe_ipsec.c
+++ b/drivers/net/intel/ixgbe/ixgbe_ipsec.c
@@ -674,9 +674,9 @@ ixgbe_crypto_add_ingress_sa_from_flow(struct rte_security_session *sess,
const struct rte_flow_item_ipv6 *ipv6 = &spec->spec.ipv6;
ic_session->src_ip.type = IPv6;
ic_session->dst_ip.type = IPv6;
- rte_memcpy(ic_session->src_ip.ipv6,
+ memcpy(ic_session->src_ip.ipv6,
&ipv6->hdr.src_addr, 16);
- rte_memcpy(ic_session->dst_ip.ipv6,
+ memcpy(ic_session->dst_ip.ipv6,
&ipv6->hdr.dst_addr, 16);
} else {
const struct rte_flow_item_ipv4 *ipv4 = &spec->spec.ipv4;
diff --git a/drivers/net/intel/ixgbe/ixgbe_pf.c b/drivers/net/intel/ixgbe/ixgbe_pf.c
index d9a775f99a..d3db571918 100644
--- a/drivers/net/intel/ixgbe/ixgbe_pf.c
+++ b/drivers/net/intel/ixgbe/ixgbe_pf.c
@@ -16,7 +16,6 @@
#include <rte_eal.h>
#include <rte_ether.h>
#include <ethdev_driver.h>
-#include <rte_memcpy.h>
#include <rte_malloc.h>
#include <rte_random.h>
@@ -452,7 +451,7 @@ ixgbe_vf_reset(struct rte_eth_dev *dev, uint16_t vf, uint32_t *msgbuf)
/* reply to reset with success and vf mac address */
msgbuf[0] = IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_SUCCESS;
- rte_memcpy(new_mac, vf_mac, RTE_ETHER_ADDR_LEN);
+ memcpy(new_mac, vf_mac, RTE_ETHER_ADDR_LEN);
/*
* Piggyback the multicast filter type so VF can compute the
* correct vectors
@@ -474,7 +473,7 @@ ixgbe_vf_set_mac_addr(struct rte_eth_dev *dev, uint32_t vf, uint32_t *msgbuf)
if (rte_is_valid_assigned_ether_addr(
(struct rte_ether_addr *)new_mac)) {
- rte_memcpy(vfinfo[vf].vf_mac_addresses, new_mac, 6);
+ memcpy(vfinfo[vf].vf_mac_addresses, new_mac, 6);
return hw->mac.ops.set_rar(hw, rar_entry, new_mac, vf, IXGBE_RAH_AV);
}
return -1;
diff --git a/drivers/net/intel/ixgbe/ixgbe_tm.c b/drivers/net/intel/ixgbe/ixgbe_tm.c
index 27a821285d..e1d8364f46 100644
--- a/drivers/net/intel/ixgbe/ixgbe_tm.c
+++ b/drivers/net/intel/ixgbe/ixgbe_tm.c
@@ -289,7 +289,7 @@ ixgbe_shaper_profile_add(struct rte_eth_dev *dev,
if (!shaper_profile)
return -ENOMEM;
shaper_profile->shaper_profile_id = shaper_profile_id;
- rte_memcpy(&shaper_profile->profile, profile,
+ memcpy(&shaper_profile->profile, profile,
sizeof(struct rte_tm_shaper_params));
TAILQ_INSERT_TAIL(&tm_conf->shaper_profile_list,
shaper_profile, node);
@@ -637,7 +637,7 @@ ixgbe_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->no = 0;
tm_node->parent = NULL;
tm_node->shaper_profile = shaper_profile;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
tm_conf->root = tm_node;
@@ -718,7 +718,7 @@ ixgbe_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->reference_count = 0;
tm_node->parent = parent_node;
tm_node->shaper_profile = shaper_profile;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
if (parent_node_type == IXGBE_TM_NODE_TYPE_PORT) {
tm_node->no = parent_node->reference_count;
diff --git a/drivers/net/intel/ixgbe/rte_pmd_ixgbe.c b/drivers/net/intel/ixgbe/rte_pmd_ixgbe.c
index c2300a8955..30dec57be8 100644
--- a/drivers/net/intel/ixgbe/rte_pmd_ixgbe.c
+++ b/drivers/net/intel/ixgbe/rte_pmd_ixgbe.c
@@ -39,7 +39,7 @@ rte_pmd_ixgbe_set_vf_mac_addr(uint16_t port, uint16_t vf,
if (rte_is_valid_assigned_ether_addr(
(struct rte_ether_addr *)new_mac)) {
- rte_memcpy(vfinfo[vf].vf_mac_addresses, new_mac,
+ memcpy(vfinfo[vf].vf_mac_addresses, new_mac,
RTE_ETHER_ADDR_LEN);
return hw->mac.ops.set_rar(hw, rar_entry, new_mac, vf,
IXGBE_RAH_AV);
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 3/8] net/i40e: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:06 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, for example, flow
configuration and so is fine using the standard libc memcpy function in
place of the DPDK-specific rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/i40e/base/i40e_osdep.h | 3 +-
drivers/net/intel/i40e/i40e_ethdev.c | 57 ++++++++++++------------
drivers/net/intel/i40e/i40e_fdir.c | 18 ++++----
drivers/net/intel/i40e/i40e_flow.c | 26 +++++------
drivers/net/intel/i40e/i40e_pf.c | 3 +-
drivers/net/intel/i40e/i40e_tm.c | 6 +--
drivers/net/intel/i40e/rte_pmd_i40e.c | 20 ++++-----
7 files changed, 65 insertions(+), 68 deletions(-)
diff --git a/drivers/net/intel/i40e/base/i40e_osdep.h b/drivers/net/intel/i40e/base/i40e_osdep.h
index 197f4678bf..9c5a94f055 100644
--- a/drivers/net/intel/i40e/base/i40e_osdep.h
+++ b/drivers/net/intel/i40e/base/i40e_osdep.h
@@ -12,7 +12,6 @@
#include <stdarg.h>
#include <rte_common.h>
-#include <rte_memcpy.h>
#include <rte_byteorder.h>
#include <rte_cycles.h>
#include <rte_spinlock.h>
@@ -226,7 +225,7 @@ struct i40e_spinlock {
#define I40E_HTONL(a) rte_cpu_to_be_32(a)
#define i40e_memset(a, b, c, d) memset((a), (b), (c))
-#define i40e_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define i40e_memcpy(a, b, c, d) memcpy((a), (b), (c))
#define DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d))
#define DELAY(x) rte_delay_us_sleep(x)
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 100a751225..adc8502736 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -23,7 +23,6 @@
#include <ethdev_pci.h>
#include <rte_memzone.h>
#include <rte_malloc.h>
-#include <rte_memcpy.h>
#include <rte_alarm.h>
#include <dev_driver.h>
#include <rte_tailq.h>
@@ -4468,7 +4467,7 @@ i40e_macaddr_add(struct rte_eth_dev *dev,
return -EINVAL;
}
- rte_memcpy(&mac_filter.mac_addr, mac_addr, RTE_ETHER_ADDR_LEN);
+ memcpy(&mac_filter.mac_addr, mac_addr, RTE_ETHER_ADDR_LEN);
if (rxmode->offloads & RTE_ETH_RX_OFFLOAD_VLAN_FILTER)
mac_filter.filter_type = I40E_MACVLAN_PERFECT_MATCH;
else
@@ -5335,7 +5334,7 @@ i40e_vsi_vlan_pvid_set(struct i40e_vsi *vsi,
vsi->info.valid_sections =
rte_cpu_to_le_16(I40E_AQ_VSI_PROP_VLAN_VALID);
memset(&ctxt, 0, sizeof(ctxt));
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.seid = vsi->seid;
hw = I40E_VSI_TO_HW(vsi);
@@ -5374,7 +5373,7 @@ i40e_vsi_update_tc_bandwidth(struct i40e_vsi *vsi, uint8_t enabled_tcmap)
return ret;
}
- rte_memcpy(vsi->info.qs_handle, tc_bw_data.qs_handles,
+ memcpy(vsi->info.qs_handle, tc_bw_data.qs_handles,
sizeof(vsi->info.qs_handle));
return I40E_SUCCESS;
}
@@ -5632,7 +5631,7 @@ i40e_update_default_filter_setting(struct i40e_vsi *vsi)
if (vsi->type != I40E_VSI_MAIN)
return I40E_ERR_CONFIG;
memset(&def_filter, 0, sizeof(def_filter));
- rte_memcpy(def_filter.mac_addr, hw->mac.perm_addr,
+ memcpy(def_filter.mac_addr, hw->mac.perm_addr,
ETH_ADDR_LEN);
def_filter.vlan_tag = 0;
def_filter.flags = I40E_AQC_MACVLAN_DEL_PERFECT_MATCH |
@@ -5651,7 +5650,7 @@ i40e_update_default_filter_setting(struct i40e_vsi *vsi)
return I40E_ERR_NO_MEMORY;
}
mac = &f->mac_info.mac_addr;
- rte_memcpy(&mac->addr_bytes, hw->mac.perm_addr,
+ memcpy(&mac->addr_bytes, hw->mac.perm_addr,
ETH_ADDR_LEN);
f->mac_info.filter_type = I40E_MACVLAN_PERFECT_MATCH;
TAILQ_INSERT_TAIL(&vsi->mac_list, f, next);
@@ -5659,7 +5658,7 @@ i40e_update_default_filter_setting(struct i40e_vsi *vsi)
return ret;
}
- rte_memcpy(&filter.mac_addr,
+ memcpy(&filter.mac_addr,
(struct rte_ether_addr *)(hw->mac.perm_addr), ETH_ADDR_LEN);
filter.filter_type = I40E_MACVLAN_PERFECT_MATCH;
return i40e_vsi_add_mac(vsi, &filter);
@@ -6016,15 +6015,15 @@ i40e_vsi_setup(struct i40e_pf *pf,
goto fail_msix_alloc;
}
- rte_memcpy(&vsi->info.tc_mapping, &ctxt.info.tc_mapping,
+ memcpy(&vsi->info.tc_mapping, &ctxt.info.tc_mapping,
sizeof(vsi->info.tc_mapping));
- rte_memcpy(&vsi->info.queue_mapping,
+ memcpy(&vsi->info.queue_mapping,
&ctxt.info.queue_mapping,
sizeof(vsi->info.queue_mapping));
vsi->info.mapping_flags = ctxt.info.mapping_flags;
vsi->info.valid_sections = 0;
- rte_memcpy(pf->dev_addr.addr_bytes, hw->mac.perm_addr,
+ memcpy(pf->dev_addr.addr_bytes, hw->mac.perm_addr,
ETH_ADDR_LEN);
/**
@@ -6168,7 +6167,7 @@ i40e_vsi_setup(struct i40e_pf *pf,
if (vsi->type != I40E_VSI_FDIR) {
/* MAC/VLAN configuration for non-FDIR VSI*/
- rte_memcpy(&filter.mac_addr, &broadcast, RTE_ETHER_ADDR_LEN);
+ memcpy(&filter.mac_addr, &broadcast, RTE_ETHER_ADDR_LEN);
filter.filter_type = I40E_MACVLAN_PERFECT_MATCH;
ret = i40e_vsi_add_mac(vsi, &filter);
@@ -6276,7 +6275,7 @@ i40e_vsi_config_vlan_stripping(struct i40e_vsi *vsi, bool on)
vsi->info.port_vlan_flags &= ~(I40E_AQ_VSI_PVLAN_EMOD_MASK);
vsi->info.port_vlan_flags |= vlan_flags;
ctxt.seid = vsi->seid;
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ret = i40e_aq_update_vsi_params(hw, &ctxt, NULL);
if (ret)
PMD_DRV_LOG(INFO, "Update VSI failed to %s vlan stripping",
@@ -7138,7 +7137,7 @@ i40e_add_macvlan_filters(struct i40e_vsi *vsi,
memset(req_list, 0, ele_buff_size);
for (i = 0; i < actual_num; i++) {
- rte_memcpy(req_list[i].mac_addr,
+ memcpy(req_list[i].mac_addr,
&filter[num + i].macaddr, ETH_ADDR_LEN);
req_list[i].vlan_tag =
rte_cpu_to_le_16(filter[num + i].vlan_id);
@@ -7211,7 +7210,7 @@ i40e_remove_macvlan_filters(struct i40e_vsi *vsi,
memset(req_list, 0, ele_buff_size);
for (i = 0; i < actual_num; i++) {
- rte_memcpy(req_list[i].mac_addr,
+ memcpy(req_list[i].mac_addr,
&filter[num + i].macaddr, ETH_ADDR_LEN);
req_list[i].vlan_tag =
rte_cpu_to_le_16(filter[num + i].vlan_id);
@@ -7363,7 +7362,7 @@ i40e_find_all_vlan_for_mac(struct i40e_vsi *vsi,
"vlan number doesn't match");
return I40E_ERR_PARAM;
}
- rte_memcpy(&mv_f[i].macaddr,
+ memcpy(&mv_f[i].macaddr,
addr, ETH_ADDR_LEN);
mv_f[i].vlan_id =
j * I40E_UINT32_BIT_SIZE + k;
@@ -7392,7 +7391,7 @@ i40e_find_all_mac_for_vlan(struct i40e_vsi *vsi,
PMD_DRV_LOG(ERR, "buffer number not match");
return I40E_ERR_PARAM;
}
- rte_memcpy(&mv_f[i].macaddr, &f->mac_info.mac_addr,
+ memcpy(&mv_f[i].macaddr, &f->mac_info.mac_addr,
ETH_ADDR_LEN);
mv_f[i].vlan_id = vlan;
mv_f[i].filter_type = f->mac_info.filter_type;
@@ -7428,7 +7427,7 @@ i40e_vsi_remove_all_macvlan_filter(struct i40e_vsi *vsi)
i = 0;
if (vsi->vlan_num == 0) {
TAILQ_FOREACH(f, &vsi->mac_list, next) {
- rte_memcpy(&mv_f[i].macaddr,
+ memcpy(&mv_f[i].macaddr,
&f->mac_info.mac_addr, ETH_ADDR_LEN);
mv_f[i].filter_type = f->mac_info.filter_type;
mv_f[i].vlan_id = 0;
@@ -7586,7 +7585,7 @@ i40e_vsi_add_mac(struct i40e_vsi *vsi, struct i40e_mac_filter_info *mac_filter)
for (i = 0; i < vlan_num; i++) {
mv_f[i].filter_type = mac_filter->filter_type;
- rte_memcpy(&mv_f[i].macaddr, &mac_filter->mac_addr,
+ memcpy(&mv_f[i].macaddr, &mac_filter->mac_addr,
ETH_ADDR_LEN);
}
@@ -7609,7 +7608,7 @@ i40e_vsi_add_mac(struct i40e_vsi *vsi, struct i40e_mac_filter_info *mac_filter)
ret = I40E_ERR_NO_MEMORY;
goto DONE;
}
- rte_memcpy(&f->mac_info.mac_addr, &mac_filter->mac_addr,
+ memcpy(&f->mac_info.mac_addr, &mac_filter->mac_addr,
ETH_ADDR_LEN);
f->mac_info.filter_type = mac_filter->filter_type;
TAILQ_INSERT_TAIL(&vsi->mac_list, f, next);
@@ -7656,7 +7655,7 @@ i40e_vsi_delete_mac(struct i40e_vsi *vsi, struct rte_ether_addr *addr)
for (i = 0; i < vlan_num; i++) {
mv_f[i].filter_type = filter_type;
- rte_memcpy(&mv_f[i].macaddr, &f->mac_info.mac_addr,
+ memcpy(&mv_f[i].macaddr, &f->mac_info.mac_addr,
ETH_ADDR_LEN);
}
if (filter_type == I40E_MACVLAN_PERFECT_MATCH ||
@@ -7943,7 +7942,7 @@ i40e_tunnel_filter_convert(
tunnel_filter->input.flags = cld_filter->element.flags;
tunnel_filter->input.tenant_id = cld_filter->element.tenant_id;
tunnel_filter->queue = cld_filter->element.queue_number;
- rte_memcpy(tunnel_filter->input.general_fields,
+ memcpy(tunnel_filter->input.general_fields,
cld_filter->general_fields,
sizeof(cld_filter->general_fields));
@@ -8481,7 +8480,7 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
ip_type = I40E_AQC_ADD_CLOUD_FLAGS_IPV4;
ipv4_addr = rte_be_to_cpu_32(tunnel_filter->ip_addr.ipv4_addr);
ipv4_addr_le = rte_cpu_to_le_32(ipv4_addr);
- rte_memcpy(&cld_filter.element.ipaddr.v4.data,
+ memcpy(&cld_filter.element.ipaddr.v4.data,
&ipv4_addr_le,
sizeof(cld_filter.element.ipaddr.v4.data));
} else {
@@ -8491,7 +8490,7 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
rte_cpu_to_le_32(rte_be_to_cpu_32(
tunnel_filter->ip_addr.ipv6_addr[i]));
}
- rte_memcpy(&cld_filter.element.ipaddr.v6.data,
+ memcpy(&cld_filter.element.ipaddr.v6.data,
&convert_ipv6,
sizeof(cld_filter.element.ipaddr.v6.data));
}
@@ -8733,7 +8732,7 @@ i40e_dev_consistent_tunnel_filter_set(struct i40e_pf *pf,
return -ENOMEM;
}
- rte_memcpy(tunnel, &check_filter, sizeof(check_filter));
+ memcpy(tunnel, &check_filter, sizeof(check_filter));
ret = i40e_sw_tunnel_filter_insert(pf, tunnel);
if (ret < 0)
rte_free(tunnel);
@@ -9855,7 +9854,7 @@ static int
i40e_ethertype_filter_convert(const struct rte_eth_ethertype_filter *input,
struct i40e_ethertype_filter *filter)
{
- rte_memcpy(&filter->input.mac_addr, &input->mac_addr,
+ memcpy(&filter->input.mac_addr, &input->mac_addr,
RTE_ETHER_ADDR_LEN);
filter->input.ether_type = input->ether_type;
filter->flags = input->flags;
@@ -10003,7 +10002,7 @@ i40e_ethertype_filter_set(struct i40e_pf *pf,
return -ENOMEM;
}
- rte_memcpy(ethertype_filter, &check_filter,
+ memcpy(ethertype_filter, &check_filter,
sizeof(check_filter));
ret = i40e_sw_ethertype_filter_insert(pf, ethertype_filter);
if (ret < 0)
@@ -10890,9 +10889,9 @@ i40e_vsi_config_tc(struct i40e_vsi *vsi, uint8_t tc_map)
goto out;
}
/* update the local VSI info with updated queue map */
- rte_memcpy(&vsi->info.tc_mapping, &ctxt.info.tc_mapping,
+ memcpy(&vsi->info.tc_mapping, &ctxt.info.tc_mapping,
sizeof(vsi->info.tc_mapping));
- rte_memcpy(&vsi->info.queue_mapping,
+ memcpy(&vsi->info.queue_mapping,
&ctxt.info.queue_mapping,
sizeof(vsi->info.queue_mapping));
vsi->info.mapping_flags = ctxt.info.mapping_flags;
@@ -11646,7 +11645,7 @@ i40e_tunnel_filter_restore(struct i40e_pf *pf)
cld_filter.element.flags = f->input.flags;
cld_filter.element.tenant_id = f->input.tenant_id;
cld_filter.element.queue_number = f->queue;
- rte_memcpy(cld_filter.general_fields,
+ memcpy(cld_filter.general_fields,
f->input.general_fields,
sizeof(f->input.general_fields));
diff --git a/drivers/net/intel/i40e/i40e_fdir.c b/drivers/net/intel/i40e/i40e_fdir.c
index 3b099d5a9e..ad256a5a11 100644
--- a/drivers/net/intel/i40e/i40e_fdir.c
+++ b/drivers/net/intel/i40e/i40e_fdir.c
@@ -464,9 +464,9 @@ fill_ip6_head(const struct i40e_fdir_input *fdir_input, unsigned char *raw_pkt,
* need to be presented in a reversed order with respect
* to the expected received packets.
*/
- rte_memcpy(&ip6->src_addr, &fdir_input->flow.ipv6_flow.dst_ip,
+ memcpy(&ip6->src_addr, &fdir_input->flow.ipv6_flow.dst_ip,
IPV6_ADDR_LEN);
- rte_memcpy(&ip6->dst_addr, &fdir_input->flow.ipv6_flow.src_ip,
+ memcpy(&ip6->dst_addr, &fdir_input->flow.ipv6_flow.src_ip,
IPV6_ADDR_LEN);
len += sizeof(struct rte_ipv6_hdr);
@@ -528,16 +528,16 @@ i40e_flow_fdir_fill_eth_ip_head(struct i40e_pf *pf,
[I40E_FILTER_PCTYPE_NONF_IPV6_OTHER] = IPPROTO_NONE,
};
- rte_memcpy(raw_pkt, &fdir_input->flow.l2_flow.dst,
+ memcpy(raw_pkt, &fdir_input->flow.l2_flow.dst,
sizeof(struct rte_ether_addr));
- rte_memcpy(raw_pkt + sizeof(struct rte_ether_addr),
+ memcpy(raw_pkt + sizeof(struct rte_ether_addr),
&fdir_input->flow.l2_flow.src,
sizeof(struct rte_ether_addr));
raw_pkt += 2 * sizeof(struct rte_ether_addr);
if (vlan && fdir_input->flow_ext.vlan_tci) {
- rte_memcpy(raw_pkt, vlan_frame, sizeof(vlan_frame));
- rte_memcpy(raw_pkt + sizeof(uint16_t),
+ memcpy(raw_pkt, vlan_frame, sizeof(vlan_frame));
+ memcpy(raw_pkt + sizeof(uint16_t),
&fdir_input->flow_ext.vlan_tci,
sizeof(uint16_t));
raw_pkt += sizeof(vlan_frame);
@@ -901,7 +901,7 @@ i40e_flow_fdir_construct_pkt(struct i40e_pf *pf,
dst = pf->fdir.flex_set[pit_idx].dst_offset * sizeof(uint16_t);
ptr = payload +
pf->fdir.flex_set[pit_idx].src_offset * sizeof(uint16_t);
- (void)rte_memcpy(ptr,
+ (void)memcpy(ptr,
&fdir_input->flow_ext.flexbytes[dst],
size * sizeof(uint16_t));
}
@@ -1001,7 +1001,7 @@ static int
i40e_fdir_filter_convert(const struct i40e_fdir_filter_conf *input,
struct i40e_fdir_filter *filter)
{
- rte_memcpy(&filter->fdir, input, sizeof(struct i40e_fdir_filter_conf));
+ memcpy(&filter->fdir, input, sizeof(struct i40e_fdir_filter_conf));
if (input->input.flow_ext.pkt_template) {
filter->fdir.input.flow.raw_flow.packet = NULL;
filter->fdir.input.flow.raw_flow.length =
@@ -1058,7 +1058,7 @@ i40e_sw_fdir_filter_insert(struct i40e_pf *pf, struct i40e_fdir_filter *filter)
return -1;
hash_filter = &fdir_info->fdir_filter_array[ret];
- rte_memcpy(hash_filter, filter, sizeof(*filter));
+ memcpy(hash_filter, filter, sizeof(*filter));
fdir_info->hash_map[ret] = hash_filter;
TAILQ_INSERT_TAIL(&fdir_info->fdir_list, hash_filter, rules);
diff --git a/drivers/net/intel/i40e/i40e_flow.c b/drivers/net/intel/i40e/i40e_flow.c
index 78191a2d22..1a1f864a22 100644
--- a/drivers/net/intel/i40e/i40e_flow.c
+++ b/drivers/net/intel/i40e/i40e_flow.c
@@ -1159,7 +1159,7 @@ i40e_pattern_skip_void_item(struct rte_flow_item *items,
pe = i40e_find_first_item(pb + 1, true);
cpy_count = pe - pb;
- rte_memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
+ memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
items += cpy_count;
@@ -1171,7 +1171,7 @@ i40e_pattern_skip_void_item(struct rte_flow_item *items,
pb = pe + 1;
}
/* Copy the END item. */
- rte_memcpy(items, pe, sizeof(struct rte_flow_item));
+ memcpy(items, pe, sizeof(struct rte_flow_item));
}
/* Check if the pattern matches a supported item type array */
@@ -2034,9 +2034,9 @@ i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
filter->input.flow_ext.oip_type =
I40E_FDIR_IPTYPE_IPV6;
- rte_memcpy(filter->input.flow.ipv6_flow.src_ip,
+ memcpy(filter->input.flow.ipv6_flow.src_ip,
&ipv6_spec->hdr.src_addr, 16);
- rte_memcpy(filter->input.flow.ipv6_flow.dst_ip,
+ memcpy(filter->input.flow.ipv6_flow.dst_ip,
&ipv6_spec->hdr.dst_addr, 16);
/* Check if it is fragment. */
@@ -2962,12 +2962,12 @@ i40e_flow_parse_vxlan_pattern(__rte_unused struct rte_eth_dev *dev,
}
if (!vxlan_flag) {
- rte_memcpy(&filter->outer_mac,
+ memcpy(&filter->outer_mac,
ð_spec->hdr.dst_addr,
RTE_ETHER_ADDR_LEN);
filter_type |= RTE_ETH_TUNNEL_FILTER_OMAC;
} else {
- rte_memcpy(&filter->inner_mac,
+ memcpy(&filter->inner_mac,
ð_spec->hdr.dst_addr,
RTE_ETHER_ADDR_LEN);
filter_type |= RTE_ETH_TUNNEL_FILTER_IMAC;
@@ -3062,7 +3062,7 @@ i40e_flow_parse_vxlan_pattern(__rte_unused struct rte_eth_dev *dev,
return -rte_errno;
}
- rte_memcpy(((uint8_t *)&tenant_id_be + 1),
+ memcpy(((uint8_t *)&tenant_id_be + 1),
vxlan_spec->hdr.vni, 3);
filter->tenant_id =
rte_be_to_cpu_32(tenant_id_be);
@@ -3191,12 +3191,12 @@ i40e_flow_parse_nvgre_pattern(__rte_unused struct rte_eth_dev *dev,
}
if (!nvgre_flag) {
- rte_memcpy(&filter->outer_mac,
+ memcpy(&filter->outer_mac,
ð_spec->hdr.dst_addr,
RTE_ETHER_ADDR_LEN);
filter_type |= RTE_ETH_TUNNEL_FILTER_OMAC;
} else {
- rte_memcpy(&filter->inner_mac,
+ memcpy(&filter->inner_mac,
ð_spec->hdr.dst_addr,
RTE_ETHER_ADDR_LEN);
filter_type |= RTE_ETH_TUNNEL_FILTER_IMAC;
@@ -3313,7 +3313,7 @@ i40e_flow_parse_nvgre_pattern(__rte_unused struct rte_eth_dev *dev,
"Invalid NVGRE item");
return -rte_errno;
}
- rte_memcpy(((uint8_t *)&tenant_id_be + 1),
+ memcpy(((uint8_t *)&tenant_id_be + 1),
nvgre_spec->tni, 3);
filter->tenant_id =
rte_be_to_cpu_32(tenant_id_be);
@@ -3481,7 +3481,7 @@ i40e_flow_parse_mpls_pattern(__rte_unused struct rte_eth_dev *dev,
"Invalid MPLS label mask");
return -rte_errno;
}
- rte_memcpy(((uint8_t *)&label_be + 1),
+ memcpy(((uint8_t *)&label_be + 1),
mpls_spec->label_tc_s, 3);
filter->tenant_id = rte_be_to_cpu_32(label_be) >> 4;
break;
@@ -4119,7 +4119,7 @@ i40e_flow_destroy_tunnel_filter(struct i40e_pf *pf,
cld_filter.element.flags = filter->input.flags;
cld_filter.element.tenant_id = filter->input.tenant_id;
cld_filter.element.queue_number = filter->queue;
- rte_memcpy(cld_filter.general_fields,
+ memcpy(cld_filter.general_fields,
filter->input.general_fields,
sizeof(cld_filter.general_fields));
@@ -4339,7 +4339,7 @@ i40e_flow_query(struct rte_eth_dev *dev __rte_unused,
"action not supported");
return -rte_errno;
}
- rte_memcpy(rss_conf,
+ memcpy(rss_conf,
&rss_rule->rss_filter_info.conf,
sizeof(struct rte_flow_action_rss));
break;
diff --git a/drivers/net/intel/i40e/i40e_pf.c b/drivers/net/intel/i40e/i40e_pf.c
index 08cdd6bc4d..c76e035a42 100644
--- a/drivers/net/intel/i40e/i40e_pf.c
+++ b/drivers/net/intel/i40e/i40e_pf.c
@@ -16,7 +16,6 @@
#include <rte_ether.h>
#include <ethdev_driver.h>
#include <rte_malloc.h>
-#include <rte_memcpy.h>
#include "i40e_logs.h"
#include "base/i40e_prototype.h"
@@ -857,7 +856,7 @@ i40e_pf_host_process_cmd_add_ether_address(struct i40e_pf_vf *vf,
for (i = 0; i < addr_list->num_elements; i++) {
mac = (struct rte_ether_addr *)(addr_list->list[i].addr);
- rte_memcpy(&filter.mac_addr, mac, RTE_ETHER_ADDR_LEN);
+ memcpy(&filter.mac_addr, mac, RTE_ETHER_ADDR_LEN);
filter.filter_type = I40E_MACVLAN_PERFECT_MATCH;
if (rte_is_zero_ether_addr(mac) ||
i40e_vsi_add_mac(vf->vsi, &filter)) {
diff --git a/drivers/net/intel/i40e/i40e_tm.c b/drivers/net/intel/i40e/i40e_tm.c
index 4c0940f355..6f7e8cc801 100644
--- a/drivers/net/intel/i40e/i40e_tm.c
+++ b/drivers/net/intel/i40e/i40e_tm.c
@@ -279,7 +279,7 @@ i40e_shaper_profile_add(struct rte_eth_dev *dev,
if (!shaper_profile)
return -ENOMEM;
shaper_profile->shaper_profile_id = shaper_profile_id;
- rte_memcpy(&shaper_profile->profile, profile,
+ memcpy(&shaper_profile->profile, profile,
sizeof(struct rte_tm_shaper_params));
TAILQ_INSERT_TAIL(&pf->tm_conf.shaper_profile_list,
shaper_profile, node);
@@ -526,7 +526,7 @@ i40e_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->reference_count = 0;
tm_node->parent = NULL;
tm_node->shaper_profile = shaper_profile;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
pf->tm_conf.root = tm_node;
@@ -600,7 +600,7 @@ i40e_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->reference_count = 0;
tm_node->parent = parent_node;
tm_node->shaper_profile = shaper_profile;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
if (parent_node_type == I40E_TM_NODE_TYPE_PORT) {
TAILQ_INSERT_TAIL(&pf->tm_conf.tc_list,
diff --git a/drivers/net/intel/i40e/rte_pmd_i40e.c b/drivers/net/intel/i40e/rte_pmd_i40e.c
index 4fdef9464b..6049b77d6f 100644
--- a/drivers/net/intel/i40e/rte_pmd_i40e.c
+++ b/drivers/net/intel/i40e/rte_pmd_i40e.c
@@ -94,7 +94,7 @@ rte_pmd_i40e_set_vf_mac_anti_spoof(uint16_t port, uint16_t vf_id, uint8_t on)
vsi->info.sec_flags &= ~I40E_AQ_VSI_SEC_FLAG_ENABLE_MAC_CHK;
memset(&ctxt, 0, sizeof(ctxt));
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.seid = vsi->seid;
hw = I40E_VSI_TO_HW(vsi);
@@ -197,7 +197,7 @@ rte_pmd_i40e_set_vf_vlan_anti_spoof(uint16_t port, uint16_t vf_id, uint8_t on)
vsi->info.sec_flags &= ~I40E_AQ_VSI_SEC_FLAG_ENABLE_VLAN_CHK;
memset(&ctxt, 0, sizeof(ctxt));
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.seid = vsi->seid;
hw = I40E_VSI_TO_HW(vsi);
@@ -242,7 +242,7 @@ i40e_vsi_rm_mac_filter(struct i40e_vsi *vsi)
for (i = 0; i < vlan_num; i++) {
mv_f[i].filter_type = filter_type;
- rte_memcpy(&mv_f[i].macaddr,
+ memcpy(&mv_f[i].macaddr,
&f->mac_info.mac_addr,
ETH_ADDR_LEN);
}
@@ -303,7 +303,7 @@ i40e_vsi_restore_mac_filter(struct i40e_vsi *vsi)
for (i = 0; i < vlan_num; i++) {
mv_f[i].filter_type = f->mac_info.filter_type;
- rte_memcpy(&mv_f[i].macaddr,
+ memcpy(&mv_f[i].macaddr,
&f->mac_info.mac_addr,
ETH_ADDR_LEN);
}
@@ -385,7 +385,7 @@ i40e_vsi_set_tx_loopback(struct i40e_vsi *vsi, uint8_t on)
vsi->info.switch_id &= ~I40E_AQ_VSI_SW_ID_FLAG_ALLOW_LB;
memset(&ctxt, 0, sizeof(ctxt));
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.seid = vsi->seid;
ret = i40e_aq_update_vsi_params(hw, &ctxt, NULL);
@@ -716,7 +716,7 @@ int rte_pmd_i40e_set_vf_vlan_insert(uint16_t port, uint16_t vf_id,
vsi->info.port_vlan_flags &= ~I40E_AQ_VSI_PVLAN_INSERT_PVID;
memset(&ctxt, 0, sizeof(ctxt));
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.seid = vsi->seid;
hw = I40E_VSI_TO_HW(vsi);
@@ -779,7 +779,7 @@ int rte_pmd_i40e_set_vf_broadcast(uint16_t port, uint16_t vf_id,
}
if (on) {
- rte_memcpy(&filter.mac_addr, &broadcast, RTE_ETHER_ADDR_LEN);
+ memcpy(&filter.mac_addr, &broadcast, RTE_ETHER_ADDR_LEN);
filter.filter_type = I40E_MACVLAN_PERFECT_MATCH;
ret = i40e_vsi_add_mac(vsi, &filter);
} else {
@@ -852,7 +852,7 @@ int rte_pmd_i40e_set_vf_vlan_tag(uint16_t port, uint16_t vf_id, uint8_t on)
}
memset(&ctxt, 0, sizeof(ctxt));
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.seid = vsi->seid;
hw = I40E_VSI_TO_HW(vsi);
@@ -2591,9 +2591,9 @@ i40e_vsi_update_queue_region_mapping(struct i40e_hw *hw,
return ret;
}
/* update the local VSI info with updated queue map */
- rte_memcpy(&vsi->info.tc_mapping, &ctxt.info.tc_mapping,
+ memcpy(&vsi->info.tc_mapping, &ctxt.info.tc_mapping,
sizeof(vsi->info.tc_mapping));
- rte_memcpy(&vsi->info.queue_mapping,
+ memcpy(&vsi->info.queue_mapping,
&ctxt.info.queue_mapping,
sizeof(vsi->info.queue_mapping));
vsi->info.mapping_flags = ctxt.info.mapping_flags;
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 2/8] net/iavf: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:05 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, for example, flow
configuration and so is fine using the standard libc memcpy function in
place of the DPDK-specific rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/iavf/base/iavf_osdep.h | 3 +-
drivers/net/intel/iavf/iavf_ethdev.c | 12 +++---
drivers/net/intel/iavf/iavf_fdir.c | 46 +++++++++++-----------
drivers/net/intel/iavf/iavf_fsub.c | 26 ++++++------
drivers/net/intel/iavf/iavf_generic_flow.c | 4 +-
drivers/net/intel/iavf/iavf_hash.c | 4 +-
drivers/net/intel/iavf/iavf_tm.c | 6 +--
drivers/net/intel/iavf/iavf_vchnl.c | 14 +++----
8 files changed, 57 insertions(+), 58 deletions(-)
diff --git a/drivers/net/intel/iavf/base/iavf_osdep.h b/drivers/net/intel/iavf/base/iavf_osdep.h
index 1b0ca933cc..cb95b848b6 100644
--- a/drivers/net/intel/iavf/base/iavf_osdep.h
+++ b/drivers/net/intel/iavf/base/iavf_osdep.h
@@ -12,7 +12,6 @@
#include <stdarg.h>
#include <rte_common.h>
-#include <rte_memcpy.h>
#include <rte_memzone.h>
#include <rte_malloc.h>
#include <rte_byteorder.h>
@@ -124,7 +123,7 @@ writeq(uint64_t value, volatile void *addr)
#endif /* __INTEL_NET_BASE_OSDEP__ */
#define iavf_memset(a, b, c, d) memset((a), (b), (c))
-#define iavf_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define iavf_memcpy(a, b, c, d) memcpy((a), (b), (c))
#define iavf_usec_delay(x) rte_delay_us_sleep(x)
#define iavf_msec_delay(x) iavf_usec_delay(1000 * (x))
diff --git a/drivers/net/intel/iavf/iavf_ethdev.c b/drivers/net/intel/iavf/iavf_ethdev.c
index 1eca20bc9a..ea1ecc1d9b 100644
--- a/drivers/net/intel/iavf/iavf_ethdev.c
+++ b/drivers/net/intel/iavf/iavf_ethdev.c
@@ -553,7 +553,7 @@ iavf_init_rss(struct iavf_adapter *adapter)
for (i = 0; i < vf->vf_res->rss_key_size; i++)
vf->rss_key[i] = (uint8_t)rte_rand();
} else
- rte_memcpy(vf->rss_key, rss_conf->rss_key,
+ memcpy(vf->rss_key, rss_conf->rss_key,
RTE_MIN(rss_conf->rss_key_len,
vf->vf_res->rss_key_size));
@@ -1536,7 +1536,7 @@ iavf_dev_rss_reta_update(struct rte_eth_dev *dev,
return -ENOMEM;
}
/* store the old lut table temporarily */
- rte_memcpy(lut, vf->rss_lut, reta_size);
+ memcpy(lut, vf->rss_lut, reta_size);
for (i = 0; i < reta_size; i++) {
idx = i / RTE_ETH_RETA_GROUP_SIZE;
@@ -1545,11 +1545,11 @@ iavf_dev_rss_reta_update(struct rte_eth_dev *dev,
lut[i] = reta_conf[idx].reta[shift];
}
- rte_memcpy(vf->rss_lut, lut, reta_size);
+ memcpy(vf->rss_lut, lut, reta_size);
/* send virtchnl ops to configure RSS */
ret = iavf_configure_rss_lut(adapter);
if (ret) /* revert back */
- rte_memcpy(vf->rss_lut, lut, reta_size);
+ memcpy(vf->rss_lut, lut, reta_size);
free(lut);
return ret;
@@ -1605,7 +1605,7 @@ iavf_set_rss_key(struct iavf_adapter *adapter, uint8_t *key, uint8_t key_len)
return -EINVAL;
}
- rte_memcpy(vf->rss_key, key, key_len);
+ memcpy(vf->rss_key, key, key_len);
return iavf_configure_rss_key(adapter);
}
@@ -1689,7 +1689,7 @@ iavf_dev_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
rss_conf->rss_key_len = vf->vf_res->rss_key_size;
- rte_memcpy(rss_conf->rss_key, vf->rss_key, rss_conf->rss_key_len);
+ memcpy(rss_conf->rss_key, vf->rss_key, rss_conf->rss_key_len);
return 0;
}
diff --git a/drivers/net/intel/iavf/iavf_fdir.c b/drivers/net/intel/iavf/iavf_fdir.c
index 9eae874800..08724d78a8 100644
--- a/drivers/net/intel/iavf/iavf_fdir.c
+++ b/drivers/net/intel/iavf/iavf_fdir.c
@@ -374,7 +374,7 @@ iavf_fdir_create(struct iavf_adapter *ad,
if (filter->mark_flag == 1)
iavf_fdir_rx_proc_enable(ad, 1);
- rte_memcpy(rule, filter, sizeof(*rule));
+ memcpy(rule, filter, sizeof(*rule));
flow->rule = rule;
return 0;
@@ -672,14 +672,14 @@ iavf_fdir_refine_input_set(const uint64_t input_set,
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, IPV4, PROT);
memset(&ipv4_spec, 0, sizeof(ipv4_spec));
ipv4_spec.hdr.next_proto_id = proto_id;
- rte_memcpy(hdr->buffer, &ipv4_spec.hdr,
+ memcpy(hdr->buffer, &ipv4_spec.hdr,
sizeof(ipv4_spec.hdr));
return true;
case VIRTCHNL_PROTO_HDR_IPV6:
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, IPV6, PROT);
memset(&ipv6_spec, 0, sizeof(ipv6_spec));
ipv6_spec.hdr.proto = proto_id;
- rte_memcpy(hdr->buffer, &ipv6_spec.hdr,
+ memcpy(hdr->buffer, &ipv6_spec.hdr,
sizeof(ipv6_spec.hdr));
return true;
default:
@@ -894,7 +894,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
ETHERTYPE);
}
- rte_memcpy(hdr1->buffer, eth_spec,
+ memcpy(hdr1->buffer, eth_spec,
sizeof(struct rte_ether_hdr));
}
@@ -985,7 +985,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
input_set |= IAVF_PROT_IPV4_INNER;
}
- rte_memcpy(hdr->buffer, &ipv4_spec->hdr,
+ memcpy(hdr->buffer, &ipv4_spec->hdr,
sizeof(ipv4_spec->hdr));
hdrs->count = ++layer;
@@ -1075,7 +1075,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
input_set |= IAVF_PROT_IPV6_INNER;
}
- rte_memcpy(hdr->buffer, &ipv6_spec->hdr,
+ memcpy(hdr->buffer, &ipv6_spec->hdr,
sizeof(ipv6_spec->hdr));
hdrs->count = ++layer;
@@ -1110,7 +1110,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr1, ETH,
ETHERTYPE);
- rte_memcpy(hdr->buffer, &ipv6_frag_spec->hdr,
+ memcpy(hdr->buffer, &ipv6_frag_spec->hdr,
sizeof(ipv6_frag_spec->hdr));
} else if (ipv6_frag_mask->hdr.id == UINT32_MAX) {
rte_flow_error_set(error, EINVAL,
@@ -1162,11 +1162,11 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
}
if (l3 == RTE_FLOW_ITEM_TYPE_IPV4)
- rte_memcpy(hdr->buffer,
+ memcpy(hdr->buffer,
&udp_spec->hdr,
sizeof(udp_spec->hdr));
else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6)
- rte_memcpy(hdr->buffer,
+ memcpy(hdr->buffer,
&udp_spec->hdr,
sizeof(udp_spec->hdr));
}
@@ -1219,11 +1219,11 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
}
if (l3 == RTE_FLOW_ITEM_TYPE_IPV4)
- rte_memcpy(hdr->buffer,
+ memcpy(hdr->buffer,
&tcp_spec->hdr,
sizeof(tcp_spec->hdr));
else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6)
- rte_memcpy(hdr->buffer,
+ memcpy(hdr->buffer,
&tcp_spec->hdr,
sizeof(tcp_spec->hdr));
}
@@ -1265,11 +1265,11 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
}
if (l3 == RTE_FLOW_ITEM_TYPE_IPV4)
- rte_memcpy(hdr->buffer,
+ memcpy(hdr->buffer,
&sctp_spec->hdr,
sizeof(sctp_spec->hdr));
else if (l3 == RTE_FLOW_ITEM_TYPE_IPV6)
- rte_memcpy(hdr->buffer,
+ memcpy(hdr->buffer,
&sctp_spec->hdr,
sizeof(sctp_spec->hdr));
}
@@ -1300,7 +1300,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, GTPU_IP, TEID);
}
- rte_memcpy(hdr->buffer,
+ memcpy(hdr->buffer,
gtp_spec, sizeof(*gtp_spec));
}
@@ -1355,7 +1355,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
psc.qfi = gtp_psc_spec->hdr.qfi;
psc.type = gtp_psc_spec->hdr.type;
psc.next = 0;
- rte_memcpy(hdr->buffer, &psc,
+ memcpy(hdr->buffer, &psc,
sizeof(struct iavf_gtp_psc_spec_hdr));
}
@@ -1376,7 +1376,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, L2TPV3, SESS_ID);
}
- rte_memcpy(hdr->buffer, l2tpv3oip_spec,
+ memcpy(hdr->buffer, l2tpv3oip_spec,
sizeof(*l2tpv3oip_spec));
}
@@ -1397,7 +1397,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, ESP, SPI);
}
- rte_memcpy(hdr->buffer, &esp_spec->hdr,
+ memcpy(hdr->buffer, &esp_spec->hdr,
sizeof(esp_spec->hdr));
}
@@ -1418,7 +1418,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, AH, SPI);
}
- rte_memcpy(hdr->buffer, ah_spec,
+ memcpy(hdr->buffer, ah_spec,
sizeof(*ah_spec));
}
@@ -1439,7 +1439,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, PFCP, S_FIELD);
}
- rte_memcpy(hdr->buffer, pfcp_spec,
+ memcpy(hdr->buffer, pfcp_spec,
sizeof(*pfcp_spec));
}
@@ -1464,7 +1464,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
PC_RTC_ID);
}
- rte_memcpy(hdr->buffer, ecpri_spec,
+ memcpy(hdr->buffer, ecpri_spec,
sizeof(*ecpri_spec));
}
@@ -1480,7 +1480,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, GRE);
if (gre_spec && gre_mask) {
- rte_memcpy(hdr->buffer, gre_spec,
+ memcpy(hdr->buffer, gre_spec,
sizeof(*gre_spec));
}
@@ -1529,7 +1529,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
SESS_ID);
}
- rte_memcpy(hdr->buffer, l2tpv2_spec,
+ memcpy(hdr->buffer, l2tpv2_spec,
sizeof(*l2tpv2_spec));
}
@@ -1547,7 +1547,7 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
VIRTCHNL_SET_PROTO_HDR_TYPE(hdr, PPP);
if (ppp_spec && ppp_mask) {
- rte_memcpy(hdr->buffer, ppp_spec,
+ memcpy(hdr->buffer, ppp_spec,
sizeof(*ppp_spec));
}
diff --git a/drivers/net/intel/iavf/iavf_fsub.c b/drivers/net/intel/iavf/iavf_fsub.c
index bfb34695de..50046af96a 100644
--- a/drivers/net/intel/iavf/iavf_fsub.c
+++ b/drivers/net/intel/iavf/iavf_fsub.c
@@ -92,7 +92,7 @@ iavf_fsub_create(struct iavf_adapter *ad, struct rte_flow *flow,
goto free_entry;
}
- rte_memcpy(rule, filter, sizeof(*rule));
+ memcpy(rule, filter, sizeof(*rule));
flow->rule = rule;
rte_free(meta);
@@ -280,9 +280,9 @@ iavf_fsub_parse_pattern(const struct rte_flow_item pattern[],
input_set_byte += 2;
}
- rte_memcpy(hdr1->buffer_spec, eth_spec,
+ memcpy(hdr1->buffer_spec, eth_spec,
sizeof(struct rte_ether_hdr));
- rte_memcpy(hdr1->buffer_mask, eth_mask,
+ memcpy(hdr1->buffer_mask, eth_mask,
sizeof(struct rte_ether_hdr));
} else {
/* flow subscribe filter will add dst mac in kernel */
@@ -333,9 +333,9 @@ iavf_fsub_parse_pattern(const struct rte_flow_item pattern[],
input_set_byte++;
}
- rte_memcpy(hdr->buffer_spec, &ipv4_spec->hdr,
+ memcpy(hdr->buffer_spec, &ipv4_spec->hdr,
sizeof(ipv4_spec->hdr));
- rte_memcpy(hdr->buffer_mask, &ipv4_mask->hdr,
+ memcpy(hdr->buffer_mask, &ipv4_mask->hdr,
sizeof(ipv4_spec->hdr));
}
@@ -396,9 +396,9 @@ iavf_fsub_parse_pattern(const struct rte_flow_item pattern[],
input_set_byte += 4;
}
- rte_memcpy(hdr->buffer_spec, &ipv6_spec->hdr,
+ memcpy(hdr->buffer_spec, &ipv6_spec->hdr,
sizeof(ipv6_spec->hdr));
- rte_memcpy(hdr->buffer_mask, &ipv6_mask->hdr,
+ memcpy(hdr->buffer_mask, &ipv6_mask->hdr,
sizeof(ipv6_spec->hdr));
}
@@ -433,9 +433,9 @@ iavf_fsub_parse_pattern(const struct rte_flow_item pattern[],
input_set_byte += 2;
}
- rte_memcpy(hdr->buffer_spec, &udp_spec->hdr,
+ memcpy(hdr->buffer_spec, &udp_spec->hdr,
sizeof(udp_spec->hdr));
- rte_memcpy(hdr->buffer_mask, &udp_mask->hdr,
+ memcpy(hdr->buffer_mask, &udp_mask->hdr,
sizeof(udp_mask->hdr));
}
@@ -474,9 +474,9 @@ iavf_fsub_parse_pattern(const struct rte_flow_item pattern[],
input_set_byte += 2;
}
- rte_memcpy(hdr->buffer_spec, &tcp_spec->hdr,
+ memcpy(hdr->buffer_spec, &tcp_spec->hdr,
sizeof(tcp_spec->hdr));
- rte_memcpy(hdr->buffer_mask, &tcp_mask->hdr,
+ memcpy(hdr->buffer_mask, &tcp_mask->hdr,
sizeof(tcp_mask->hdr));
}
@@ -506,9 +506,9 @@ iavf_fsub_parse_pattern(const struct rte_flow_item pattern[],
return -rte_errno;
}
- rte_memcpy(hdr->buffer_spec, &vlan_spec->hdr,
+ memcpy(hdr->buffer_spec, &vlan_spec->hdr,
sizeof(vlan_spec->hdr));
- rte_memcpy(hdr->buffer_mask, &vlan_mask->hdr,
+ memcpy(hdr->buffer_mask, &vlan_mask->hdr,
sizeof(vlan_mask->hdr));
}
diff --git a/drivers/net/intel/iavf/iavf_generic_flow.c b/drivers/net/intel/iavf/iavf_generic_flow.c
index 42ecc90d1d..2a9f3a8acb 100644
--- a/drivers/net/intel/iavf/iavf_generic_flow.c
+++ b/drivers/net/intel/iavf/iavf_generic_flow.c
@@ -2016,7 +2016,7 @@ iavf_pattern_skip_void_item(struct rte_flow_item *items,
pe = iavf_find_first_item(pb + 1, true);
cpy_count = pe - pb;
- rte_memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
+ memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
items += cpy_count;
@@ -2026,7 +2026,7 @@ iavf_pattern_skip_void_item(struct rte_flow_item *items,
pb = pe + 1;
}
/* Copy the END item. */
- rte_memcpy(items, pe, sizeof(struct rte_flow_item));
+ memcpy(items, pe, sizeof(struct rte_flow_item));
}
/* Check if the pattern matches a supported item type array */
diff --git a/drivers/net/intel/iavf/iavf_hash.c b/drivers/net/intel/iavf/iavf_hash.c
index cb10eeab78..e9ea2a8401 100644
--- a/drivers/net/intel/iavf/iavf_hash.c
+++ b/drivers/net/intel/iavf/iavf_hash.c
@@ -953,8 +953,8 @@ iavf_hash_parse_raw_pattern(const struct rte_flow_item *item,
msk_buf[j] = tmp_val * 16 + tmp_c - '0';
}
- rte_memcpy(meta->proto_hdrs.raw.spec, pkt_buf, pkt_len);
- rte_memcpy(meta->proto_hdrs.raw.mask, msk_buf, pkt_len);
+ memcpy(meta->proto_hdrs.raw.spec, pkt_buf, pkt_len);
+ memcpy(meta->proto_hdrs.raw.mask, msk_buf, pkt_len);
meta->proto_hdrs.raw.pkt_len = pkt_len;
rte_free(pkt_buf);
diff --git a/drivers/net/intel/iavf/iavf_tm.c b/drivers/net/intel/iavf/iavf_tm.c
index 1d12196ba6..54ec6e9750 100644
--- a/drivers/net/intel/iavf/iavf_tm.c
+++ b/drivers/net/intel/iavf/iavf_tm.c
@@ -342,7 +342,7 @@ iavf_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->id = node_id;
tm_node->parent = NULL;
tm_node->reference_count = 0;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
vf->tm_conf.root = tm_node;
return 0;
@@ -403,7 +403,7 @@ iavf_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->reference_count = 0;
tm_node->parent = parent_node;
tm_node->shaper_profile = shaper_profile;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
if (parent_node_type == IAVF_TM_NODE_TYPE_PORT) {
TAILQ_INSERT_TAIL(&vf->tm_conf.tc_list,
@@ -543,7 +543,7 @@ iavf_shaper_profile_add(struct rte_eth_dev *dev,
if (!shaper_profile)
return -ENOMEM;
shaper_profile->shaper_profile_id = shaper_profile_id;
- rte_memcpy(&shaper_profile->profile, profile,
+ memcpy(&shaper_profile->profile, profile,
sizeof(struct rte_tm_shaper_params));
TAILQ_INSERT_TAIL(&vf->tm_conf.shaper_profile_list,
shaper_profile, node);
diff --git a/drivers/net/intel/iavf/iavf_vchnl.c b/drivers/net/intel/iavf/iavf_vchnl.c
index 08dd6f2d7f..14bd869be4 100644
--- a/drivers/net/intel/iavf/iavf_vchnl.c
+++ b/drivers/net/intel/iavf/iavf_vchnl.c
@@ -112,7 +112,7 @@ iavf_dev_event_post(struct rte_eth_dev *dev,
elem->param = param;
elem->param_alloc_size = param_alloc_size;
if (param && param_alloc_size) {
- rte_memcpy(elem->param_alloc_data, param, param_alloc_size);
+ memcpy(elem->param_alloc_data, param, param_alloc_size);
elem->param = elem->param_alloc_data;
}
@@ -732,7 +732,7 @@ iavf_get_vf_resource(struct iavf_adapter *adapter)
len = sizeof(struct virtchnl_vf_resource) +
IAVF_MAX_VF_VSI * sizeof(struct virtchnl_vsi_resource);
- rte_memcpy(vf->vf_res, args.out_buffer,
+ memcpy(vf->vf_res, args.out_buffer,
RTE_MIN(args.out_size, len));
/* parse VF config message back from PF*/
iavf_vf_parse_hw_config(hw, vf->vf_res);
@@ -921,7 +921,7 @@ iavf_get_vlan_offload_caps_v2(struct iavf_adapter *adapter)
return ret;
}
- rte_memcpy(&vf->vlan_v2_caps, vf->aq_resp, sizeof(vf->vlan_v2_caps));
+ memcpy(&vf->vlan_v2_caps, vf->aq_resp, sizeof(vf->vlan_v2_caps));
return 0;
}
@@ -1151,7 +1151,7 @@ iavf_configure_rss_lut(struct iavf_adapter *adapter)
rss_lut->vsi_id = vf->vsi_res->vsi_id;
rss_lut->lut_entries = vf->vf_res->rss_lut_size;
- rte_memcpy(rss_lut->lut, vf->rss_lut, vf->vf_res->rss_lut_size);
+ memcpy(rss_lut->lut, vf->rss_lut, vf->vf_res->rss_lut_size);
args.ops = VIRTCHNL_OP_CONFIG_RSS_LUT;
args.in_args = (u8 *)rss_lut;
@@ -1183,7 +1183,7 @@ iavf_configure_rss_key(struct iavf_adapter *adapter)
rss_key->vsi_id = vf->vsi_res->vsi_id;
rss_key->key_len = vf->vf_res->rss_key_size;
- rte_memcpy(rss_key->key, vf->rss_key, vf->vf_res->rss_key_size);
+ memcpy(rss_key->key, vf->rss_key, vf->vf_res->rss_key_size);
args.ops = VIRTCHNL_OP_CONFIG_RSS_KEY;
args.in_args = (u8 *)rss_key;
@@ -1574,7 +1574,7 @@ iavf_add_del_eth_addr(struct iavf_adapter *adapter, struct rte_ether_addr *addr,
list->vsi_id = vf->vsi_res->vsi_id;
list->num_elements = 1;
list->list[0].type = type;
- rte_memcpy(list->list[0].addr, addr->addr_bytes,
+ memcpy(list->list[0].addr, addr->addr_bytes,
sizeof(addr->addr_bytes));
args.ops = add ? VIRTCHNL_OP_ADD_ETH_ADDR : VIRTCHNL_OP_DEL_ETH_ADDR;
@@ -2001,7 +2001,7 @@ iavf_get_qos_cap(struct iavf_adapter *adapter)
len = sizeof(struct virtchnl_qos_cap_list) +
IAVF_MAX_TRAFFIC_CLASS * sizeof(struct virtchnl_qos_cap_elem);
- rte_memcpy(vf->qos_cap, args.out_buffer,
+ memcpy(vf->qos_cap, args.out_buffer,
RTE_MIN(args.out_size, len));
return 0;
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 1/8] net/ice: replace use of DPDK-specific memcpy function
From: Bruce Richardson @ 2026-05-19 16:05 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260519160823.1004973-1-bruce.richardson@intel.com>
The driver use of memcpy is for control path work, for example, flow
configuration and so is fine using the standard libc memcpy function in
place of the DPDK-specific rte_memcpy version.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
drivers/net/intel/ice/base/ice_osdep.h | 5 ++--
drivers/net/intel/ice/ice_dcf.c | 14 ++++-----
drivers/net/intel/ice/ice_dcf_ethdev.c | 10 +++----
drivers/net/intel/ice/ice_dcf_parent.c | 2 +-
drivers/net/intel/ice/ice_dcf_sched.c | 6 ++--
drivers/net/intel/ice/ice_diagnose.c | 4 +--
drivers/net/intel/ice/ice_ethdev.c | 12 ++++----
drivers/net/intel/ice/ice_fdir_filter.c | 36 ++++++++++++------------
drivers/net/intel/ice/ice_generic_flow.c | 4 +--
drivers/net/intel/ice/ice_hash.c | 2 +-
drivers/net/intel/ice/ice_tm.c | 2 +-
11 files changed, 48 insertions(+), 49 deletions(-)
diff --git a/drivers/net/intel/ice/base/ice_osdep.h b/drivers/net/intel/ice/base/ice_osdep.h
index 21c460d007..5d92cf5aa4 100644
--- a/drivers/net/intel/ice/base/ice_osdep.h
+++ b/drivers/net/intel/ice/base/ice_osdep.h
@@ -14,7 +14,6 @@
#include <stdbool.h>
#include <rte_common.h>
-#include <rte_memcpy.h>
#include <rte_malloc.h>
#include <rte_memzone.h>
#include <rte_byteorder.h>
@@ -210,7 +209,7 @@ struct __rte_packed_begin ice_virt_mem {
#define ice_free(h, m) ((void)h, rte_free(m))
#define ice_memset(a, b, c, d) memset((a), (b), (c))
-#define ice_memcpy(a, b, c, d) rte_memcpy((a), (b), (c))
+#define ice_memcpy(a, b, c, d) memcpy((a), (b), (c))
/* Memory fence barrier */
#define ice_memfence_read() rte_io_rmb()
@@ -238,7 +237,7 @@ ice_memdup(__rte_unused struct ice_hw *hw, const void *src, size_t size,
p = ice_malloc(hw, size);
if (p)
- rte_memcpy(p, src, size);
+ memcpy(p, src, size);
return p;
}
diff --git a/drivers/net/intel/ice/ice_dcf.c b/drivers/net/intel/ice/ice_dcf.c
index 51716a4d5b..98ae0f8980 100644
--- a/drivers/net/intel/ice/ice_dcf.c
+++ b/drivers/net/intel/ice/ice_dcf.c
@@ -268,7 +268,7 @@ ice_dcf_aq_cmd_handle(struct ice_dcf_hw *hw, struct iavf_arq_event_info *info)
cmd->rsp_msglen = RTE_MIN(info->msg_len,
cmd->rsp_buflen);
if (likely(cmd->rsp_msglen != 0))
- rte_memcpy(cmd->rsp_msgbuf, info->msg_buf,
+ memcpy(cmd->rsp_msgbuf, info->msg_buf,
cmd->rsp_msglen);
/* prevent compiler reordering */
@@ -453,7 +453,7 @@ ice_dcf_get_vf_vsi_map(struct ice_dcf_hw *hw)
return 1;
}
- rte_memcpy(hw->vf_vsi_map, vsi_map->vf_vsi, len);
+ memcpy(hw->vf_vsi_map, vsi_map->vf_vsi, len);
return 0;
}
@@ -731,7 +731,7 @@ dcf_get_vlan_offload_caps_v2(struct ice_dcf_hw *hw)
return ret;
}
- rte_memcpy(&hw->vlan_v2_caps, &vlan_v2_caps, sizeof(vlan_v2_caps));
+ memcpy(&hw->vlan_v2_caps, &vlan_v2_caps, sizeof(vlan_v2_caps));
return 0;
}
@@ -931,7 +931,7 @@ ice_dcf_configure_rss_key(struct ice_dcf_hw *hw)
rss_key->vsi_id = hw->vsi_res->vsi_id;
rss_key->key_len = hw->vf_res->rss_key_size;
- rte_memcpy(rss_key->key, hw->rss_key, hw->vf_res->rss_key_size);
+ memcpy(rss_key->key, hw->rss_key, hw->vf_res->rss_key_size);
args.v_op = VIRTCHNL_OP_CONFIG_RSS_KEY;
args.req_msglen = len;
@@ -963,7 +963,7 @@ ice_dcf_configure_rss_lut(struct ice_dcf_hw *hw)
rss_lut->vsi_id = hw->vsi_res->vsi_id;
rss_lut->lut_entries = hw->vf_res->rss_lut_size;
- rte_memcpy(rss_lut->lut, hw->rss_lut, hw->vf_res->rss_lut_size);
+ memcpy(rss_lut->lut, hw->rss_lut, hw->vf_res->rss_lut_size);
args.v_op = VIRTCHNL_OP_CONFIG_RSS_LUT;
args.req_msglen = len;
@@ -1130,7 +1130,7 @@ ice_dcf_init_rss(struct ice_dcf_hw *hw)
for (i = 0; i < hw->vf_res->rss_key_size; i++)
hw->rss_key[i] = (uint8_t)rte_rand();
else
- rte_memcpy(hw->rss_key, rss_conf->rss_key,
+ memcpy(hw->rss_key, rss_conf->rss_key,
RTE_MIN(rss_conf->rss_key_len,
hw->vf_res->rss_key_size));
@@ -1406,7 +1406,7 @@ ice_dcf_add_del_all_mac_addr(struct ice_dcf_hw *hw,
return -ENOMEM;
}
- rte_memcpy(list->list[0].addr, addr->addr_bytes,
+ memcpy(list->list[0].addr, addr->addr_bytes,
sizeof(addr->addr_bytes));
PMD_DRV_LOG(DEBUG, "add/rm mac:" RTE_ETHER_ADDR_PRT_FMT,
diff --git a/drivers/net/intel/ice/ice_dcf_ethdev.c b/drivers/net/intel/ice/ice_dcf_ethdev.c
index 0d3599d7d0..4fce59617e 100644
--- a/drivers/net/intel/ice/ice_dcf_ethdev.c
+++ b/drivers/net/intel/ice/ice_dcf_ethdev.c
@@ -1336,7 +1336,7 @@ ice_dcf_dev_rss_reta_update(struct rte_eth_dev *dev,
return -ENOMEM;
}
/* store the old lut table temporarily */
- rte_memcpy(lut, hw->rss_lut, reta_size);
+ memcpy(lut, hw->rss_lut, reta_size);
for (i = 0; i < reta_size; i++) {
idx = i / RTE_ETH_RETA_GROUP_SIZE;
@@ -1345,11 +1345,11 @@ ice_dcf_dev_rss_reta_update(struct rte_eth_dev *dev,
lut[i] = reta_conf[idx].reta[shift];
}
- rte_memcpy(hw->rss_lut, lut, reta_size);
+ memcpy(hw->rss_lut, lut, reta_size);
/* send virtchnnl ops to configure rss*/
ret = ice_dcf_configure_rss_lut(hw);
if (ret) /* revert back */
- rte_memcpy(hw->rss_lut, lut, reta_size);
+ memcpy(hw->rss_lut, lut, reta_size);
free(lut);
return ret;
@@ -1399,7 +1399,7 @@ ice_dcf_set_rss_key(struct ice_dcf_hw *hw, uint8_t *key, uint8_t key_len)
return -EINVAL;
}
- rte_memcpy(hw->rss_key, key, key_len);
+ memcpy(hw->rss_key, key, key_len);
return ice_dcf_configure_rss_key(hw);
}
@@ -1456,7 +1456,7 @@ ice_dcf_dev_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
rss_conf->rss_key_len = hw->vf_res->rss_key_size;
- rte_memcpy(rss_conf->rss_key, hw->rss_key, rss_conf->rss_key_len);
+ memcpy(rss_conf->rss_key, hw->rss_key, rss_conf->rss_key_len);
return 0;
}
diff --git a/drivers/net/intel/ice/ice_dcf_parent.c b/drivers/net/intel/ice/ice_dcf_parent.c
index f92bd5e726..2c7d94b50d 100644
--- a/drivers/net/intel/ice/ice_dcf_parent.c
+++ b/drivers/net/intel/ice/ice_dcf_parent.c
@@ -418,7 +418,7 @@ ice_dcf_load_pkg(struct ice_adapter *adapter)
use_dsn = ice_dcf_execute_virtchnl_cmd(&dcf_adapter->real_hw, &vc_cmd) == 0;
if (use_dsn)
- rte_memcpy(&dsn, pkg_info.dsn, sizeof(dsn));
+ memcpy(&dsn, pkg_info.dsn, sizeof(dsn));
return ice_load_pkg(adapter, use_dsn, dsn);
}
diff --git a/drivers/net/intel/ice/ice_dcf_sched.c b/drivers/net/intel/ice/ice_dcf_sched.c
index 948774a282..19185ca308 100644
--- a/drivers/net/intel/ice/ice_dcf_sched.c
+++ b/drivers/net/intel/ice/ice_dcf_sched.c
@@ -310,7 +310,7 @@ ice_dcf_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->id = node_id;
tm_node->parent = NULL;
tm_node->reference_count = 0;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
hw->tm_conf.root = tm_node;
@@ -375,7 +375,7 @@ ice_dcf_node_add(struct rte_eth_dev *dev, uint32_t node_id,
tm_node->shaper_profile = shaper_profile;
tm_node->reference_count = 0;
tm_node->parent = parent_node;
- rte_memcpy(&tm_node->params, params,
+ memcpy(&tm_node->params, params,
sizeof(struct rte_tm_node_params));
if (parent_node_type == ICE_DCF_TM_NODE_TYPE_PORT) {
TAILQ_INSERT_TAIL(&hw->tm_conf.tc_list,
@@ -522,7 +522,7 @@ ice_dcf_shaper_profile_add(struct rte_eth_dev *dev,
if (!shaper_profile)
return -ENOMEM;
shaper_profile->shaper_profile_id = shaper_profile_id;
- rte_memcpy(&shaper_profile->profile, profile,
+ memcpy(&shaper_profile->profile, profile,
sizeof(struct rte_tm_shaper_params));
TAILQ_INSERT_TAIL(&hw->tm_conf.shaper_profile_list,
shaper_profile, node);
diff --git a/drivers/net/intel/ice/ice_diagnose.c b/drivers/net/intel/ice/ice_diagnose.c
index a58279a116..4946ace7ae 100644
--- a/drivers/net/intel/ice/ice_diagnose.c
+++ b/drivers/net/intel/ice/ice_diagnose.c
@@ -363,13 +363,13 @@ ice_dump_pkg(struct rte_eth_dev *dev, uint8_t **buff, uint32_t *size)
count = *size / ICE_PKG_BUF_SIZE;
for (i = 0; i < count; i++) {
next_buff = (uint8_t *)(*buff) + i * ICE_PKG_BUF_SIZE;
- rte_memcpy(pkg_buff.buf, next_buff, ICE_PKG_BUF_SIZE);
+ memcpy(pkg_buff.buf, next_buff, ICE_PKG_BUF_SIZE);
if (ice_aq_upload_section(hw,
(struct ice_buf_hdr *)&pkg_buff.buf[0],
ICE_PKG_BUF_SIZE,
NULL))
return -EINVAL;
- rte_memcpy(next_buff, pkg_buff.buf, ICE_PKG_BUF_SIZE);
+ memcpy(next_buff, pkg_buff.buf, ICE_PKG_BUF_SIZE);
}
cache_size = sizeof(struct ice_package_header) + *size;
diff --git a/drivers/net/intel/ice/ice_ethdev.c b/drivers/net/intel/ice/ice_ethdev.c
index 0f2e7aee14..9ffbce2ae5 100644
--- a/drivers/net/intel/ice/ice_ethdev.c
+++ b/drivers/net/intel/ice/ice_ethdev.c
@@ -3697,7 +3697,7 @@ ice_get_default_rss_key(uint8_t *rss_key, uint32_t rss_key_size)
key[i] = (uint8_t)rte_rand();
default_key_done = true;
}
- rte_memcpy(rss_key, key, RTE_MIN(rss_key_size, sizeof(default_key)));
+ memcpy(rss_key, key, RTE_MIN(rss_key_size, sizeof(default_key)));
}
static int ice_init_rss(struct ice_pf *pf)
@@ -3752,13 +3752,13 @@ static int ice_init_rss(struct ice_pf *pf)
if (!rss_conf->rss_key)
ice_get_default_rss_key(vsi->rss_key, vsi->rss_key_size);
else
- rte_memcpy(vsi->rss_key, rss_conf->rss_key,
+ memcpy(vsi->rss_key, rss_conf->rss_key,
RTE_MIN(rss_conf->rss_key_len,
vsi->rss_key_size));
- rte_memcpy(key.standard_rss_key, vsi->rss_key,
+ memcpy(key.standard_rss_key, vsi->rss_key,
ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE);
- rte_memcpy(key.extended_hash_key,
+ memcpy(key.extended_hash_key,
&vsi->rss_key[ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE],
ICE_AQC_GET_SET_RSS_KEY_DATA_HASH_KEY_SIZE);
ret = ice_aq_set_rss_key(hw, vsi->idx, &key);
@@ -5156,7 +5156,7 @@ ice_vsi_config_vlan_filter(struct ice_vsi *vsi, bool on)
vsi->info.sw_flags2 &= ~sw_flags2;
vsi->info.sw_id = hw->port_info->sw_id;
- (void)rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ (void)memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.info.valid_sections =
rte_cpu_to_le_16(ICE_AQ_VSI_PROP_SW_VALID |
ICE_AQ_VSI_PROP_SECURITY_VALID);
@@ -5911,7 +5911,7 @@ ice_vsi_vlan_pvid_set(struct ice_vsi *vsi, struct ice_vsi_vlan_pvid_info *info)
ICE_AQ_VSI_INNER_VLAN_EMODE_M);
vsi->info.inner_vlan_flags |= vlan_flags;
memset(&ctxt, 0, sizeof(ctxt));
- rte_memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
+ memcpy(&ctxt.info, &vsi->info, sizeof(vsi->info));
ctxt.info.valid_sections =
rte_cpu_to_le_16(ICE_AQ_VSI_PROP_VLAN_VALID);
ctxt.vsi_num = vsi->vsi_id;
diff --git a/drivers/net/intel/ice/ice_fdir_filter.c b/drivers/net/intel/ice/ice_fdir_filter.c
index 3522d77123..2276a198fc 100644
--- a/drivers/net/intel/ice/ice_fdir_filter.c
+++ b/drivers/net/intel/ice/ice_fdir_filter.c
@@ -1268,7 +1268,7 @@ ice_fdir_add_del_raw(struct ice_pf *pf,
struct ice_hw *hw = ICE_PF_TO_HW(pf);
unsigned char *pkt = (unsigned char *)pf->fdir.prg_pkt;
- rte_memcpy(pkt, filter->pkt_buf, filter->pkt_len);
+ memcpy(pkt, filter->pkt_buf, filter->pkt_len);
struct ice_fltr_desc desc;
memset(&desc, 0, sizeof(desc));
@@ -1315,16 +1315,16 @@ ice_fdir_extract_fltr_key(struct ice_fdir_fltr_pattern *key,
memset(key, 0, sizeof(*key));
key->flow_type = input->flow_type;
- rte_memcpy(&key->ip, &input->ip, sizeof(key->ip));
- rte_memcpy(&key->mask, &input->mask, sizeof(key->mask));
- rte_memcpy(&key->ext_data, &input->ext_data, sizeof(key->ext_data));
- rte_memcpy(&key->ext_mask, &input->ext_mask, sizeof(key->ext_mask));
+ memcpy(&key->ip, &input->ip, sizeof(key->ip));
+ memcpy(&key->mask, &input->mask, sizeof(key->mask));
+ memcpy(&key->ext_data, &input->ext_data, sizeof(key->ext_data));
+ memcpy(&key->ext_mask, &input->ext_mask, sizeof(key->ext_mask));
- rte_memcpy(&key->gtpu_data, &input->gtpu_data, sizeof(key->gtpu_data));
- rte_memcpy(&key->gtpu_mask, &input->gtpu_mask, sizeof(key->gtpu_mask));
+ memcpy(&key->gtpu_data, &input->gtpu_data, sizeof(key->gtpu_data));
+ memcpy(&key->gtpu_mask, &input->gtpu_mask, sizeof(key->gtpu_mask));
- rte_memcpy(&key->l2tpv2_data, &input->l2tpv2_data, sizeof(key->l2tpv2_data));
- rte_memcpy(&key->l2tpv2_mask, &input->l2tpv2_mask, sizeof(key->l2tpv2_mask));
+ memcpy(&key->l2tpv2_data, &input->l2tpv2_data, sizeof(key->l2tpv2_data));
+ memcpy(&key->l2tpv2_mask, &input->l2tpv2_mask, sizeof(key->l2tpv2_mask));
key->tunnel_type = filter->tunnel_type;
}
@@ -1452,7 +1452,7 @@ ice_fdir_create_filter(struct ice_adapter *ad,
if (!entry)
goto error;
- rte_memcpy(entry, filter, sizeof(*filter));
+ memcpy(entry, filter, sizeof(*filter));
flow->rule = entry;
@@ -1513,7 +1513,7 @@ ice_fdir_create_filter(struct ice_adapter *ad,
if (filter->mark_flag == 1)
ice_fdir_rx_parsing_enable(ad, 1);
- rte_memcpy(entry, filter, sizeof(*entry));
+ memcpy(entry, filter, sizeof(*entry));
ret = ice_fdir_entry_insert(pf, entry, &key);
if (ret) {
rte_flow_error_set(error, -ret,
@@ -1812,7 +1812,7 @@ ice_fdir_parse_action(struct ice_adapter *ad,
act_count = actions->conf;
filter->input.cnt_ena = ICE_FXD_FLTR_QW0_STAT_ENA_PKTS;
- rte_memcpy(&filter->act_count, act_count,
+ memcpy(&filter->act_count, act_count,
sizeof(filter->act_count));
break;
@@ -2036,7 +2036,7 @@ ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
ret_val = -ENOMEM;
goto raw_error;
}
- rte_memcpy(pkt_buf, tmp_spec, pkt_len);
+ memcpy(pkt_buf, tmp_spec, pkt_len);
filter->pkt_buf = pkt_buf;
filter->pkt_len = pkt_len;
@@ -2086,11 +2086,11 @@ ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
p_ext_data = (tunnel_type && is_outer) ?
&filter->input.ext_data_outer :
&filter->input.ext_data;
- rte_memcpy(&p_ext_data->src_mac,
+ memcpy(&p_ext_data->src_mac,
ð_spec->hdr.src_addr, RTE_ETHER_ADDR_LEN);
- rte_memcpy(&p_ext_data->dst_mac,
+ memcpy(&p_ext_data->dst_mac,
ð_spec->hdr.dst_addr, RTE_ETHER_ADDR_LEN);
- rte_memcpy(&p_ext_data->ether_type,
+ memcpy(&p_ext_data->ether_type,
ð_spec->hdr.ether_type, sizeof(eth_spec->hdr.ether_type));
break;
case RTE_FLOW_ITEM_TYPE_IPV4:
@@ -2256,8 +2256,8 @@ ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
if (ipv6_mask->hdr.hop_limits == UINT8_MAX)
*input_set |= ICE_INSET_IPV6_HOP_LIMIT;
- rte_memcpy(&p_v6->dst_ip, &ipv6_spec->hdr.dst_addr, 16);
- rte_memcpy(&p_v6->src_ip, &ipv6_spec->hdr.src_addr, 16);
+ memcpy(&p_v6->dst_ip, &ipv6_spec->hdr.dst_addr, 16);
+ memcpy(&p_v6->src_ip, &ipv6_spec->hdr.src_addr, 16);
vtc_flow_cpu = rte_be_to_cpu_32(ipv6_spec->hdr.vtc_flow);
p_v6->tc = (uint8_t)(vtc_flow_cpu >> ICE_FDIR_IPV6_TC_OFFSET);
p_v6->proto = ipv6_spec->hdr.proto;
diff --git a/drivers/net/intel/ice/ice_generic_flow.c b/drivers/net/intel/ice/ice_generic_flow.c
index 62f0c334a1..a3ddbc65a6 100644
--- a/drivers/net/intel/ice/ice_generic_flow.c
+++ b/drivers/net/intel/ice/ice_generic_flow.c
@@ -2121,7 +2121,7 @@ ice_pattern_skip_void_item(struct rte_flow_item *items,
pe = ice_find_first_item(pb + 1, true);
cpy_count = pe - pb;
- rte_memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
+ memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
items += cpy_count;
@@ -2132,7 +2132,7 @@ ice_pattern_skip_void_item(struct rte_flow_item *items,
pb = pe + 1;
}
/* Copy the END item. */
- rte_memcpy(items, pe, sizeof(struct rte_flow_item));
+ memcpy(items, pe, sizeof(struct rte_flow_item));
}
/* Check if the pattern matches a supported item type array */
diff --git a/drivers/net/intel/ice/ice_hash.c b/drivers/net/intel/ice/ice_hash.c
index 77829e607b..51916ff6d7 100644
--- a/drivers/net/intel/ice/ice_hash.c
+++ b/drivers/net/intel/ice/ice_hash.c
@@ -751,7 +751,7 @@ ice_hash_parse_raw_pattern(struct ice_adapter *ad,
if (ret)
goto free_mem;
- rte_memcpy(&meta->raw.prof, &prof, sizeof(prof));
+ memcpy(&meta->raw.prof, &prof, sizeof(prof));
free_mem:
free(pkt_buf);
diff --git a/drivers/net/intel/ice/ice_tm.c b/drivers/net/intel/ice/ice_tm.c
index ff53f2acfd..0285ee3be8 100644
--- a/drivers/net/intel/ice/ice_tm.c
+++ b/drivers/net/intel/ice/ice_tm.c
@@ -356,7 +356,7 @@ ice_shaper_profile_add(struct rte_eth_dev *dev,
if (!shaper_profile)
return -ENOMEM;
shaper_profile->shaper_profile_id = shaper_profile_id;
- rte_memcpy(&shaper_profile->profile, profile,
+ memcpy(&shaper_profile->profile, profile,
sizeof(struct rte_tm_shaper_params));
TAILQ_INSERT_TAIL(&pf->tm_conf.shaper_profile_list,
shaper_profile, node);
--
2.51.0
^ permalink raw reply related
* [RFC PATCH 0/8] remove use of rte_memcpy from net/intel
From: Bruce Richardson @ 2026-05-19 16:05 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
This RFC proposed to replace all instances of rte_memcpy in Intel
(and former-Intel) net drivers with just regular memcpy. This is
done on the basis that the memcpy use is not datapath, but is used
for flow configuration, virt-channel (to firmware or PF) messaging
and other control path functions.
Bruce Richardson (8):
net/ice: replace use of DPDK-specific memcpy function
net/iavf: replace use of DPDK-specific memcpy function
net/i40e: replace use of DPDK-specific memcpy function
net/ixgbe: replace use of DPDK-specific memcpy function
net/e1000: replace use of DPDK-specific memcpy function
net/idpf: replace use of DPDK-specific memcpy function
net/cpfl: replace use of DPDK-specific memcpy function
net/ipn3ke: replace use of DPDK-specific memcpy function
drivers/net/intel/cpfl/cpfl_ethdev.c | 4 +-
drivers/net/intel/cpfl/cpfl_vchnl.c | 6 +-
drivers/net/intel/e1000/em_rxtx.c | 1 -
drivers/net/intel/e1000/igb_flow.c | 8 +--
drivers/net/intel/e1000/igb_pf.c | 5 +-
drivers/net/intel/e1000/igb_rxtx.c | 1 -
drivers/net/intel/i40e/base/i40e_osdep.h | 3 +-
drivers/net/intel/i40e/i40e_ethdev.c | 57 +++++++++----------
drivers/net/intel/i40e/i40e_fdir.c | 18 +++---
drivers/net/intel/i40e/i40e_flow.c | 26 ++++-----
drivers/net/intel/i40e/i40e_pf.c | 3 +-
drivers/net/intel/i40e/i40e_tm.c | 6 +-
drivers/net/intel/i40e/rte_pmd_i40e.c | 20 +++----
drivers/net/intel/iavf/base/iavf_osdep.h | 3 +-
drivers/net/intel/iavf/iavf_ethdev.c | 12 ++--
drivers/net/intel/iavf/iavf_fdir.c | 46 +++++++--------
drivers/net/intel/iavf/iavf_fsub.c | 26 ++++-----
drivers/net/intel/iavf/iavf_generic_flow.c | 4 +-
drivers/net/intel/iavf/iavf_hash.c | 4 +-
drivers/net/intel/iavf/iavf_tm.c | 6 +-
drivers/net/intel/iavf/iavf_vchnl.c | 14 ++---
drivers/net/intel/ice/base/ice_osdep.h | 5 +-
drivers/net/intel/ice/ice_dcf.c | 14 ++---
drivers/net/intel/ice/ice_dcf_ethdev.c | 10 ++--
drivers/net/intel/ice/ice_dcf_parent.c | 2 +-
drivers/net/intel/ice/ice_dcf_sched.c | 6 +-
drivers/net/intel/ice/ice_diagnose.c | 4 +-
drivers/net/intel/ice/ice_ethdev.c | 12 ++--
drivers/net/intel/ice/ice_fdir_filter.c | 36 ++++++------
drivers/net/intel/ice/ice_generic_flow.c | 4 +-
drivers/net/intel/ice/ice_hash.c | 2 +-
drivers/net/intel/ice/ice_tm.c | 2 +-
drivers/net/intel/idpf/base/idpf_osdep.h | 5 +-
drivers/net/intel/idpf/idpf_common_device.c | 2 +-
drivers/net/intel/idpf/idpf_common_virtchnl.c | 20 +++----
drivers/net/intel/idpf/idpf_ethdev.c | 10 ++--
drivers/net/intel/idpf/idpf_rxtx.c | 8 +--
drivers/net/intel/ipn3ke/ipn3ke_ethdev.h | 1 -
drivers/net/intel/ipn3ke/ipn3ke_flow.c | 26 ++++-----
drivers/net/intel/ipn3ke/ipn3ke_representor.c | 8 +--
drivers/net/intel/ipn3ke/ipn3ke_tm.c | 6 +-
drivers/net/intel/ixgbe/ixgbe_ethdev.c | 4 +-
drivers/net/intel/ixgbe/ixgbe_fdir.c | 6 +-
drivers/net/intel/ixgbe/ixgbe_flow.c | 32 +++++------
drivers/net/intel/ixgbe/ixgbe_ipsec.c | 4 +-
drivers/net/intel/ixgbe/ixgbe_pf.c | 5 +-
drivers/net/intel/ixgbe/ixgbe_tm.c | 6 +-
drivers/net/intel/ixgbe/rte_pmd_ixgbe.c | 2 +-
48 files changed, 252 insertions(+), 263 deletions(-)
--
2.51.0
^ permalink raw reply
* Re: [PATCH v2] ethdev: fix pointer check in GENEVE and RAW flow copy
From: Stephen Hemminger @ 2026-05-19 14:40 UTC (permalink / raw)
To: Denis Lyulin
Cc: Ori Kam, Thomas Monjalon, Andrew Rybchenko, Michael Baum,
Ferruh Yigit, dev, stable, adrien.mazarguil
In-Reply-To: <20260519072335.16601-1-lyulin.2003@mail.ru>
On Tue, 19 May 2026 10:23:35 +0300
Denis Lyulin <lyulin.2003@mail.ru> wrote:
> When rte_flow_conv_item_spec() is called from rte_flow_conv_pattern(),
> the spec, last and mask pointers are checked separately. If the API
> is used incorrectly, the spec pointer may be NULL while last and mask
> may be valid pointers. Also call of rte_flow_conv() with
> RTE_FLOW_CONV_OP_ITEM_MASK and item->spec == NULL may lead to the
> same problem.
>
> In rte_flow_conv_item_spec() for GENVE_OPT and RAW item types the spec
> pointer is used even if the function is called to copy last or mask.
> It may cause a NULL pointer (spec) dereference.
>
> This commit adds extra check of item->spec and if it is NULL, does not
> copy further data relying on it
>
> Fixes: 841a0445442d ("ethdev: fix GENEVE option item conversion")
> Cc: michaelba@nvidia.com
> Cc: adrien.mazarguil@6wind.com
> Cc: stephen@networkplumber.org
> Cc: stable@dpdk.org
>
> Signed-off-by: Denis Lyulin <lyulin.2003@mail.ru>
> ---
Applied to next-net
^ permalink raw reply
* Re: [PATCH v2] ethdev: fix pointer check in GENEVE and RAW flow copy
From: Stephen Hemminger @ 2026-05-19 14:40 UTC (permalink / raw)
To: Denis Lyulin
Cc: Ori Kam, Thomas Monjalon, Andrew Rybchenko, Michael Baum,
Ferruh Yigit, dev, stable, adrien.mazarguil
In-Reply-To: <20260519072335.16601-1-lyulin.2003@mail.ru>
On Tue, 19 May 2026 10:23:35 +0300
Denis Lyulin <lyulin.2003@mail.ru> wrote:
> When rte_flow_conv_item_spec() is called from rte_flow_conv_pattern(),
> the spec, last and mask pointers are checked separately. If the API
> is used incorrectly, the spec pointer may be NULL while last and mask
> may be valid pointers. Also call of rte_flow_conv() with
> RTE_FLOW_CONV_OP_ITEM_MASK and item->spec == NULL may lead to the
> same problem.
>
> In rte_flow_conv_item_spec() for GENVE_OPT and RAW item types the spec
> pointer is used even if the function is called to copy last or mask.
> It may cause a NULL pointer (spec) dereference.
>
> This commit adds extra check of item->spec and if it is NULL, does not
> copy further data relying on it
>
> Fixes: 841a0445442d ("ethdev: fix GENEVE option item conversion")
> Cc: michaelba@nvidia.com
> Cc: adrien.mazarguil@6wind.com
> Cc: stephen@networkplumber.org
> Cc: stable@dpdk.org
>
> Signed-off-by: Denis Lyulin <lyulin.2003@mail.ru>
> ---
Applied to next-net
^ permalink raw reply
* Re: [PATCH v1 1/1] net/iavf: fix large VF IRQ mapping
From: Bruce Richardson @ 2026-05-19 15:20 UTC (permalink / raw)
To: Morten Brørup
Cc: Burakov, Anatoly, Vladimir Medvedkin, dev, David Marchand,
stephen
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35F65861@smartserver.smartshare.dk>
On Fri, May 08, 2026 at 12:16:01PM +0200, Morten Brørup wrote:
> > From: Morten Brørup
> > Sent: Friday, 8 May 2026 11.16
> >
> > > From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> > > Sent: Thursday, 7 May 2026 10.09
> > >
> > > On 5/6/2026 5:58 PM, David Marchand wrote:
> > > > On Wed, 6 May 2026 at 16:07, Anatoly Burakov
> > > <anatoly.burakov@intel.com> wrote:
> > > >>
> > > >
> > > > - all those virtchnl list struct have the same elems[1] issue.
> > > > Kernel side did some cleanups some time ago, maybe time for DPDK to
> > > do
> > > > the same...?
> > > >
> > >
> > > Yes, it is indeed time to do the same, but not as part of this
> > > patchset,
> > > and not before the base driver code is updated to do the same. There
> > is
> > > some background work happening on that front already, but there are a
> > > lot of dependencies and moving parts, so we can't just change this
> > > willy
> > > nilly.
> >
> > Is there a timeline for this fix?
> >
> > With the performance improved rte_memcpy() patch [1], one of the CI
> > compilers complains about buffer overflows when writing beyond these
> > undersize arrays [2].
> > And I'd like to see the performance improved rte_memcpy() merged in
> > 26.07.
>
> With v10 of the rte_memcpy() patch [3], I have reverted the removal of the workaround that ignores stringop-overflow warnings in rte_memcpy(), so the patch doesn't depend on fixing the drivers.
>
> Please take note to remove the workaround from rte_memcpy() when the flex array issue in the drivers - using elems[1] instead of elems[] - has been fixed.
> Not ignoring buffer overflows in rte_memcpy() might help reveal bugs elsewhere.
>
Actually, I'm more wondering if we really need to use rte_memcpy in our
drivers. Any use of memcpy is likely on the control path, and not
performance critical enough to warrant use of anything but the built-in
memcpy.
/Bruce
^ permalink raw reply
* Re: [PATCH v3 1/3] spinlock: remove volatile qualifier
From: David Marchand @ 2026-05-19 15:02 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Stephen Hemminger, Konstantin Ananyev, Bruce Richardson,
Robin Jarry, stable
In-Reply-To: <20260519103640.3986710-1-thomas@monjalon.net>
On Tue, 19 May 2026 at 12:37, Thomas Monjalon <thomas@monjalon.net> wrote:
>
> When compiling with C++20 standard requirement (default in GCC 16),
> the increment and decrement of volatile variables are rejected:
>
> rte_spinlock.h:241:14: error:
> '++' expression of 'volatile'-qualified type is deprecated
> rte_spinlock.h:252:21: error:
> '--' expression of 'volatile'-qualified type is deprecated
> rte_spinlock.h:278:14: error:
> '++' expression of 'volatile'-qualified type is deprecated
>
> The count field of rte_spinlock_recursive_t
> does not need the volatile qualifier
> because it is only accessed by the thread holding the lock,
> which already provides the necessary memory ordering.
>
> The user field can be accessed outside of the lock,
> so it must handled as a C11 atomic variable.
> The name is also changed from user to owner.
> It will break if an application is accessing this field directly,
> which should never happen.
>
> Fixes: af75078fece3 ("first public release")
> Cc: stable@dpdk.org
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Series applied, thanks.
--
David Marchand
^ permalink raw reply
* Re: [PATCH] net/ice: fix TM node ID validation against configured queues
From: Bruce Richardson @ 2026-05-19 14:58 UTC (permalink / raw)
To: Ciara Loftus; +Cc: dev, stable
In-Reply-To: <20260505123157.1387791-1-ciara.loftus@intel.com>
On Tue, May 05, 2026 at 12:31:57PM +0000, Ciara Loftus wrote:
> The leaf node ID boundary is checked against the compile-time constant
> `RTE_MAX_QUEUES_PER_PORT` (1024) rather than the number of Tx queues
> configured for the port. The rte_tm specification reserves IDs 0 to
> (N-1) for leaf nodes, where N is the configured queue count, so using
> the constant produces wrong results whenever N is less than 1024. Fix by
> replacing the constant with the runtime queue count.
>
> Also add an explicit check in the non-leaf validation path that rejects
> IDs in the leaf-reserved range. This condition can be triggered
> two ways: adding a leaf node before its parent chain is complete (the
> node resolves to a non-leaf level), or assigning a leaf-range ID to a
> node intended as non-leaf.
>
> Fixes: 715d449a965b ("net/ice: enhance Tx scheduler hierarchy support")
I think this fix will cause other issues when we want to use large numbers
of queues. From the commit log for 715d449a965b:
"If the HW/firmware allows it, allow creating up to 2k child
nodes per scheduler node. Also expand the number of supported layers to
the max available, rather than always just having 3 layers. One
restriction on this change is that the topology needs to be configured
and enabled before port queue setup"
The last part is significant - the number of configured queues must be
supported by a scheduler hierarchy with enough nodes at the next,
queue-group level. Therefore, before configuring large numbers of queues we
need to configure a non-default hierarchy. Therefore, when running this
check the value of nb_txq is not known, which is why the original patch
changed the check to RTE_MAX_QUEUES_PER_PORT.
However, you correctly point out in this fix, that that is not according to
the spec for rte_tm. Therefore, I suggest changing the check, if possible:
* if nb_txq == 0 (i.e. no queues configured yet), then keep as-is with
MAX_QUEUES
* for cases where nb_txq != 0, then use nb_txq.
Does that seem sensible, or any better alternatives you can see?
/Bruce
> Cc: stable@dpdk.org
>
> Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
> ---
> drivers/net/intel/ice/ice_tm.c | 14 +++++++++++---
> 1 file changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/intel/ice/ice_tm.c b/drivers/net/intel/ice/ice_tm.c
> index ff53f2acfd..7ee4070007 100644
> --- a/drivers/net/intel/ice/ice_tm.c
> +++ b/drivers/net/intel/ice/ice_tm.c
> @@ -88,6 +88,7 @@ ice_node_param_check(uint32_t node_id,
> uint32_t priority, uint32_t weight,
> const struct rte_tm_node_params *params,
> bool is_leaf,
> + uint16_t nb_txq,
> struct rte_tm_error *error)
> {
> /* checked all the unsupported parameter */
> @@ -123,6 +124,11 @@ ice_node_param_check(uint32_t node_id,
>
> /* for non-leaf node */
> if (!is_leaf) {
> + if (node_id < nb_txq) {
> + error->type = RTE_TM_ERROR_TYPE_NODE_ID;
> + error->message = "node ID is reserved for leaf nodes";
> + return -EINVAL;
> + }
> if (params->nonleaf.wfq_weight_mode) {
> error->type =
> RTE_TM_ERROR_TYPE_NODE_PARAMS_WFQ_WEIGHT_MODE;
> @@ -146,7 +152,7 @@ ice_node_param_check(uint32_t node_id,
> }
>
> /* for leaf node */
> - if (node_id >= RTE_MAX_QUEUES_PER_PORT) {
> + if (node_id >= nb_txq) {
> error->type = RTE_TM_ERROR_TYPE_NODE_ID;
> error->message = "Node ID out of range for a leaf node.";
> return -EINVAL;
> @@ -440,7 +446,8 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
> return -EINVAL;
> }
>
> - ret = ice_node_param_check(node_id, priority, weight, params, false, error);
> + ret = ice_node_param_check(node_id, priority, weight, params, false,
> + dev->data->nb_tx_queues, error);
> if (ret)
> return ret;
>
> @@ -479,7 +486,8 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
> }
>
> ret = ice_node_param_check(node_id, priority, weight,
> - params, level_id == ice_get_leaf_level(pf), error);
> + params, level_id == ice_get_leaf_level(pf),
> + dev->data->nb_tx_queues, error);
> if (ret)
> return ret;
>
> --
> 2.43.0
>
^ permalink raw reply
* [PATCH 3/3] net/i40e/base: update version info
From: Ciara Loftus @ 2026-05-19 14:52 UTC (permalink / raw)
To: dev; +Cc: Ciara Loftus
In-Reply-To: <20260519145227.308814-1-ciara.loftus@intel.com>
Update README with the date of when the latest snapshot was generated.
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
---
drivers/net/intel/i40e/base/README | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/intel/i40e/base/README b/drivers/net/intel/i40e/base/README
index c5305ffae0..59cae0fc90 100644
--- a/drivers/net/intel/i40e/base/README
+++ b/drivers/net/intel/i40e/base/README
@@ -1,12 +1,12 @@
/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2017-2024 Intel Corporation
+ * Copyright(c) 2017-2026 Intel Corporation
*/
Intel® I40E driver
==================
This directory contains source code of i40e base driver generated on
-2024-08-30 released by the team which develops
+2026-05-07 released by the team which develops
basic drivers for any i40e NIC. The directory of base/ contains the
original source package.
This driver is valid for the product(s) listed below
--
2.43.0
^ permalink raw reply related
* [PATCH 2/3] net/i40e/base: fix loop counter width in DCB parsing
From: Ciara Loftus @ 2026-05-19 14:52 UTC (permalink / raw)
To: dev; +Cc: Deepthi Kavalur, stable, Ciara Loftus
In-Reply-To: <20260519145227.308814-1-ciara.loftus@intel.com>
From: Deepthi Kavalur <deepthi.kavalur@intel.com>
Loop counters in i40e_parse_cee_app_tlv() and
i40e_add_ieee_app_pri_tlv() are declared as u8 but compared against
numapps which is u32. If numapps exceeds 255 the counter wraps,
truncating the iteration. Widen the counters to u32.
Fixes: 166dceeeeafc ("i40e/base: add parsing for CEE DCBX TLVs")
Fixes: 0d9d27bb8684 ("i40e/base: prepare local LLDP MIB in TLV")
Cc: stable@dpdk.org
Signed-off-by: Deepthi Kavalur <deepthi.kavalur@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
---
.mailmap | 1 +
drivers/net/intel/i40e/base/i40e_dcb.c | 6 +++---
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/.mailmap b/.mailmap
index f3130df686..6121833c25 100644
--- a/.mailmap
+++ b/.mailmap
@@ -371,6 +371,7 @@ Dean Marx <dmarx@iol.unh.edu>
Declan Doherty <declan.doherty@intel.com>
Deepak Khandelwal <deepak.khandelwal@intel.com>
Deepak Kumar Jain <deepak.k.jain@intel.com>
+Deepthi Kavalur <deepthi.kavalur@intel.com>
Deirdre O'Connor <deirdre.o.connor@intel.com>
Dekel Peled <dekelp@nvidia.com> <dekelp@mellanox.com>
Dengdui Huang <huangdengdui@huawei.com>
diff --git a/drivers/net/intel/i40e/base/i40e_dcb.c b/drivers/net/intel/i40e/base/i40e_dcb.c
index 04322ea034..c1d97fabab 100644
--- a/drivers/net/intel/i40e/base/i40e_dcb.c
+++ b/drivers/net/intel/i40e/base/i40e_dcb.c
@@ -364,7 +364,7 @@ static void i40e_parse_cee_app_tlv(struct i40e_cee_feat_tlv *tlv,
{
u16 length, typelength, offset = 0;
struct i40e_cee_app_prio *app;
- u8 i;
+ u32 i;
typelength = I40E_NTOHS(tlv->hdr.typelen);
length = (u16)((typelength & I40E_LLDP_TLV_LEN_MASK) >>
@@ -1169,9 +1169,9 @@ static void i40e_add_ieee_app_pri_tlv(struct i40e_lldp_org_tlv *tlv,
struct i40e_dcbx_config *dcbcfg)
{
u16 typelength, length, offset = 0;
- u8 priority, selector, i = 0;
u8 *buf = tlv->tlvinfo;
- u32 ouisubtype;
+ u32 ouisubtype, i = 0;
+ u8 priority, selector;
/* No APP TLVs then just return */
if (dcbcfg->numapps == 0)
--
2.43.0
^ permalink raw reply related
* [PATCH 1/3] net/i40e/base: fix integer overflow in NVM timing logic
From: Ciara Loftus @ 2026-05-19 14:52 UTC (permalink / raw)
To: dev; +Cc: Chinh Cao, stable, Ciara Loftus
In-Reply-To: <20260519145227.308814-1-ciara.loftus@intel.com>
From: Chinh Cao <chinh.t.cao@intel.com>
Rename gtime to gtime_start/gtime_current with elapsed tracking
to avoid signed/unsigned comparison overflow. Widen sr_size from
u16 to u32 and NVM buffer offset parameters from u16 to u32.
Add helper get_elapsed_time() for wrap-safe timer arithmetic.
Fixes: c61390d94d46 ("net/i40e/base: make semaphore timeout 32-bit")
Fixes: cb593a832630 ("net/i40e/base: reduce size of time variables")
Fixes: 8db9e2a1b232 ("i40e: base driver")
Cc: stable@dpdk.org
Signed-off-by: Chinh Cao <chinh.t.cao@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
---
drivers/net/intel/i40e/base/i40e_nvm.c | 67 +++++++++++++-------
drivers/net/intel/i40e/base/i40e_prototype.h | 2 +-
drivers/net/intel/i40e/base/i40e_type.h | 2 +-
drivers/net/intel/i40e/i40e_ethdev.c | 3 +-
4 files changed, 49 insertions(+), 25 deletions(-)
diff --git a/drivers/net/intel/i40e/base/i40e_nvm.c b/drivers/net/intel/i40e/base/i40e_nvm.c
index 00a207ca81..fd1a987c56 100644
--- a/drivers/net/intel/i40e/base/i40e_nvm.c
+++ b/drivers/net/intel/i40e/base/i40e_nvm.c
@@ -49,6 +49,20 @@ enum i40e_status_code i40e_init_nvm(struct i40e_hw *hw)
return ret_code;
}
+/**
+ * get_elapsed_time - Compute elapsed hardware timer ticks
+ * @from: The start timer value
+ * @to: The current timer value
+ *
+ * Returns the elapsed time as the difference between the current and start
+ * timer values. The subtraction is wrap-safe for unsigned 32-bit values,
+ * meaning it correctly handles timer wrap-around.
+ **/
+static u32 get_elapsed_time(u32 from, u32 to)
+{
+ return (u32)(to - from); /* wrap-safe */
+}
+
/**
* i40e_acquire_nvm - Generic request for acquiring the NVM ownership
* @hw: pointer to the HW structure
@@ -61,7 +75,7 @@ enum i40e_status_code i40e_acquire_nvm(struct i40e_hw *hw,
enum i40e_aq_resource_access_type access)
{
enum i40e_status_code ret_code = I40E_SUCCESS;
- u32 gtime, timeout;
+ u32 gtime_start, gtime_current, timeout, elapsed;
u32 time_left = 0;
DEBUGFUNC("i40e_acquire_nvm");
@@ -72,10 +86,12 @@ enum i40e_status_code i40e_acquire_nvm(struct i40e_hw *hw,
ret_code = i40e_aq_request_resource(hw, I40E_NVM_RESOURCE_ID, access,
0, &time_left, NULL);
/* Reading the Global Device Timer */
- gtime = rd32(hw, I40E_GLVFGEN_TIMER);
+ gtime_start = rd32(hw, I40E_GLVFGEN_TIMER);
+ gtime_current = gtime_start;
+ elapsed = 0;
/* Store the timeout */
- hw->nvm.hw_semaphore_timeout = I40E_MS_TO_GTIME(time_left) + gtime;
+ hw->nvm.hw_semaphore_timeout = I40E_MS_TO_GTIME(time_left) + gtime_start;
if (ret_code)
i40e_debug(hw, I40E_DEBUG_NVM,
@@ -84,17 +100,20 @@ enum i40e_status_code i40e_acquire_nvm(struct i40e_hw *hw,
if (ret_code && time_left) {
/* Poll until the current NVM owner timeouts */
- timeout = I40E_MS_TO_GTIME(I40E_MAX_NVM_TIMEOUT) + gtime;
- while ((s32)(gtime - timeout) < 0 && time_left) {
+ timeout = I40E_MS_TO_GTIME(I40E_MAX_NVM_TIMEOUT);
+
+ while (elapsed < timeout && time_left) {
i40e_msec_delay(10);
- gtime = rd32(hw, I40E_GLVFGEN_TIMER);
+ gtime_current = rd32(hw, I40E_GLVFGEN_TIMER);
+ elapsed = get_elapsed_time(gtime_start, gtime_current);
ret_code = i40e_aq_request_resource(hw,
I40E_NVM_RESOURCE_ID,
access, 0, &time_left,
NULL);
if (ret_code == I40E_SUCCESS) {
hw->nvm.hw_semaphore_timeout =
- I40E_MS_TO_GTIME(time_left) + gtime;
+ I40E_MS_TO_GTIME(time_left) +
+ gtime_current;
break;
}
}
@@ -110,7 +129,6 @@ enum i40e_status_code i40e_acquire_nvm(struct i40e_hw *hw,
return ret_code;
}
-
/**
* i40e_acquire_nvm_ex - Specific request only for
* OID_INTEL_FLASH_INFO_TIMEOUT for acquiring the NVM ownership
@@ -121,13 +139,12 @@ enum i40e_status_code i40e_acquire_nvm(struct i40e_hw *hw,
* This function will request NVM ownership for reading
* via the proper Admin Command.
**/
-
enum i40e_status_code i40e_acquire_nvm_ex(struct i40e_hw *hw,
enum i40e_aq_resource_access_type access,
u32 custom_timeout)
{
enum i40e_status_code ret_code = I40E_SUCCESS;
- u32 gtime, timeout;
+ u32 gtime_start, gtime_current, timeout, elapsed;
u32 time_left = 0;
DEBUGFUNC("i40e_acquire_nvm");
@@ -138,10 +155,12 @@ enum i40e_status_code i40e_acquire_nvm_ex(struct i40e_hw *hw,
ret_code = i40e_aq_request_resource(hw, I40E_NVM_RESOURCE_ID, access,
0, &time_left, NULL);
/* Reading the Global Device Timer */
- gtime = rd32(hw, I40E_GLVFGEN_TIMER);
+ gtime_start = rd32(hw, I40E_GLVFGEN_TIMER);
+ gtime_current = gtime_start;
+ elapsed = 0;
/* Store the timeout */
- hw->nvm.hw_semaphore_timeout = I40E_MS_TO_GTIME(time_left) + gtime;
+ hw->nvm.hw_semaphore_timeout = I40E_MS_TO_GTIME(time_left) + gtime_start;
if (ret_code)
i40e_debug(hw, I40E_DEBUG_NVM,
@@ -151,17 +170,20 @@ enum i40e_status_code i40e_acquire_nvm_ex(struct i40e_hw *hw,
if (ret_code && time_left) {
/* Poll until the current NVM owner timeouts */
- timeout = I40E_MS_TO_GTIME(custom_timeout) + gtime;
- while ((gtime < timeout) && time_left) {
+ timeout = I40E_MS_TO_GTIME(custom_timeout);
+
+ while (elapsed < timeout && time_left) {
i40e_msec_delay(10);
- gtime = rd32(hw, I40E_GLVFGEN_TIMER);
+ gtime_current = rd32(hw, I40E_GLVFGEN_TIMER);
+ elapsed = get_elapsed_time(gtime_start, gtime_current);
ret_code = i40e_aq_request_resource(hw,
I40E_NVM_RESOURCE_ID,
access, 0, &time_left,
NULL);
if (ret_code == I40E_SUCCESS) {
hw->nvm.hw_semaphore_timeout =
- I40E_MS_TO_GTIME(time_left) + gtime;
+ I40E_MS_TO_GTIME(time_left) +
+ gtime_current;
break;
}
}
@@ -245,7 +267,7 @@ static enum i40e_status_code i40e_poll_sr_srctl_done_bit(struct i40e_hw *hw)
* Reads one 16 bit word from the Shadow RAM using the GLNVM_SRCTL register.
**/
STATIC enum i40e_status_code i40e_read_nvm_word_srctl(struct i40e_hw *hw,
- u16 offset,
+ u32 offset,
u16 *data)
{
enum i40e_status_code ret_code = I40E_ERR_TIMEOUT;
@@ -516,11 +538,12 @@ i40e_read_nvm_module_data(struct i40e_hw *hw, u8 module_ptr, u16 module_offset,
* method. The buffer read is preceded by the NVM ownership take
* and followed by the release.
**/
-STATIC enum i40e_status_code i40e_read_nvm_buffer_srctl(struct i40e_hw *hw, u16 offset,
+STATIC enum i40e_status_code i40e_read_nvm_buffer_srctl(struct i40e_hw *hw, u32 offset,
u16 *words, u16 *data)
{
enum i40e_status_code ret_code = I40E_SUCCESS;
- u16 index, word;
+ u32 index;
+ u16 word;
DEBUGFUNC("i40e_read_nvm_buffer_srctl");
@@ -549,7 +572,7 @@ STATIC enum i40e_status_code i40e_read_nvm_buffer_srctl(struct i40e_hw *hw, u16
* method. The buffer read is preceded by the NVM ownership take
* and followed by the release.
**/
-STATIC enum i40e_status_code i40e_read_nvm_buffer_aq(struct i40e_hw *hw, u16 offset,
+STATIC enum i40e_status_code i40e_read_nvm_buffer_aq(struct i40e_hw *hw, u32 offset,
u16 *words, u16 *data)
{
enum i40e_status_code ret_code;
@@ -608,7 +631,7 @@ STATIC enum i40e_status_code i40e_read_nvm_buffer_aq(struct i40e_hw *hw, u16 off
* method.
**/
enum i40e_status_code __i40e_read_nvm_buffer(struct i40e_hw *hw,
- u16 offset,
+ u32 offset,
u16 *words, u16 *data)
{
if (hw->flags & I40E_HW_FLAG_AQ_SRCTL_ACCESS_ENABLE)
@@ -767,7 +790,7 @@ enum i40e_status_code i40e_calc_nvm_checksum(struct i40e_hw *hw, u16 *checksum)
u16 checksum_local = 0;
u16 vpd_module = 0;
u16 *data;
- u16 i = 0;
+ u32 i = 0;
DEBUGFUNC("i40e_calc_nvm_checksum");
diff --git a/drivers/net/intel/i40e/base/i40e_prototype.h b/drivers/net/intel/i40e/base/i40e_prototype.h
index e7e6d4c427..6f6bafa43c 100644
--- a/drivers/net/intel/i40e/base/i40e_prototype.h
+++ b/drivers/net/intel/i40e/base/i40e_prototype.h
@@ -482,7 +482,7 @@ enum i40e_status_code i40e_write_nvm_aq(struct i40e_hw *hw, u8 module,
bool last_command);
enum i40e_status_code __i40e_read_nvm_word(struct i40e_hw *hw, u16 offset,
u16 *data);
-enum i40e_status_code __i40e_read_nvm_buffer(struct i40e_hw *hw, u16 offset,
+enum i40e_status_code __i40e_read_nvm_buffer(struct i40e_hw *hw, u32 offset,
u16 *words, u16 *data);
enum i40e_status_code __i40e_write_nvm_word(struct i40e_hw *hw, u32 offset,
void *data);
diff --git a/drivers/net/intel/i40e/base/i40e_type.h b/drivers/net/intel/i40e/base/i40e_type.h
index 968e1982a6..05a08b2057 100644
--- a/drivers/net/intel/i40e/base/i40e_type.h
+++ b/drivers/net/intel/i40e/base/i40e_type.h
@@ -449,7 +449,7 @@ enum i40e_aq_resource_access_type {
struct i40e_nvm_info {
u32 hw_semaphore_timeout; /* usec global time (GTIME resolution) */
u32 timeout; /* [ms] */
- u16 sr_size; /* Shadow RAM size in words */
+ u32 sr_size; /* Shadow RAM size in words */
bool blank_nvm_mode; /* is NVM empty (no FW present)*/
u16 version; /* NVM package version */
u32 eetrack; /* NVM data version */
diff --git a/drivers/net/intel/i40e/i40e_ethdev.c b/drivers/net/intel/i40e/i40e_ethdev.c
index 100a751225..13f3c23fef 100644
--- a/drivers/net/intel/i40e/i40e_ethdev.c
+++ b/drivers/net/intel/i40e/i40e_ethdev.c
@@ -11357,7 +11357,8 @@ static int i40e_get_eeprom(struct rte_eth_dev *dev,
{
struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
uint16_t *data = eeprom->data;
- uint16_t offset, length, cnt_words;
+ uint32_t offset, length;
+ uint16_t cnt_words;
int ret_code;
offset = eeprom->offset >> 1;
--
2.43.0
^ permalink raw reply related
* [PATCH 0/3] i40e base code update
From: Ciara Loftus @ 2026-05-19 14:52 UTC (permalink / raw)
To: dev; +Cc: Ciara Loftus
Update the i40e base code. Both patches address integer type width issues
that can cause incorrect behaviour and are candidates for stable backport.
Chinh Cao (1):
net/i40e/base: fix integer overflow in NVM timing logic
Ciara Loftus (1):
net/i40e/base: update version info
Deepthi Kavalur (1):
net/i40e/base: fix loop counter width in DCB parsing
.mailmap | 1 +
drivers/net/intel/i40e/base/README | 4 +-
drivers/net/intel/i40e/base/i40e_dcb.c | 6 +-
drivers/net/intel/i40e/base/i40e_nvm.c | 67 +++++++++++++-------
drivers/net/intel/i40e/base/i40e_prototype.h | 2 +-
drivers/net/intel/i40e/base/i40e_type.h | 2 +-
drivers/net/intel/i40e/i40e_ethdev.c | 3 +-
7 files changed, 55 insertions(+), 30 deletions(-)
--
2.43.0
^ permalink raw reply
* Re: [PATCH] net/ixgbe: fix flow control frame byte adjustment
From: Bruce Richardson @ 2026-05-19 14:50 UTC (permalink / raw)
To: Daniil Iskhakov
Cc: Anatoly Burakov, Vladimir Medvedkin, dev, stable, sdl.dpdk, rrv
In-Reply-To: <agx3WX8nLoJT6JbS@bricha3-mobl1.ger.corp.intel.com>
On Tue, May 19, 2026 at 03:44:41PM +0100, Bruce Richardson wrote:
> On Mon, May 04, 2026 at 05:55:02PM +0300, Daniil Iskhakov wrote:
> > LXONTXC and LXOFFTXC are 32-bit counters for transmitted XON and
> > XOFF packets. ixgbe_read_stats_registers() sums their deltas and uses
> > the result to adjust the transmitted byte counters by the minimum
> > Ethernet frame length.
> >
> > The sum is currently computed as:
> >
> > total = lxon + lxoff
> >
> > Since both operands are 32-bit, the addition is performed in 32 bits
> > and may wrap before the result is stored in total. The wrapped value is
> > then used in the byte adjustment, which may make the software byte
> > counters incorrect.
> >
> > Make total 64-bit and cast lxon before the addition so the XON/XOFF
> > packet sum and the following byte adjustment are computed without
> > 32-bit overflow.
> >
> > Found by Linux Verification Center (linuxtesting.org) with SVACE.
> >
> > Fixes: af75078fece3 ("first public release")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Daniil Iskhakov <dish@amicon.ru>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>
Applied to dpdk-next-net-intel.
Thanks,
/Bruce
^ permalink raw reply
* [PATCH v18 08/11] net/sxe2: support queue setup and control
From: liujie5 @ 2026-05-19 14:48 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260519144810.3951202-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Add support for Rx and Tx queue setup, release, and management.
Implement eth_dev_ops callbacks for rx_queue_setup, tx_queue_setup,
rx_queue_release, and tx_queue_release.
This includes:
- Allocating memory for hardware ring descriptors.
- Initializing software ring structures and hardware head/tail pointers.
- Implementing proper resource cleanup logic to prevent memory leaks
during queue reconfiguration or device close.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/meson.build | 2 +
drivers/net/sxe2/sxe2_ethdev.c | 82 +++--
drivers/net/sxe2/sxe2_ethdev.h | 15 +-
drivers/net/sxe2/sxe2_rx.c | 554 +++++++++++++++++++++++++++++++++
drivers/net/sxe2/sxe2_rx.h | 32 ++
drivers/net/sxe2/sxe2_tx.c | 420 +++++++++++++++++++++++++
drivers/net/sxe2/sxe2_tx.h | 32 ++
7 files changed, 1111 insertions(+), 26 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_rx.c
create mode 100644 drivers/net/sxe2/sxe2_rx.h
create mode 100644 drivers/net/sxe2/sxe2_tx.c
create mode 100644 drivers/net/sxe2/sxe2_tx.h
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 00c38b147c..3dfe54903a 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -18,6 +18,8 @@ sources += files(
'sxe2_cmd_chnl.c',
'sxe2_vsi.c',
'sxe2_queue.c',
+ 'sxe2_tx.c',
+ 'sxe2_rx.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 204add9c98..6abb4672f6 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -24,6 +24,8 @@
#include "sxe2_ethdev.h"
#include "sxe2_drv_cmd.h"
#include "sxe2_cmd_chnl.h"
+#include "sxe2_tx.h"
+#include "sxe2_rx.h"
#include "sxe2_common.h"
#include "sxe2_common_log.h"
#include "sxe2_host_regs.h"
@@ -86,14 +88,6 @@ static int32_t sxe2_dev_configure(struct rte_eth_dev *dev)
return ret;
}
-static void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev __rte_unused)
-{
-}
-
-static void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev __rte_unused)
-{
-}
-
static int32_t sxe2_dev_stop(struct rte_eth_dev *dev)
{
int32_t ret = 0;
@@ -112,16 +106,6 @@ static int32_t sxe2_dev_stop(struct rte_eth_dev *dev)
return ret;
}
-static int32_t __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev __rte_unused)
-{
- return 0;
-}
-
-static int32_t __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev __rte_unused)
-{
- return 0;
-}
-
static int32_t sxe2_queues_start(struct rte_eth_dev *dev)
{
int32_t ret = 0;
@@ -307,10 +291,18 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_stop = sxe2_dev_stop,
.dev_close = sxe2_dev_close,
.dev_infos_get = sxe2_dev_infos_get,
+
+ .rx_queue_setup = sxe2_rx_queue_setup,
+ .tx_queue_setup = sxe2_tx_queue_setup,
+ .rx_queue_release = sxe2_rx_queue_release,
+ .tx_queue_release = sxe2_tx_queue_release,
+
+ .rxq_info_get = sxe2_rx_queue_info_get,
+ .txq_info_get = sxe2_tx_queue_info_get,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type)
+ enum sxe2_pci_map_resource res_type)
{
struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
struct sxe2_pci_map_bar_info *bar_info = NULL;
@@ -334,6 +326,48 @@ struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter
return bar_info;
}
+void *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type,
+ uint16_t idx_in_func)
+{
+ struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
+ struct sxe2_pci_map_segment_info *seg_info = NULL;
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ void *addr = NULL;
+ uintptr_t calc_addr = 0;
+ uint8_t reg_width = 0;
+ uint8_t i = 0;
+
+ bar_info = sxe2_dev_get_bar_info(adapter, res_type);
+ if (bar_info == NULL) {
+ PMD_DEV_LOG_WARN(adapter, INIT, "Failed to get bar info, res_type=[%d]",
+ res_type);
+ goto l_end;
+ }
+ seg_info = bar_info->seg_info;
+
+ reg_width = map_ctxt->addr_info[res_type].reg_width;
+ if (reg_width == 0) {
+ PMD_DEV_LOG_WARN(adapter, INIT, "Invalid reg width with resource type %d",
+ res_type);
+ goto l_end;
+ }
+
+ for (i = 0; i < bar_info->map_cnt; i++) {
+ seg_info = &bar_info->seg_info[i];
+ if (res_type == seg_info->type) {
+ calc_addr = (uintptr_t)seg_info->addr;
+ calc_addr += (uintptr_t)seg_info->page_inner_offset;
+ calc_addr += (uintptr_t)reg_width * (uintptr_t)idx_in_func;
+ addr = (void *)calc_addr;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return addr;
+}
+
static void sxe2_drv_dev_caps_set(struct sxe2_adapter *adapter,
struct sxe2_drv_dev_caps_resp *dev_caps)
{
@@ -402,7 +436,9 @@ static int32_t sxe2_dev_caps_get(struct sxe2_adapter *adapter)
}
int32_t sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type, uint64_t org_len, uint64_t org_offset)
+ enum sxe2_pci_map_resource res_type,
+ uint64_t org_len,
+ uint64_t org_offset)
{
struct sxe2_pci_map_bar_info *bar_info = NULL;
struct sxe2_pci_map_segment_info *seg_info = NULL;
@@ -478,8 +514,10 @@ static int32_t sxe2_hw_init(struct rte_eth_dev *dev)
return ret;
}
-int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter, uint32_t res_type,
- uint32_t item_cnt, uint32_t item_base)
+int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter,
+ uint32_t res_type,
+ uint32_t item_cnt,
+ uint32_t item_base)
{
struct sxe2_pci_map_addr_info *addr_info = NULL;
int32_t ret = 0;
diff --git a/drivers/net/sxe2/sxe2_ethdev.h b/drivers/net/sxe2/sxe2_ethdev.h
index 843e652616..001413e75a 100644
--- a/drivers/net/sxe2/sxe2_ethdev.h
+++ b/drivers/net/sxe2/sxe2_ethdev.h
@@ -293,14 +293,21 @@ struct sxe2_adapter {
#define SXE2_DEV_TO_PCI(eth_dev) \
RTE_DEV_TO_PCI((eth_dev)->device)
+void *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type,
+ uint16_t idx_in_func);
+
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type);
+ enum sxe2_pci_map_resource res_type);
int32_t sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type, uint64_t org_len, uint64_t org_offset);
+ enum sxe2_pci_map_resource res_type,
+ uint64_t org_len, uint64_t org_offset);
-int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter, uint32_t res_type,
- uint32_t item_cnt, uint32_t item_base);
+int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter,
+ uint32_t res_type,
+ uint32_t item_cnt,
+ uint32_t item_base);
void sxe2_dev_pci_seg_unmap(struct sxe2_adapter *adapter, uint32_t res_type);
diff --git a/drivers/net/sxe2/sxe2_rx.c b/drivers/net/sxe2/sxe2_rx.c
new file mode 100644
index 0000000000..28832d5f71
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_rx.c
@@ -0,0 +1,554 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+
+#include "sxe2_ethdev.h"
+#include "sxe2_queue.h"
+#include "sxe2_rx.h"
+#include "sxe2_cmd_chnl.h"
+
+#include "sxe2_osal.h"
+#include "sxe2_common_log.h"
+
+static void *sxe2_rx_doorbell_tail_addr_get(struct sxe2_adapter *adapter, uint16_t queue_id)
+{
+ return sxe2_pci_map_addr_get(adapter, SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL,
+ queue_id);
+}
+
+static void sxe2_rx_head_tail_init(struct sxe2_adapter *adapter, struct sxe2_rx_queue *rxq)
+{
+ rxq->rdt_reg_addr = sxe2_rx_doorbell_tail_addr_get(adapter, rxq->queue_id);
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, 0);
+}
+
+static void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq)
+{
+ uint16_t i = 0;
+ uint16_t len = 0;
+ static const union sxe2_rx_desc zeroed_desc = {{0}};
+
+ len = rxq->ring_depth + SXE2_RX_PKTS_BURST_BATCH_NUM;
+ for (i = 0; i < len; ++i)
+ rxq->desc_ring[i] = zeroed_desc;
+
+ memset(&rxq->fake_mbuf, 0, sizeof(rxq->fake_mbuf));
+ for (i = rxq->ring_depth; i < len; i++)
+ rxq->buffer_ring[i] = &rxq->fake_mbuf;
+
+ rxq->hold_num = 0;
+ rxq->next_ret_pkt = 0;
+ rxq->processing_idx = 0;
+ rxq->completed_pkts_num = 0;
+ rxq->batch_alloc_trigger = rxq->rx_free_thresh - 1;
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+
+ rxq->realloc_num = 0;
+ rxq->realloc_start = 0;
+}
+
+void __rte_cold sxe2_rx_queue_mbufs_release(struct sxe2_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->buffer_ring != NULL) {
+ for (i = 0; i < rxq->ring_depth; i++) {
+ if (rxq->buffer_ring[i] != NULL) {
+ rte_pktmbuf_free(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ }
+
+ if (rxq->completed_pkts_num) {
+ for (i = 0; i < rxq->completed_pkts_num; ++i) {
+ if (rxq->completed_buf[rxq->next_ret_pkt + i] != NULL) {
+ rte_pktmbuf_free(rxq->completed_buf[rxq->next_ret_pkt + i]);
+ rxq->completed_buf[rxq->next_ret_pkt + i] = NULL;
+ }
+ }
+ rxq->completed_pkts_num = 0;
+ }
+}
+
+const struct sxe2_rxq_ops sxe2_default_rxq_ops = {
+ .queue_reset = sxe2_rx_queue_reset,
+ .mbufs_release = sxe2_rx_queue_mbufs_release,
+};
+
+static struct sxe2_rxq_ops sxe2_rx_default_ops_get(void)
+{
+ return sxe2_default_rxq_ops;
+}
+
+void __rte_cold sxe2_rx_queue_info_get(struct rte_eth_dev *dev,
+ uint16_t queue_id, struct rte_eth_rxq_info *qinfo)
+{
+ struct sxe2_rx_queue *rxq = NULL;
+
+ if (queue_id >= dev->data->nb_rx_queues) {
+ PMD_LOG_ERR(RX, "rx queue:%u is out of range:%u",
+ queue_id, dev->data->nb_rx_queues);
+ goto end;
+ }
+
+ rxq = dev->data->rx_queues[queue_id];
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue:%u is NULL", queue_id);
+ goto end;
+ }
+
+ qinfo->mp = rxq->mb_pool;
+ qinfo->nb_desc = rxq->ring_depth;
+ qinfo->scattered_rx = dev->data->scattered_rx;
+ qinfo->conf.rx_free_thresh = rxq->rx_free_thresh;
+ qinfo->conf.rx_drop_en = rxq->drop_en;
+ qinfo->conf.rx_deferred_start = rxq->rx_deferred_start;
+
+end:
+ return;
+}
+
+int32_t __rte_cold sxe2_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_rx_queue *rxq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (dev->data->rx_queue_state[rx_queue_id] ==
+ RTE_ETH_QUEUE_STATE_STOPPED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (rxq == NULL) {
+ ret = 0;
+ goto l_end;
+ }
+ ret = sxe2_drv_rxq_switch(adapter, rxq, false);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Failed to switch rx queue %u off, ret = %d",
+ rx_queue_id, ret);
+ if (ret == -EPERM)
+ goto l_free;
+ goto l_end;
+ }
+
+l_free:
+ rxq->ops.mbufs_release(rxq);
+ rxq->ops.queue_reset(rxq);
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STOPPED;
+l_end:
+ return ret;
+}
+
+static void __rte_cold sxe2_rx_queue_free(struct sxe2_rx_queue *rxq)
+{
+ if (rxq != NULL) {
+ rxq->ops.mbufs_release(rxq);
+ if (rxq->buffer_ring != NULL) {
+ rte_free(rxq->buffer_ring);
+ rxq->buffer_ring = NULL;
+ }
+ rte_memzone_free(rxq->mz);
+ rte_free(rxq);
+ }
+}
+
+void __rte_cold sxe2_rx_queue_release(struct rte_eth_dev *dev,
+ uint16_t queue_idx)
+{
+ (void)sxe2_rx_queue_stop(dev, queue_idx);
+ sxe2_rx_queue_free(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+}
+
+void __rte_cold sxe2_all_rxqs_release(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ uint16_t nb_rxq;
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ if (data->rx_queues[nb_rxq] == NULL)
+ continue;
+ sxe2_rx_queue_release(dev, nb_rxq);
+ data->rx_queues[nb_rxq] = NULL;
+ }
+ data->nb_rx_queues = 0;
+}
+
+static struct sxe2_rx_queue *sxe2_rx_queue_alloc(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t ring_depth, uint32_t socket_id)
+{
+ struct sxe2_rx_queue *rxq;
+ const struct rte_memzone *tz;
+ uint16_t len;
+
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ sxe2_rx_queue_release(dev, queue_idx);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ rxq = rte_zmalloc_socket("rx_queue", sizeof(*rxq),
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue[%d] alloc failed", queue_idx);
+ goto l_end;
+ }
+
+ rxq->ring_depth = ring_depth;
+ len = rxq->ring_depth + SXE2_RX_PKTS_BURST_BATCH_NUM;
+
+ rxq->buffer_ring = rte_zmalloc_socket("rx_buffer_ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (!rxq->buffer_ring) {
+ PMD_LOG_ERR(RX, "Rxq malloc mbuf mem failed");
+ rte_free(rxq);
+ rxq = NULL;
+ goto l_end;
+ }
+
+ tz = rte_eth_dma_zone_reserve(dev, "rx_dma", queue_idx,
+ SXE2_RX_RING_SIZE, SXE2_DESC_ADDR_ALIGN, socket_id);
+ if (tz == NULL) {
+ PMD_LOG_ERR(RX, "Rxq malloc desc mem failed");
+ rte_free(rxq->buffer_ring);
+ rxq->buffer_ring = NULL;
+ rte_free(rxq);
+ rxq = NULL;
+ goto l_end;
+ }
+
+ rxq->mz = tz;
+ memset(tz->addr, 0, SXE2_RX_RING_SIZE);
+ rxq->base_addr = tz->iova;
+ rxq->desc_ring = (union sxe2_rx_desc *)tz->addr;
+
+l_end:
+ return rxq;
+}
+
+int32_t __rte_cold sxe2_rx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ struct sxe2_rx_queue *rxq;
+ uint64_t offloads;
+ int32_t ret;
+ uint16_t rx_nseg;
+ uint16_t i;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % SXE2_RX_DESC_RING_ALIGN != 0 ||
+ nb_desc > SXE2_MAX_RING_DESC ||
+ nb_desc < SXE2_MIN_RING_DESC) {
+ PMD_LOG_ERR(RX, "param desc num:%u is invalid", nb_desc);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (mp != NULL)
+ rx_nseg = 1;
+ else
+ rx_nseg = rx_conf->rx_nseg;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (rx_nseg > 1 && !(offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ PMD_LOG_ERR(RX, "Port %u queue %u Buffer split offload not configured, but rx_nseg is %u",
+ dev->data->port_id, queue_idx, rx_nseg);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) && !(rx_nseg > 1)) {
+ PMD_LOG_ERR(RX, "Port %u queue %u Buffer split offload configured, but rx_nseg is %u",
+ dev->data->port_id, queue_idx, rx_nseg);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO) &&
+ (offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)) {
+ PMD_LOG_ERR(RX, "port_id %u queue %u, LRO can't be configure with Keep crc.",
+ dev->data->port_id, queue_idx);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ rxq = sxe2_rx_queue_alloc(dev, queue_idx, nb_desc, socket_id);
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue[%d] resource alloc failed", queue_idx);
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ if (offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
+ dev->data->lro = 1;
+
+ if (rx_nseg > 1) {
+ for (i = 0; i < rx_nseg; i++) {
+ rte_memcpy(&rxq->rx_seg[i], &rx_conf->rx_seg[i].split,
+ sizeof(struct rte_eth_rxseg_split));
+ }
+ rxq->mb_pool = rxq->rx_seg[0].mp;
+ } else {
+ rxq->mb_pool = mp;
+ }
+
+ rxq->rx_free_thresh = rx_conf->rx_free_thresh;
+ rxq->port_id = dev->data->port_id;
+ rxq->offloads = offloads;
+ if (offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ rxq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ rxq->crc_len = 0;
+
+ rxq->queue_id = queue_idx;
+ rxq->idx_in_func = vsi->rxqs.base_idx_in_func + queue_idx;
+ rxq->drop_en = rx_conf->rx_drop_en;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->vsi = vsi;
+ rxq->ops = sxe2_rx_default_ops_get();
+ rxq->ops.queue_reset(rxq);
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ ret = 0;
+l_end:
+ return ret;
+}
+
+static int32_t __rte_cold sxe2_rx_queue_mbufs_alloc(struct sxe2_rx_queue *rxq)
+{
+ struct rte_mbuf **buf_ring = rxq->buffer_ring;
+ struct rte_mbuf *mbuf = NULL;
+ struct rte_mbuf *mbuf_pay;
+ volatile union sxe2_rx_desc *desc;
+ uint64_t dma_addr;
+ int32_t ret;
+ uint16_t i, j;
+
+ for (i = 0; i < rxq->ring_depth; i++) {
+ mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (mbuf == NULL) {
+ PMD_LOG_ERR(RX, "Rx queue is not available or setup");
+ ret = -ENOMEM;
+ goto l_err_free_mbuf;
+ }
+
+ buf_ring[i] = mbuf;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+ desc = &rxq->desc_ring[i];
+ if (!(rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ mbuf->next = NULL;
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr = dma_addr;
+ } else {
+ mbuf_pay = rte_mbuf_raw_alloc(rxq->rx_seg[1].mp);
+ if (unlikely(!mbuf_pay)) {
+ PMD_LOG_ERR(RX, "Failed to allocate payload mbuf for RX");
+ ret = -ENOMEM;
+ goto l_err_free_mbuf;
+ }
+
+ mbuf_pay->next = NULL;
+ mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf_pay->nb_segs = 1;
+ mbuf_pay->port = rxq->port_id;
+ mbuf->next = mbuf_pay;
+
+ desc->read.hdr_addr = dma_addr;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf_pay));
+ }
+
+#ifndef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+ desc->read.rsvd1 = 0;
+ desc->read.rsvd2 = 0;
+#endif
+ }
+
+ ret = 0;
+ goto l_end;
+
+l_err_free_mbuf:
+ for (j = 0; j <= i; j++) {
+ if (buf_ring[j] != NULL && buf_ring[j]->next != NULL) {
+ rte_pktmbuf_free(buf_ring[j]->next);
+ buf_ring[j]->next = NULL;
+ }
+
+ if (buf_ring[j] != NULL) {
+ rte_pktmbuf_free(buf_ring[j]);
+ buf_ring[j] = NULL;
+ }
+ }
+
+l_end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct sxe2_rx_queue *rxq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "Rx queue %u is not available or setup",
+ rx_queue_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (dev->data->rx_queue_state[rx_queue_id] ==
+ RTE_ETH_QUEUE_STATE_STARTED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ ret = sxe2_rx_queue_mbufs_alloc(rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Rx queue %u apply desc ring fail",
+ rx_queue_id);
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ sxe2_rx_head_tail_init(adapter, rxq);
+
+ ret = sxe2_drv_rxq_ctxt_cfg(adapter, rxq, 1);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Rx queue %u config ctxt fail, ret=%d",
+ rx_queue_id, ret);
+
+ (void)sxe2_drv_rxq_switch(adapter, rxq, false);
+ rxq->ops.mbufs_release(rxq);
+ rxq->ops.queue_reset(rxq);
+ goto l_end;
+ }
+
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, rxq->ring_depth - 1);
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+
+l_end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_rx_queue *rxq;
+ uint16_t nb_rxq;
+ uint16_t nb_started_rxq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ rxq = dev->data->rx_queues[nb_rxq];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+
+ ret = sxe2_rx_queue_start(dev, nb_rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Fail to start rx queue %u", nb_rxq);
+ goto l_free_started_queue;
+ }
+
+ rte_atomic_store_explicit(&rxq->sw_stats.pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.bytes, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.drop_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.drop_bytes, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.unicast_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.broadcast_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.multicast_pkts, 0,
+ rte_memory_order_relaxed);
+ }
+ ret = 0;
+ goto l_end;
+
+l_free_started_queue:
+ for (nb_started_rxq = 0; nb_started_rxq <= nb_rxq; nb_started_rxq++)
+ (void)sxe2_rx_queue_stop(dev, nb_started_rxq);
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ struct sxe2_stats *sw_stats_prev = &vsi->vsi_stats.vsi_sw_stats_prev;
+ struct sxe2_rx_queue *rxq = NULL;
+ int32_t ret;
+ uint16_t nb_rxq;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ ret = sxe2_rx_queue_stop(dev, nb_rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Fail to stop rx queue %u", nb_rxq);
+ continue;
+ }
+
+ rxq = dev->data->rx_queues[nb_rxq];
+ if (rxq) {
+ sw_stats_prev->ipackets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->ierrors +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->ibytes +=
+ rte_atomic_load_explicit(&rxq->sw_stats.bytes,
+ rte_memory_order_relaxed);
+
+ sw_stats_prev->rx_sw_unicast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.unicast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_broadcast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.broadcast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_multicast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.multicast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_drop_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_drop_bytes +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_bytes,
+ rte_memory_order_relaxed);
+ }
+ }
+}
diff --git a/drivers/net/sxe2/sxe2_rx.h b/drivers/net/sxe2/sxe2_rx.h
new file mode 100644
index 0000000000..1c53f7f559
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_rx.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_RX_H__
+#define __SXE2_RX_H__
+
+#include "sxe2_queue.h"
+
+int32_t __rte_cold sxe2_rx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+
+int32_t __rte_cold sxe2_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+
+void __rte_cold sxe2_rx_queue_mbufs_release(struct sxe2_rx_queue *rxq);
+
+void __rte_cold sxe2_rx_queue_release(struct rte_eth_dev *dev, uint16_t queue_idx);
+
+void __rte_cold sxe2_all_rxqs_release(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_rxq_info *qinfo);
+
+int32_t __rte_cold sxe2_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+
+int32_t __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev);
+
+#endif /* __SXE2_RX_H__ */
diff --git a/drivers/net/sxe2/sxe2_tx.c b/drivers/net/sxe2/sxe2_tx.c
new file mode 100644
index 0000000000..a05beb8c7a
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_tx.c
@@ -0,0 +1,420 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include "sxe2_tx.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_cmd_chnl.h"
+
+static void *sxe2_tx_doorbell_addr_get(struct sxe2_adapter *adapter, uint16_t queue_id)
+{
+ return sxe2_pci_map_addr_get(adapter, SXE2_PCI_MAP_RES_DOORBELL_TX,
+ queue_id);
+}
+
+static void sxe2_tx_tail_init(struct sxe2_adapter *adapter, struct sxe2_tx_queue *txq)
+{
+ txq->tdt_reg_addr = sxe2_tx_doorbell_addr_get(adapter, txq->queue_id);
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, 0);
+}
+
+void __rte_cold sxe2_tx_queue_reset(struct sxe2_tx_queue *txq)
+{
+ uint16_t prev, i;
+ volatile union sxe2_tx_data_desc *txd;
+ static const union sxe2_tx_data_desc zeroed_desc = {{0}};
+ struct sxe2_tx_buffer *tx_buffer = txq->buffer_ring;
+
+ for (i = 0; i < txq->ring_depth; i++)
+ txq->desc_ring[i] = zeroed_desc;
+
+ prev = txq->ring_depth - 1;
+ for (i = 0; i < txq->ring_depth; i++) {
+ txd = &txq->desc_ring[i];
+ if (txd == NULL)
+ continue;
+
+ txd->wb.dd = rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE);
+ tx_buffer[i].mbuf = NULL;
+ tx_buffer[i].last_id = i;
+ tx_buffer[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->desc_used_num = 0;
+ txq->desc_free_num = txq->ring_depth - 1;
+ txq->next_use = 0;
+ txq->next_clean = txq->ring_depth - 1;
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+void __rte_cold sxe2_tx_queue_mbufs_release(struct sxe2_tx_queue *txq)
+{
+ uint32_t i;
+
+ if (txq != NULL && txq->buffer_ring != NULL) {
+ for (i = 0; i < txq->ring_depth; i++) {
+ if (txq->buffer_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->buffer_ring[i].mbuf);
+ txq->buffer_ring[i].mbuf = NULL;
+ }
+ }
+ }
+}
+
+static void sxe2_tx_buffer_ring_free(struct sxe2_tx_queue *txq)
+{
+ if (txq != NULL && txq->buffer_ring != NULL)
+ rte_free(txq->buffer_ring);
+}
+
+const struct sxe2_txq_ops sxe2_default_txq_ops = {
+ .queue_reset = sxe2_tx_queue_reset,
+ .mbufs_release = sxe2_tx_queue_mbufs_release,
+ .buffer_ring_free = sxe2_tx_buffer_ring_free,
+};
+
+static struct sxe2_txq_ops sxe2_tx_default_ops_get(void)
+{
+ return sxe2_default_txq_ops;
+}
+
+static int32_t sxe2_txq_arg_validate(struct rte_eth_dev *dev, uint16_t ring_depth,
+ uint16_t *rs_thresh, uint16_t *free_thresh, const struct rte_eth_txconf *tx_conf)
+{
+ int32_t ret = 0;
+
+ if ((ring_depth % SXE2_TX_DESC_RING_ALIGN) != 0 ||
+ ring_depth > SXE2_MAX_RING_DESC ||
+ ring_depth < SXE2_MIN_RING_DESC) {
+ PMD_LOG_ERR(TX, "number:%u of receive descriptors is invalid", ring_depth);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ *free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : DEFAULT_TX_FREE_THRESH);
+ *rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : DEFAULT_TX_RS_THRESH);
+
+ if (*rs_thresh >= (ring_depth - 2)) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be less than the number "
+ "of tx descriptors minus 2. (tx_rs_thresh:%u port:%u)",
+ *rs_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (*free_thresh >= (ring_depth - 3)) {
+ PMD_LOG_ERR(TX, "tx_free_thresh must be less than the number "
+ "of tx descriptors minus 3. (tx_free_thresh:%u port:%u)",
+ *free_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (*rs_thresh > *free_thresh) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be less than or equal to "
+ "tx_free_thresh. (tx_free_thresh:%u tx_rs_thresh:%u port:%u)",
+ *free_thresh, *rs_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((ring_depth % *rs_thresh) != 0) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be a divisor of the "
+ "number of tx descriptors. (tx_rs_thresh:%u port:%d ring_depth:%u)",
+ *rs_thresh, dev->data->port_id, ring_depth);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_txq_info *qinfo)
+{
+ struct sxe2_tx_queue *txq = NULL;
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ PMD_LOG_WARN(TX, "tx queue:%u is NULL", queue_id);
+ goto end;
+ }
+
+ qinfo->nb_desc = txq->ring_depth;
+
+ qinfo->conf.tx_thresh.pthresh = txq->pthresh;
+ qinfo->conf.tx_thresh.hthresh = txq->hthresh;
+ qinfo->conf.tx_thresh.wthresh = txq->wthresh;
+ qinfo->conf.tx_free_thresh = txq->free_thresh;
+ qinfo->conf.tx_rs_thresh = txq->rs_thresh;
+ qinfo->conf.offloads = txq->offloads;
+ qinfo->conf.tx_deferred_start = txq->tx_deferred_start;
+
+end:
+ return;
+}
+
+int32_t __rte_cold sxe2_tx_queue_stop(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_tx_queue *txq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (dev->data->tx_queue_state[queue_id] ==
+ RTE_ETH_QUEUE_STATE_STOPPED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ ret = 0;
+ goto l_end;
+ }
+
+ ret = sxe2_drv_txq_switch(adapter, txq, false);
+ if (ret) {
+ PMD_LOG_ERR(TX, "Failed to switch tx queue %u off",
+ queue_id);
+ goto l_end;
+ }
+
+ txq->ops.mbufs_release(txq);
+ txq->ops.queue_reset(txq);
+ dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+static void __rte_cold sxe2_tx_queue_free(struct sxe2_tx_queue *txq)
+{
+ if (txq != NULL) {
+ txq->ops.mbufs_release(txq);
+ txq->ops.buffer_ring_free(txq);
+
+ rte_memzone_free(txq->mz);
+ rte_free(txq);
+ }
+}
+
+void __rte_cold sxe2_tx_queue_release(struct rte_eth_dev *dev, uint16_t queue_idx)
+{
+ (void)sxe2_tx_queue_stop(dev, queue_idx);
+ sxe2_tx_queue_free(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+}
+
+void __rte_cold sxe2_all_txqs_release(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ uint16_t nb_txq;
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ if (data->tx_queues[nb_txq] == NULL)
+ continue;
+
+ sxe2_tx_queue_release(dev, nb_txq);
+ data->tx_queues[nb_txq] = NULL;
+ }
+ data->nb_tx_queues = 0;
+}
+
+static struct sxe2_tx_queue
+*sxe2_tx_queue_alloc(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t ring_depth, uint32_t socket_id)
+{
+ struct sxe2_tx_queue *txq;
+ const struct rte_memzone *tz;
+
+ if (dev->data->tx_queues[queue_idx]) {
+ sxe2_tx_queue_release(dev, queue_idx);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ txq = rte_zmalloc_socket("tx_queue", sizeof(struct sxe2_tx_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "tx queue:%d alloc failed", queue_idx);
+ goto l_end;
+ }
+
+ tz = rte_eth_dma_zone_reserve(dev, "tx_dma", queue_idx,
+ sizeof(union sxe2_tx_data_desc) * SXE2_MAX_RING_DESC,
+ SXE2_DESC_ADDR_ALIGN, socket_id);
+ if (tz == NULL) {
+ PMD_LOG_ERR(TX, "tx desc ring alloc failed, queue_id:%d", queue_idx);
+ rte_free(txq);
+ txq = NULL;
+ goto l_end;
+ }
+
+ txq->buffer_ring = rte_zmalloc_socket("tx_buffer_ring",
+ sizeof(struct sxe2_tx_buffer) * ring_depth,
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq->buffer_ring == NULL) {
+ PMD_LOG_ERR(TX, "tx buffer alloc failed, queue_id:%d", queue_idx);
+ rte_memzone_free(tz);
+ rte_free(txq);
+ txq = NULL;
+ goto l_end;
+ }
+
+ txq->mz = tz;
+ txq->base_addr = tz->iova;
+ txq->desc_ring = (volatile union sxe2_tx_data_desc *)tz->addr;
+
+l_end:
+ return txq;
+}
+
+int32_t __rte_cold sxe2_tx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ int32_t ret = 0;
+ uint16_t tx_rs_thresh;
+ uint16_t tx_free_thresh;
+ struct sxe2_tx_queue *txq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ uint64_t offloads;
+ PMD_INIT_FUNC_TRACE();
+
+ ret = sxe2_txq_arg_validate(dev, nb_desc, &tx_rs_thresh, &tx_free_thresh, tx_conf);
+ if (ret) {
+ PMD_LOG_ERR(TX, "tx queue:%u arg validate failed", queue_idx);
+ goto end;
+ }
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ txq = sxe2_tx_queue_alloc(dev, queue_idx, nb_desc, socket_id);
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "failed to alloc sxe2vf tx queue:%u resource", queue_idx);
+ ret = -ENOMEM;
+ goto end;
+ }
+
+ txq->vlan_flag = SXE2_TX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ txq->ring_depth = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->pthresh = tx_conf->tx_thresh.pthresh;
+ txq->hthresh = tx_conf->tx_thresh.hthresh;
+ txq->wthresh = tx_conf->tx_thresh.wthresh;
+ txq->queue_id = queue_idx;
+ txq->idx_in_func = vsi->txqs.base_idx_in_func + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+ txq->vsi = vsi;
+ txq->ops = sxe2_tx_default_ops_get();
+ txq->ops.queue_reset(txq);
+
+ dev->data->tx_queues[queue_idx] = txq;
+ ret = 0;
+
+end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_tx_queue_start(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ int32_t ret = 0;
+ struct sxe2_tx_queue *txq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ PMD_INIT_FUNC_TRACE();
+
+ if (dev->data->tx_queue_state[queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "tx queue:%u is not available or setup", queue_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ ret = sxe2_drv_txq_ctxt_cfg(adapter, txq, 1);
+ if (ret) {
+ PMD_LOG_ERR(TX, "tx queue:%u config ctxt fail", queue_id);
+
+ (void)sxe2_drv_txq_switch(adapter, txq, false);
+ txq->ops.mbufs_release(txq);
+ txq->ops.queue_reset(txq);
+ goto l_end;
+ }
+
+ sxe2_tx_tail_init(adapter, txq);
+
+ dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_tx_queue *txq;
+ uint16_t nb_txq;
+ uint16_t nb_started_txq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ txq = dev->data->tx_queues[nb_txq];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+
+ ret = sxe2_tx_queue_start(dev, nb_txq);
+ if (ret) {
+ PMD_LOG_ERR(TX, "Fail to start tx queue %u", nb_txq);
+ goto l_free_started_queue;
+ }
+ }
+ ret = 0;
+ goto l_end;
+
+l_free_started_queue:
+ for (nb_started_txq = 0; nb_started_txq <= nb_txq; nb_started_txq++)
+ (void)sxe2_tx_queue_stop(dev, nb_started_txq);
+
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ uint16_t nb_txq;
+ int32_t ret;
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ ret = sxe2_tx_queue_stop(dev, nb_txq);
+ if (ret) {
+ PMD_LOG_WARN(TX, "Fail to stop tx queue %u", nb_txq);
+ continue;
+ }
+ }
+}
diff --git a/drivers/net/sxe2/sxe2_tx.h b/drivers/net/sxe2/sxe2_tx.h
new file mode 100644
index 0000000000..c929b1bee2
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_tx.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TX_H__
+#define __SXE2_TX_H__
+#include "sxe2_queue.h"
+
+void __rte_cold sxe2_tx_queue_reset(struct sxe2_tx_queue *txq);
+
+int32_t __rte_cold sxe2_tx_queue_start(struct rte_eth_dev *dev, uint16_t queue_id);
+
+void sxe2_tx_queue_mbufs_release(struct sxe2_tx_queue *txq);
+
+int32_t __rte_cold sxe2_tx_queue_stop(struct rte_eth_dev *dev, uint16_t queue_id);
+
+int32_t __rte_cold sxe2_tx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+void __rte_cold sxe2_tx_queue_release(struct rte_eth_dev *dev, uint16_t queue_idx);
+
+void __rte_cold sxe2_all_txqs_release(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_txq_info *qinfo);
+
+int32_t __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev);
+
+#endif /* __SXE2_TX_H__ */
--
2.47.3
^ permalink raw reply related
* [PATCH v18 10/11] net/sxe2: add vectorized Rx and Tx
From: liujie5 @ 2026-05-19 14:48 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260519144810.3951202-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch implements the vectorized data path for the sxe2 PMD.
It utilizes SIMD instructions (e.g., SSE) to process multiple
packets simultaneously, significantly improving throughput for
small packet processing.
The implementation includes:
* Vectorized Rx burst function for bulk descriptor processing.
* Vectorized Tx burst function with optimized resource cleanup.
* Capability flags update to reflect vectorized path support.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/meson.build | 5 +
drivers/net/sxe2/sxe2_ethdev.c | 29 +-
drivers/net/sxe2/sxe2_queue.c | 28 ++
drivers/net/sxe2/sxe2_queue.h | 4 +
drivers/net/sxe2/sxe2_txrx.c | 204 ++++++---
drivers/net/sxe2/sxe2_txrx.h | 11 +-
drivers/net/sxe2/sxe2_txrx_poll.c | 26 ++
drivers/net/sxe2/sxe2_txrx_poll.h | 4 +
drivers/net/sxe2/sxe2_txrx_vec.c | 201 +++++++++
drivers/net/sxe2/sxe2_txrx_vec.h | 63 +++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 235 ++++++++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 549 ++++++++++++++++++++++++
12 files changed, 1284 insertions(+), 75 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_sse.c
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 5645e3ad61..3df57aee8c 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -13,6 +13,10 @@ deps += ['common_sxe2', 'hash','cryptodev','security']
includes += include_directories('../../common/sxe2')
+if arch_subdir == 'x86'
+ sources += files('sxe2_txrx_vec_sse.c')
+endif
+
sources += files(
'sxe2_ethdev.c',
'sxe2_cmd_chnl.c',
@@ -22,6 +26,7 @@ sources += files(
'sxe2_rx.c',
'sxe2_txrx_poll.c',
'sxe2_txrx.c',
+ 'sxe2_txrx_vec.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index e47e788d78..d1bdc22bd0 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -107,25 +107,6 @@ static int32_t sxe2_dev_stop(struct rte_eth_dev *dev)
return ret;
}
-static int32_t sxe2_queues_start(struct rte_eth_dev *dev)
-{
- int32_t ret = 0;
- ret = sxe2_txqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start tx queue.");
- goto l_end;
- }
-
- ret = sxe2_rxqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start rx queue.");
- sxe2_txqs_all_stop(dev);
- }
-
-l_end:
- return ret;
-}
-
static int32_t sxe2_dev_start(struct rte_eth_dev *dev)
{
int32_t ret = 0;
@@ -158,7 +139,7 @@ static int32_t sxe2_dev_start(struct rte_eth_dev *dev)
static int32_t sxe2_dev_close(struct rte_eth_dev *dev)
{
(void)sxe2_dev_stop(dev);
-
+ (void)sxe2_queues_release(dev);
sxe2_vsi_uninit(dev);
sxe2_dev_pci_map_uinit(dev);
@@ -296,13 +277,19 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_close = sxe2_dev_close,
.dev_infos_get = sxe2_dev_infos_get,
+ .rx_queue_start = sxe2_rx_queue_start,
+ .rx_queue_stop = sxe2_rx_queue_stop,
+ .tx_queue_start = sxe2_tx_queue_start,
+ .tx_queue_stop = sxe2_tx_queue_stop,
.rx_queue_setup = sxe2_rx_queue_setup,
- .tx_queue_setup = sxe2_tx_queue_setup,
.rx_queue_release = sxe2_rx_queue_release,
+ .tx_queue_setup = sxe2_tx_queue_setup,
.tx_queue_release = sxe2_tx_queue_release,
.rxq_info_get = sxe2_rx_queue_info_get,
.txq_info_get = sxe2_tx_queue_info_get,
+ .rx_burst_mode_get = sxe2_rx_burst_mode_get,
+ .tx_burst_mode_get = sxe2_tx_burst_mode_get,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
diff --git a/drivers/net/sxe2/sxe2_queue.c b/drivers/net/sxe2/sxe2_queue.c
index 93f8236381..1786d6ea4f 100644
--- a/drivers/net/sxe2/sxe2_queue.c
+++ b/drivers/net/sxe2/sxe2_queue.c
@@ -5,6 +5,8 @@
#include "sxe2_ethdev.h"
#include "sxe2_queue.h"
#include "sxe2_common_log.h"
+#include "sxe2_tx.h"
+#include "sxe2_rx.h"
void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
struct sxe2_drv_queue_caps *q_caps)
@@ -36,3 +38,29 @@ int32_t sxe2_queues_init(struct rte_eth_dev *dev)
return ret;
}
+
+int32_t sxe2_queues_start(struct rte_eth_dev *dev)
+{
+ int32_t ret = 0;
+
+ ret = sxe2_txqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start tx queue.");
+ goto l_end;
+ }
+
+ ret = sxe2_rxqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start rx queue.");
+ sxe2_txqs_all_stop(dev);
+ }
+l_end:
+ return ret;
+}
+
+void sxe2_queues_release(struct rte_eth_dev *dev)
+{
+ sxe2_all_rxqs_release(dev);
+
+ sxe2_all_txqs_release(dev);
+}
diff --git a/drivers/net/sxe2/sxe2_queue.h b/drivers/net/sxe2/sxe2_queue.h
index e587e582fa..5195e2dd16 100644
--- a/drivers/net/sxe2/sxe2_queue.h
+++ b/drivers/net/sxe2/sxe2_queue.h
@@ -188,4 +188,8 @@ void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
int32_t sxe2_queues_init(struct rte_eth_dev *dev);
+int32_t sxe2_queues_start(struct rte_eth_dev *dev);
+
+void sxe2_queues_release(struct rte_eth_dev *dev);
+
#endif /* __SXE2_QUEUE_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
index 0384c2091e..4ee72c0879 100644
--- a/drivers/net/sxe2/sxe2_txrx.c
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -9,12 +9,11 @@
#include <rte_memzone.h>
#include <ethdev_driver.h>
#include <unistd.h>
-
#include "sxe2_txrx.h"
#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_vec.h"
#include "sxe2_txrx_poll.h"
#include "sxe2_ethdev.h"
-
#include "sxe2_common_log.h"
#include "sxe2_osal.h"
#include "sxe2_cmd_chnl.h"
@@ -22,7 +21,32 @@
#include <rte_cpuflags.h>
#endif
+int32_t __rte_cold
+sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ uint32_t *batch_flags)
+{
+ struct sxe2_tx_queue *txq;
+ int32_t ret = 0;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+ if (txq->offloads != (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) ||
+ txq->rs_thresh < SXE2_TX_PKTS_BURST_BATCH_NUM) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ }
+ *batch_flags = SXE2_TX_MODE_SIMPLE_BATCH;
+l_end:
+ return ret;
+}
static int32_t sxe2_tx_descriptor_status(void *tx_queue, uint16_t offset)
+
{
struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
int32_t ret;
@@ -32,7 +56,6 @@ static int32_t sxe2_tx_descriptor_status(void *tx_queue, uint16_t offset)
ret = -EINVAL;
goto l_end;
}
-
desc_idx = txq->next_use + offset;
desc_idx = SXE2_DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
if (desc_idx >= txq->ring_depth) {
@@ -40,19 +63,16 @@ static int32_t sxe2_tx_descriptor_status(void *tx_queue, uint16_t offset)
if (desc_idx >= txq->ring_depth)
desc_idx -= txq->ring_depth;
}
-
if (desc_idx == 0)
desc_idx = txq->rs_thresh - 1;
else
desc_idx -= 1;
-
if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
(txq->desc_ring[desc_idx].wb.dd &
rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
ret = RTE_ETH_TX_DESC_DONE;
else
ret = RTE_ETH_TX_DESC_FULL;
-
l_end:
return ret;
}
@@ -60,7 +80,6 @@ static int32_t sxe2_tx_descriptor_status(void *tx_queue, uint16_t offset)
static inline int32_t sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
{
struct rte_mbuf *m_seg = mbuf;
-
while (m_seg != NULL) {
if (m_seg->data_len == 0)
return -EINVAL;
@@ -68,6 +87,7 @@ static inline int32_t sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
}
return 0;
+
}
uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
@@ -97,12 +117,10 @@ uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -EINVAL;
goto l_end;
}
-
if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
rte_errno = -EINVAL;
goto l_end;
}
-
#ifdef RTE_ETHDEV_DEBUG_TX
ret = rte_validate_tx_offload(mbuf);
if (ret != 0) {
@@ -115,14 +133,12 @@ uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -ret;
goto l_end;
}
-
ret = sxe2_tx_mbuf_empty_check(mbuf);
if (ret != 0) {
rte_errno = -ret;
goto l_end;
}
}
-
l_end:
return i;
}
@@ -130,15 +146,85 @@ uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
- uint32_t tx_mode_flags = 0;
+ uint32_t tx_mode_flags;
+ int32_t ret;
+ uint32_t vec_flags = 0;
+ uint32_t batch_flags = 0;
PMD_INIT_FUNC_TRACE();
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ tx_mode_flags = 0;
+ ret = sxe2_tx_vec_support_check(dev, &vec_flags);
+ if (ret == 0 &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ tx_mode_flags = vec_flags;
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ ret = sxe2_tx_queues_vec_prepare(dev);
+ if (ret != 0)
+ tx_mode_flags &= ~SXE2_TX_MODE_VEC_SET_MASK;
+ }
+ }
+ ret = sxe2_tx_simple_batch_support_check(dev, &batch_flags);
+ if (ret == 0 && batch_flags == SXE2_TX_MODE_SIMPLE_BATCH)
+ tx_mode_flags |= SXE2_TX_MODE_SIMPLE_BATCH;
- dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
- dev->tx_pkt_burst = sxe2_tx_pkts;
- adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
- PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
- tx_mode_flags, dev->data->port_id);
+ adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
+ } else {
+ tx_mode_flags = adapter->q_ctxt.tx_mode_flags;
+ }
+
+#ifdef RTE_ARCH_X86
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_OFFLOAD) {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse;
+ } else {
+ dev->tx_pkt_prepare = NULL;
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse_simple;
+ }
+ } else {
+#endif
+ if (tx_mode_flags & SXE2_TX_MODE_SIMPLE_BATCH) {
+ dev->tx_pkt_prepare = NULL;
+ dev->tx_pkt_burst = sxe2_tx_pkts_simple;
+ } else {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ }
+#ifdef RTE_ARCH_X86
+ }
+#endif
+}
+
+static const struct {
+ eth_tx_burst_t tx_burst;
+ const char *info;
+} sxe2_tx_burst_infos[] = {
+ { sxe2_tx_pkts, "Scalar" },
+#ifdef RTE_ARCH_X86
+ { sxe2_tx_pkts_vec_sse, "Vector SSE" },
+ { sxe2_tx_pkts_vec_sse_simple, "Vector SSE Simple" },
+#endif
+};
+
+int32_t sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_tx_burst_t pkt_burst = dev->tx_pkt_burst;
+ int32_t ret = -EINVAL;
+ uint32_t i;
+ uint32_t size;
+
+ size = RTE_DIM(sxe2_tx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_tx_burst_infos[i].tx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_tx_burst_infos[i].info);
+ ret = 0;
+ break;
+ }
+ }
+ return ret;
}
static int32_t sxe2_rx_descriptor_status(void *rx_queue, uint16_t offset)
@@ -151,22 +237,18 @@ static int32_t sxe2_rx_descriptor_status(void *rx_queue, uint16_t offset)
ret = -EINVAL;
goto l_end;
}
-
if (offset >= rxq->ring_depth - rxq->hold_num) {
ret = RTE_ETH_RX_DESC_UNAVAIL;
goto l_end;
}
-
if (rxq->processing_idx + offset >= rxq->ring_depth)
desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
else
desc = &rxq->desc_ring[rxq->processing_idx + offset];
-
if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
ret = RTE_ETH_RX_DESC_DONE;
else
ret = RTE_ETH_RX_DESC_AVAIL;
-
l_end:
PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
offset, ret, rxq->queue_id, rxq->port_id);
@@ -189,55 +271,79 @@ static int32_t sxe2_rx_queue_count(void *rx_queue)
else
desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
}
-
PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
done_num, rxq->queue_id, rxq->port_id);
-
return done_num;
}
-static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload)
-{
- struct sxe2_rx_queue *rxq;
- bool en = false;
- uint16_t i;
-
- for (i = 0; i < dev->data->nb_rx_queues; ++i) {
- rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
- if (rxq == NULL)
- continue;
-
- if (0 != (rxq->offloads & offload)) {
- en = true;
- goto l_end;
- }
- }
-
-l_end:
- return en;
-}
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
uint32_t rx_mode_flags = 0;
-
+ int32_t ret;
+ uint32_t vec_flags = 0;
PMD_INIT_FUNC_TRACE();
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = sxe2_rx_vec_support_check(dev, &vec_flags);
+ if (ret == 0 &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ rx_mode_flags = vec_flags;
+ if ((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) != 0) {
+ ret = sxe2_rx_queues_vec_prepare(dev);
+ if (ret != 0)
+ rx_mode_flags &= ~SXE2_RX_MODE_VEC_SET_MASK;
+ }
+ }
+ adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
+ } else {
+ rx_mode_flags = adapter->q_ctxt.rx_mode_flags;
+ }
+
+#ifdef RTE_ARCH_X86
+ if (rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) {
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_vec_sse_offload;
+ return;
+ }
+#endif
if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
else
dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
+}
- PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
- rx_mode_flags, dev->data->port_id);
- adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
+static const struct {
+ eth_rx_burst_t rx_burst;
+ const char *info;
+} sxe2_rx_burst_infos[] = {
+ { sxe2_rx_pkts_scattered, "Scalar Scattered" },
+ { sxe2_rx_pkts_scattered_split, "Scalar Scattered split" },
+#ifdef RTE_ARCH_X86
+ { sxe2_rx_pkts_scattered_vec_sse_offload, "Vector SSE Scattered" },
+#endif
+};
+
+int32_t sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_rx_burst_t pkt_burst = dev->rx_pkt_burst;
+ int32_t ret = -EINVAL;
+ uint32_t i, size;
+ size = RTE_DIM(sxe2_rx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_rx_burst_infos[i].rx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_rx_burst_infos[i].info);
+ ret = 0;
+ break;
+ }
+ }
+ return ret;
}
void sxe2_set_common_function(struct rte_eth_dev *dev)
{
PMD_INIT_FUNC_TRACE();
-
dev->rx_queue_count = sxe2_rx_queue_count;
dev->rx_descriptor_status = sxe2_rx_descriptor_status;
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
index f6558e2189..61c6641e49 100644
--- a/drivers/net/sxe2/sxe2_txrx.h
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -6,16 +6,17 @@
#define SXE2_TXRX_H
#include <ethdev_driver.h>
#include "sxe2_queue.h"
-
void sxe2_set_common_function(struct rte_eth_dev *dev);
+int32_t __rte_cold sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ uint32_t *batch_flags);
uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
-
void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
-
void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
-
+int32_t sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
+int32_t sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
#endif /* __SXE2_TXRX_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.c b/drivers/net/sxe2/sxe2_txrx_poll.c
index ce56f9086f..2931f52445 100644
--- a/drivers/net/sxe2/sxe2_txrx_poll.c
+++ b/drivers/net/sxe2/sxe2_txrx_poll.c
@@ -480,6 +480,32 @@ static inline uint16_t sxe2_tx_pkts_batch(void *tx_queue,
return nb_pkts;
}
+uint16_t sxe2_tx_pkts_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ uint16_t tx_done_num;
+ uint16_t tx_once_num;
+ uint16_t tx_need_num;
+ if (likely(nb_pkts <= SXE2_TX_PKTS_BURST_BATCH_NUM)) {
+ tx_done_num = sxe2_tx_pkts_batch(tx_queue,
+ tx_pkts, nb_pkts);
+ goto l_end;
+ }
+ tx_done_num = 0;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, SXE2_TX_PKTS_BURST_BATCH_NUM);
+ tx_once_num = sxe2_tx_pkts_batch(tx_queue,
+ &tx_pkts[tx_done_num],
+ tx_need_num);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+l_end:
+ return tx_done_num;
+}
+
static inline void
sxe2_update_rx_tail(struct sxe2_rx_queue *rxq, uint16_t hold_num, uint16_t rx_id)
{
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.h b/drivers/net/sxe2/sxe2_txrx_poll.h
index f45e33f9b7..6bb2238a2f 100644
--- a/drivers/net/sxe2/sxe2_txrx_poll.h
+++ b/drivers/net/sxe2/sxe2_txrx_poll.h
@@ -9,6 +9,10 @@
uint16_t sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+uint16_t sxe2_tx_pkts_simple(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+
+uint16_t sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
+
uint16_t sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
uint16_t sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.c b/drivers/net/sxe2/sxe2_txrx_vec.c
new file mode 100644
index 0000000000..8df4954d86
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.c
@@ -0,0 +1,201 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+
+int32_t __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags)
+{
+ struct sxe2_rx_queue *rxq;
+ int32_t ret = 0;
+ uint16_t i;
+
+ *vec_flags = SXE2_RX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+ if (!rte_is_power_of_2(rxq->ring_depth)) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if (rxq->rx_free_thresh < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC &&
+ (rxq->ring_depth % rxq->rx_free_thresh) != 0) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_RX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+ if ((rxq->offloads & offload) != 0) {
+ en = true;
+ goto l_end;
+ }
+ }
+l_end:
+ return en;
+}
+
+static inline void sxe2_rx_queue_mbufs_release_vec(struct sxe2_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->ring_depth - 1;
+ uint16_t i;
+
+ if (unlikely(!rxq->buffer_ring)) {
+ PMD_LOG_DEBUG(RX, "Rx queue release mbufs vec, buffer_ring if NULL."
+ "port_id:%u queue_id:%u", rxq->port_id, rxq->queue_id);
+ return;
+ }
+ if (rxq->realloc_num >= rxq->ring_depth)
+ return;
+ if (rxq->realloc_num == 0) {
+ for (i = 0; i < rxq->ring_depth; ++i) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ } else {
+ for (i = rxq->processing_idx;
+ i != rxq->realloc_start;
+ i = (i + 1) & mask) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ }
+ rxq->realloc_num = rxq->ring_depth;
+ memset(rxq->buffer_ring, 0, rxq->ring_depth * sizeof(rxq->buffer_ring[0]));
+}
+
+static inline void sxe2_rx_queue_vec_init(struct sxe2_rx_queue *rxq)
+{
+ uintptr_t data;
+ struct rte_mbuf mbuf_def;
+
+ memset(&mbuf_def, 0, sizeof(mbuf_def));
+ mbuf_def.buf_addr = 0;
+ mbuf_def.nb_segs = 1;
+ mbuf_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mbuf_def, 1);
+ rte_compiler_barrier();
+ data = (uintptr_t)&mbuf_def.rearm_data;
+ rxq->mbuf_init_value = *(uint64_t *)data;
+}
+
+int32_t __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_rx_queue *rxq = NULL;
+ int32_t ret = 0;
+ uint16_t i;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ PMD_LOG_INFO(RX, "Failed to prepare rx queue, rxq[%d] is NULL", i);
+ continue;
+ }
+ rxq->ops.mbufs_release = sxe2_rx_queue_mbufs_release_vec;
+ sxe2_rx_queue_vec_init(rxq);
+ }
+ return ret;
+}
+
+int32_t __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags)
+{
+ struct sxe2_tx_queue *txq;
+ int32_t ret = 0;
+ uint32_t i;
+
+ *vec_flags = SXE2_TX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+ if (txq->rs_thresh < SXE2_TX_RS_THRESH_MIN_VEC ||
+ txq->rs_thresh > SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_TX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+static void sxe2_tx_queue_mbufs_release_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ uint16_t i;
+
+ if (unlikely(txq == NULL || txq->buffer_ring == NULL)) {
+ PMD_LOG_ERR(TX, "Tx release mbufs vec, invalid params.");
+ return;
+ }
+ i = txq->next_dd - (txq->rs_thresh - 1);
+ buffer = txq->buffer_ring;
+ if (txq->next_use < i) {
+ for ( ; i < txq->ring_depth; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+ i = 0;
+ }
+ for (; i < txq->next_use; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+}
+
+int32_t __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_tx_queue *txq = NULL;
+ int32_t ret = 0;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL) {
+ PMD_LOG_INFO(TX, "Failed to prepare tx queue, txq[%d] is NULL", i);
+ continue;
+ }
+ txq->ops.mbufs_release = sxe2_tx_queue_mbufs_release_vec;
+ }
+ return ret;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.h b/drivers/net/sxe2/sxe2_txrx_vec.h
new file mode 100644
index 0000000000..4aef93d140
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef _SXE2_TXRX_VEC_H_
+#define _SXE2_TXRX_VEC_H_
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+
+#define SXE2_RX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_RX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_RX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_RX_MODE_BATCH_ALLOC RTE_BIT32(10)
+#define SXE2_RX_MODE_VEC_SET_MASK (SXE2_RX_MODE_VEC_SIMPLE | \
+ SXE2_RX_MODE_VEC_OFFLOAD | SXE2_RX_MODE_VEC_SSE)
+#define SXE2_TX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_TX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_TX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_TX_MODE_SIMPLE_BATCH RTE_BIT32(10)
+#define SXE2_TX_MODE_VEC_SET_MASK (SXE2_TX_MODE_VEC_SIMPLE | \
+ SXE2_TX_MODE_VEC_OFFLOAD | SXE2_TX_MODE_VEC_SSE)
+#define SXE2_TX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS | \
+ RTE_ETH_TX_OFFLOAD_QINQ_INSERT | \
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_SECURITY | \
+ RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM)
+#define SXE2_TX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_VLAN_INSERT | \
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM)
+#define SXE2_RX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP | \
+ RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT | \
+ RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_SECURITY | \
+ RTE_ETH_RX_OFFLOAD_QINQ_STRIP)
+#define SXE2_RX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_CHECKSUM | \
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_VLAN_STRIP | \
+ RTE_ETH_RX_OFFLOAD_VLAN_FILTER | \
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)
+#ifdef RTE_ARCH_X86
+uint16_t sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+uint16_t sxe2_tx_pkts_vec_sse_simple(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+uint16_t sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
+#endif
+int32_t __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags);
+int32_t __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev);
+int32_t __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags);
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload);
+int32_t __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev);
+
+#endif /* __SXE2_TXRX_VEC_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_common.h b/drivers/net/sxe2/sxe2_txrx_vec_common.h
new file mode 100644
index 0000000000..1ce687e09f
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_common.h
@@ -0,0 +1,235 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TXRX_VEC_COMMON_H__
+#define __SXE2_TXRX_VEC_COMMON_H__
+#include <rte_atomic.h>
+#ifdef PCLINT
+#include "avx_stub.h"
+#endif
+#include "sxe2_rx.h"
+#include "sxe2_queue.h"
+#include "sxe2_tx.h"
+#include "sxe2_vsi.h"
+#include "sxe2_ethdev.h"
+#define SXE2_RX_NUM_PER_LOOP_SSE 4
+#define SXE2_RX_NUM_PER_LOOP_AVX 8
+#define SXE2_RX_NUM_PER_LOOP_NEON 4
+#define SXE2_RX_REARM_THRESH_VEC 64
+#define SXE2_RX_PKTS_BURST_BATCH_NUM_VEC 32
+#define SXE2_TX_RS_THRESH_MIN_VEC 32
+#define SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC 64
+
+static __rte_always_inline void
+sxe2_tx_pkts_mbuf_fill(struct sxe2_tx_buffer *buffer,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ uint16_t i;
+ for (i = 0; i < nb_pkts; ++i)
+ buffer[i].mbuf = tx_pkts[i];
+}
+
+static __rte_always_inline int32_t
+sxe2_tx_bufs_free_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC];
+ int32_t ret;
+ uint32_t i;
+ uint16_t rs_thresh;
+ uint16_t free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - (rs_thresh - 1)];
+ mbuf = rte_pktmbuf_prefree_seg(buffer[0].mbuf);
+ if (likely(mbuf)) {
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (likely(mbuf)) {
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline void
+sxe2_tx_desc_fill_offloads(struct rte_mbuf *mbuf, uint64_t *desc_qw1)
+{
+ uint64_t offloads = mbuf->ol_flags;
+ uint32_t desc_cmd = 0;
+ uint32_t desc_offset = 0;
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ }
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ default:
+ break;
+ }
+ *desc_qw1 |= ((uint64_t)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ *desc_qw1 |= ((uint64_t)mbuf->vlan_tci) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT;
+ }
+ *desc_qw1 |= ((uint64_t)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT;
+}
+#define SXE2_RX_UMBCAST_FLAGS_VAL_GET(_flags) \
+ (((_flags) & 0x30) >> 4)
+
+static inline void sxe2_vf_rx_vec_sw_stats_cnt(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf *mbuf, uint8_t umbcast_flag)
+{
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN, rte_memory_order_relaxed);
+ switch (SXE2_RX_UMBCAST_FLAGS_VAL_GET(umbcast_flag)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static inline uint16_t
+sxe2_rx_pkts_refactor(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **mbuf_bufs, uint16_t mbuf_num,
+ uint8_t *split_rxe_flags, uint8_t *umbcast_flags)
+{
+ struct rte_mbuf *done_pkts[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ struct rte_mbuf *first_seg = rxq->pkt_first_seg;
+ struct rte_mbuf *last_seg = rxq->pkt_last_seg;
+ struct rte_mbuf *tmp_seg;
+ uint16_t done_num, buf_idx;
+ done_num = 0;
+ for (buf_idx = 0; buf_idx < mbuf_num; buf_idx++) {
+ if (last_seg) {
+ last_seg->next = mbuf_bufs[buf_idx];
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ first_seg->nb_segs++;
+ first_seg->pkt_len += mbuf_bufs[buf_idx]->data_len;
+ last_seg = last_seg->next;
+ if (split_rxe_flags[buf_idx] == 0) {
+ first_seg->hash = last_seg->hash;
+ first_seg->vlan_tci = last_seg->vlan_tci;
+ first_seg->ol_flags = last_seg->ol_flags;
+ first_seg->pkt_len -= rxq->crc_len;
+ if (last_seg->data_len > rxq->crc_len) {
+ last_seg->data_len -= rxq->crc_len;
+ } else {
+ tmp_seg = first_seg;
+ first_seg->nb_segs--;
+ while (tmp_seg->next != last_seg)
+ tmp_seg = tmp_seg->next;
+ tmp_seg->data_len -= (rxq->crc_len - last_seg->data_len);
+ tmp_seg->next = NULL;
+ rte_pktmbuf_free_seg(last_seg);
+ last_seg = NULL;
+ }
+ done_pkts[done_num++] = first_seg;
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, first_seg, umbcast_flags[buf_idx]);
+ first_seg = NULL;
+ last_seg = NULL;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ continue;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ last_seg = NULL;
+ continue;
+ }
+ } else {
+ if (split_rxe_flags[buf_idx] == 0) {
+ done_pkts[done_num++] = mbuf_bufs[buf_idx];
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, mbuf_bufs[buf_idx],
+ umbcast_flags[buf_idx]);
+ continue;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ first_seg = mbuf_bufs[buf_idx];
+ last_seg = first_seg;
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ mbuf_bufs[buf_idx]->pkt_len += rxq->crc_len;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ mbuf_bufs[buf_idx]->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free_seg(mbuf_bufs[buf_idx]);
+ continue;
+ }
+ }
+ }
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+ rte_memcpy(mbuf_bufs, done_pkts, done_num * (sizeof(struct rte_mbuf *)));
+ return done_num;
+}
+#endif /* __SXE2_TXRX_VEC_COMMON_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_sse.c b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
new file mode 100644
index 0000000000..f6e3f45937
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
@@ -0,0 +1,549 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <ethdev_driver.h>
+#include <rte_bitops.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_vect.h>
+#include "rte_common.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_queue.h"
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_vsi.h"
+
+static __rte_always_inline void
+sxe2_tx_desc_fill_one_sse(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf *pkt,
+ uint64_t desc_cmd, bool with_offloads)
+{
+ __m128i data_desc;
+ uint64_t desc_qw1;
+ uint32_t desc_offset;
+ desc_qw1 = (SXE2_TX_DESC_DTYPE_DATA |
+ ((uint64_t)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT |
+ ((uint64_t)pkt->data_len) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL(pkt->l2_len);
+ desc_qw1 |= ((uint64_t)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (with_offloads)
+ sxe2_tx_desc_fill_offloads(pkt, &desc_qw1);
+ data_desc = _mm_set_epi64x(desc_qw1, rte_pktmbuf_iova(pkt));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, desc), data_desc);
+}
+
+static __rte_always_inline uint16_t
+sxe2_tx_pkts_vec_sse_batch(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts, bool with_offloads)
+{
+ volatile union sxe2_tx_data_desc *desc;
+ struct sxe2_tx_buffer *buffer;
+ uint16_t next_use;
+ uint16_t res_num;
+ uint16_t tx_num;
+ uint16_t i;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free_vec(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx pkts sse batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ tx_num = nb_pkts;
+ next_use = txq->next_use;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ txq->desc_free_num -= nb_pkts;
+ res_num = txq->ring_depth - txq->next_use;
+ if (tx_num >= res_num) {
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, res_num);
+ for (i = 0; i < res_num - 1; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts++,
+ (SXE2_TX_DATA_DESC_CMD_EOP | SXE2_TX_DATA_DESC_CMD_RS),
+ with_offloads);
+ tx_num -= res_num;
+ next_use = 0;
+ txq->next_rs = txq->rs_thresh - 1;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ }
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, tx_num);
+ for (i = 0; i < tx_num; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ next_use += tx_num;
+ if (next_use > txq->next_rs) {
+ txq->desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ }
+ txq->next_use = next_use;
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, nb_pkts);
+l_end:
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+sxe2_tx_pkts_vec_sse_common(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts, bool with_offloads)
+{
+ uint16_t tx_done_num = 0;
+ uint16_t tx_once_num;
+ uint16_t tx_need_num;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, txq->rs_thresh);
+ tx_once_num = sxe2_tx_pkts_vec_sse_batch(txq,
+ tx_pkts + tx_done_num,
+ tx_need_num, with_offloads);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+ return tx_done_num;
+}
+
+uint16_t sxe2_tx_pkts_vec_sse_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, false);
+}
+uint16_t sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, true);
+}
+
+static inline void sxe2_rx_queue_rearm_sse(struct sxe2_rx_queue *rxq)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ struct rte_mbuf *mbuf0, *mbuf1;
+ __m128i dma_addr0, dma_addr1;
+ __m128i virt_addr0, virt_addr1;
+ __m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
+ RTE_PKTMBUF_HEADROOM);
+ int32_t ret;
+ uint16_t i;
+ uint16_t new_tail;
+
+ buffer = &rxq->buffer_ring[rxq->realloc_start];
+ desc = &rxq->desc_ring[rxq->realloc_start];
+ ret = rte_mempool_get_bulk(rxq->mb_pool, (void *)buffer,
+ SXE2_RX_REARM_THRESH_VEC);
+ if (ret != 0) {
+ PMD_LOG_INFO(RX, "Rx mbuf vec alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ if ((rxq->realloc_num + SXE2_RX_REARM_THRESH_VEC) >= rxq->ring_depth) {
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < SXE2_RX_NUM_PER_LOOP_SSE; ++i) {
+ buffer[i] = &rxq->fake_mbuf;
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc[i].read),
+ dma_addr0);
+ }
+ }
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed +=
+ SXE2_RX_REARM_THRESH_VEC;
+ goto l_end;
+ }
+ for (i = 0; i < SXE2_RX_REARM_THRESH_VEC; i += 2, buffer += 2) {
+ mbuf0 = buffer[0];
+ mbuf1 = buffer[1];
+#if RTE_IOVA_IN_MBUF
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+#endif
+ virt_addr0 = _mm_loadu_si128((__m128i *)&mbuf0->buf_addr);
+ virt_addr1 = _mm_loadu_si128((__m128i *)&mbuf1->buf_addr);
+#if RTE_IOVA_IN_MBUF
+ dma_addr0 = _mm_unpackhi_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpackhi_epi64(virt_addr1, virt_addr1);
+#else
+ dma_addr0 = _mm_unpacklo_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpacklo_epi64(virt_addr1, virt_addr1);
+#endif
+ dma_addr0 = _mm_add_epi64(dma_addr0, hdr_room);
+ dma_addr1 = _mm_add_epi64(dma_addr1, hdr_room);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr0);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr1);
+ }
+ rxq->realloc_start += SXE2_RX_REARM_THRESH_VEC;
+ if (rxq->realloc_start >= rxq->ring_depth)
+ rxq->realloc_start = 0;
+ rxq->realloc_num -= SXE2_RX_REARM_THRESH_VEC;
+ new_tail = (rxq->realloc_start == 0) ?
+ (rxq->ring_depth - 1) : (rxq->realloc_start - 1);
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, new_tail);
+l_end:
+ return;
+}
+
+static __rte_always_inline __m128i
+sxe2_rx_desc_fnav_flags_sse(__m128i descs_arr[4])
+{
+ __m128i descs_tmp1, descs_tmp2;
+ __m128i descs_fnav_vld;
+ __m128i v_zeros, v_ffff, v_u32_one;
+ __m128i m_flags;
+ const __m128i fdir_flags = _mm_set1_epi32(RTE_MBUF_F_RX_FDIR | RTE_MBUF_F_RX_FDIR_ID);
+ descs_tmp1 = _mm_unpacklo_epi32(descs_arr[0], descs_arr[1]);
+ descs_tmp2 = _mm_unpacklo_epi32(descs_arr[2], descs_arr[3]);
+ descs_fnav_vld = _mm_unpacklo_epi64(descs_tmp1, descs_tmp2);
+ descs_fnav_vld = _mm_slli_epi32(descs_fnav_vld, 26);
+ descs_fnav_vld = _mm_srli_epi32(descs_fnav_vld, 31);
+ v_zeros = _mm_setzero_si128();
+ v_ffff = _mm_cmpeq_epi32(v_zeros, v_zeros);
+ v_u32_one = _mm_srli_epi32(v_ffff, 31);
+ m_flags = _mm_cmpeq_epi32(descs_fnav_vld, v_u32_one);
+ m_flags = _mm_and_si128(m_flags, fdir_flags);
+ return m_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_desc_offloads_para_fill_sse(struct sxe2_rx_queue *rxq,
+ volatile union sxe2_rx_desc *desc __rte_unused,
+ __m128i descs_arr[4],
+ struct rte_mbuf **rx_pkts)
+{
+ const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_init_value);
+ __m128i rearm_arr[4];
+ __m128i tmp_desc_lo, tmp_desc_hi, flags, tmp_flags;
+ const __m128i desc_flags_mask = _mm_set_epi32(0x00001C04, 0x00001C04,
+ 0x00001C04, 0x00001C04);
+ const __m128i desc_flags_rss_mask = _mm_set_epi32(0x20000000, 0x20000000,
+ 0x20000000, 0x20000000);
+ const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ 0, 0, 0, 0);
+ const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, RTE_MBUF_F_RX_RSS_HASH,
+ 0, 0, 0, 0);
+ const __m128i cksum_flags =
+ _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1));
+ const __m128i cksum_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD);
+ const __m128i vlan_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED);
+ flags = _mm_unpackhi_epi32(descs_arr[0], descs_arr[1]);
+ tmp_flags = _mm_unpackhi_epi32(descs_arr[2], descs_arr[3]);
+ tmp_desc_lo = _mm_unpacklo_epi64(flags, tmp_flags);
+ tmp_desc_hi = _mm_unpackhi_epi64(flags, tmp_flags);
+ tmp_desc_lo = _mm_and_si128(tmp_desc_lo, desc_flags_mask);
+ tmp_desc_hi = _mm_and_si128(tmp_desc_hi, desc_flags_rss_mask);
+ tmp_flags = _mm_shuffle_epi8(vlan_flags, tmp_desc_lo);
+ flags = _mm_and_si128(tmp_flags, vlan_mask);
+ tmp_desc_lo = _mm_srli_epi32(tmp_desc_lo, 10);
+ tmp_flags = _mm_shuffle_epi8(cksum_flags, tmp_desc_lo);
+ tmp_flags = _mm_slli_epi32(tmp_flags, 1);
+ tmp_flags = _mm_and_si128(tmp_flags, cksum_mask);
+ flags = _mm_or_si128(flags, tmp_flags);
+ tmp_desc_hi = _mm_srli_epi32(tmp_desc_hi, 27);
+ tmp_flags = _mm_shuffle_epi8(rss_flags, tmp_desc_hi);
+ flags = _mm_or_si128(flags, tmp_flags);
+#ifndef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+ if (rxq->fnav_enable) {
+ __m128i tmp_fnav_flags = sxe2_rx_desc_fnav_flags_sse(descs_arr);
+ flags = _mm_or_si128(flags, tmp_fnav_flags);
+ rx_pkts[0]->hash.fdir.hi = desc[0].wb.fd_filter_id;
+ rx_pkts[1]->hash.fdir.hi = desc[1].wb.fd_filter_id;
+ rx_pkts[2]->hash.fdir.hi = desc[2].wb.fd_filter_id;
+ rx_pkts[3]->hash.fdir.hi = desc[3].wb.fd_filter_id;
+ }
+#endif
+ rearm_arr[0] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x30);
+ rearm_arr[1] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x30);
+ rearm_arr[2] = _mm_blend_epi16(mbuf_init, flags, 0x30);
+ rearm_arr[3] = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x30);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[0]->rearm_data), rearm_arr[0]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[1]->rearm_data), rearm_arr[1]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[2]->rearm_data), rearm_arr[2]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[3]->rearm_data), rearm_arr[3]);
+}
+
+static inline uint16_t
+sxe2_rx_pkts_common_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts, uint8_t *split_rxe_flags,
+ uint8_t *umbcast_flags)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ __m128i descs_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i mbuf_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
+ __m128i pmbuf0;
+ __m128i ptype_all;
+#ifdef RTE_ARCH_X86_64
+ __m128i pmbuf1;
+#endif
+ uint32_t i;
+ uint32_t bit_num;
+ uint16_t done_num = 0;
+ const uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ const __m128i crc_adjust =
+ _mm_set_epi16(0, 0, 0,
+ -rxq->crc_len,
+ 0, -rxq->crc_len,
+ 0, 0);
+ const __m128i rvp_shuf_mask =
+ _mm_set_epi8(7, 6, 5, 4,
+ 3, 2,
+ 13, 12,
+ 0XFF, 0xFF, 13, 12,
+ 0xFF, 0xFF, 0xFF, 0xFF);
+ const __m128i dd_mask = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ const __m128i eop_mask = _mm_slli_epi32(dd_mask,
+ SXE2_RX_DESC_STATUS_EOP_SHIFT);
+ const __m128i rxe_mask = _mm_set_epi64x(0x0000208000002080LL,
+ 0x0000208000002080LL);
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+ const __m128i ptype_mask = _mm_set_epi16(SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ rte_prefetch0(desc);
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, SXE2_RX_NUM_PER_LOOP_SSE);
+ if (rxq->realloc_num > SXE2_RX_REARM_THRESH_VEC)
+ sxe2_rx_queue_rearm_sse(rxq);
+ if ((rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK) == 0)
+ goto l_end;
+ buffer = &rxq->buffer_ring[rxq->processing_idx];
+ for (i = 0; i < nb_pkts; i += SXE2_RX_NUM_PER_LOOP_SSE,
+ desc += SXE2_RX_NUM_PER_LOOP_SSE) {
+ pmbuf0 = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, &buffer[i]));
+ descs_arr[3] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 3));
+ rte_compiler_barrier();
+ _mm_storeu_si128((__m128i *)&rx_pkts[i], pmbuf0);
+#ifdef RTE_ARCH_X86_64
+ pmbuf1 = _mm_loadu_si128((__m128i *)&buffer[i + 2]);
+#endif
+ descs_arr[2] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 2));
+ rte_compiler_barrier();
+ descs_arr[1] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 1));
+ rte_compiler_barrier();
+ descs_arr[0] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc));
+#ifdef RTE_ARCH_X86_64
+ _mm_storeu_si128((__m128i *)&rx_pkts[i + 2], pmbuf1);
+#endif
+ if (split_rxe_flags) {
+ rte_mbuf_prefetch_part2(rx_pkts[i]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 1]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 2]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 3]);
+ }
+ rte_compiler_barrier();
+ mbuf_arr[3] = _mm_shuffle_epi8(descs_arr[3], rvp_shuf_mask);
+ mbuf_arr[2] = _mm_shuffle_epi8(descs_arr[2], rvp_shuf_mask);
+ mbuf_arr[1] = _mm_shuffle_epi8(descs_arr[1], rvp_shuf_mask);
+ mbuf_arr[0] = _mm_shuffle_epi8(descs_arr[0], rvp_shuf_mask);
+ sterr_tmp2 = _mm_unpackhi_epi32(descs_arr[3], descs_arr[2]);
+ sterr_tmp1 = _mm_unpackhi_epi32(descs_arr[1], descs_arr[0]);
+ sxe2_rx_desc_offloads_para_fill_sse(rxq, desc, descs_arr, rx_pkts);
+ mbuf_arr[3] = _mm_add_epi16(mbuf_arr[3], crc_adjust);
+ mbuf_arr[2] = _mm_add_epi16(mbuf_arr[2], crc_adjust);
+ mbuf_arr[1] = _mm_add_epi16(mbuf_arr[1], crc_adjust);
+ mbuf_arr[0] = _mm_add_epi16(mbuf_arr[0], crc_adjust);
+ staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
+ ptype_all = _mm_and_si128(staterr, ptype_mask);
+ _mm_storeu_si128((void *)&rx_pkts[i + 3]->rx_descriptor_fields1,
+ mbuf_arr[3]);
+ _mm_storeu_si128((void *)&rx_pkts[i + 2]->rx_descriptor_fields1,
+ mbuf_arr[2]);
+ if (umbcast_flags != NULL) {
+ const __m128i umbcast_mask =
+ _mm_set_epi32(SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK);
+ const __m128i umbcast_shuf_mask =
+ _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x07, 0x0F,
+ 0x03, 0x0B);
+ __m128i umbcast_bits = _mm_and_si128(staterr, umbcast_mask);
+ umbcast_bits = _mm_shuffle_epi8(umbcast_bits, umbcast_shuf_mask);
+ *(int32_t *)umbcast_flags = _mm_cvtsi128_si32(umbcast_bits);
+ umbcast_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ if (split_rxe_flags != NULL) {
+ __m128i eop_bits = _mm_andnot_si128(staterr, eop_mask);
+ __m128i rxe_bits = _mm_and_si128(staterr, rxe_mask);
+ rxe_bits = _mm_srli_epi32(rxe_bits, 7);
+ eop_bits = _mm_or_si128(eop_bits, rxe_bits);
+ eop_bits = _mm_shuffle_epi8(eop_bits, eop_shuf_mask);
+ *(int32_t *)split_rxe_flags = _mm_cvtsi128_si32(eop_bits);
+ split_rxe_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ staterr = _mm_and_si128(staterr, dd_mask);
+ staterr = _mm_packs_epi32(staterr, _mm_setzero_si128());
+ _mm_storeu_si128((void *)&rx_pkts[i + 1]->rx_descriptor_fields1,
+ mbuf_arr[1]);
+ _mm_storeu_si128((void *)&rx_pkts[i]->rx_descriptor_fields1,
+ mbuf_arr[0]);
+ rx_pkts[i + 3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[i + 2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
+ rx_pkts[i + 1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[i]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ bit_num = rte_popcount64(_mm_cvtsi128_si64(staterr));
+ done_num += bit_num;
+ if (likely(bit_num != SXE2_RX_NUM_PER_LOOP_SSE))
+ break;
+ }
+ rxq->processing_idx += done_num;
+ rxq->processing_idx &= (rxq->ring_depth - 1);
+ rxq->realloc_num += done_num;
+ PMD_LOG_DEBUG(RX, "port_id=%u queue_id=%u last_id=%u recv_pkts=%d",
+ rxq->port_id, rxq->queue_id, rxq->processing_idx, done_num);
+l_end:
+ return done_num;
+}
+
+static __rte_always_inline uint16_t
+sxe2_rx_pkts_scattered_batch_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ const uint64_t *split_rxe_flags64;
+ uint8_t split_rxe_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ uint8_t umbcast_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ uint16_t rx_done_num;
+ uint16_t rx_pkt_done_num;
+ rx_pkt_done_num = 0;
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, umbcast_flags);
+ } else {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, NULL);
+ }
+ if (rx_done_num == 0)
+ goto l_end;
+ if (!rxq->vsi->adapter->devargs.sw_stats_en) {
+ split_rxe_flags64 = (uint64_t *)split_rxe_flags;
+ if (rxq->pkt_first_seg == NULL &&
+ split_rxe_flags64[0] == 0 &&
+ split_rxe_flags64[1] == 0 &&
+ split_rxe_flags64[2] == 0 &&
+ split_rxe_flags64[3] == 0) {
+ rx_pkt_done_num = rx_done_num;
+ goto l_end;
+ }
+ if (rxq->pkt_first_seg == NULL) {
+ while (rx_pkt_done_num < rx_done_num &&
+ split_rxe_flags[rx_pkt_done_num] == 0)
+ rx_pkt_done_num++;
+ if (rx_pkt_done_num == rx_done_num)
+ goto l_end;
+ rxq->pkt_first_seg = rx_pkts[rx_pkt_done_num];
+ }
+ }
+ rx_pkt_done_num += sxe2_rx_pkts_refactor(rxq, &rx_pkts[rx_pkt_done_num],
+ rx_done_num - rx_pkt_done_num, &split_rxe_flags[rx_pkt_done_num],
+ &umbcast_flags[rx_pkt_done_num]);
+l_end:
+ return rx_pkt_done_num;
+}
+
+uint16_t sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ uint16_t done_num = 0;
+ uint16_t once_num;
+
+ while (nb_pkts > SXE2_RX_PKTS_BURST_BATCH_NUM_VEC) {
+ once_num =
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num,
+ SXE2_RX_PKTS_BURST_BATCH_NUM_VEC);
+ done_num += once_num;
+ nb_pkts -= once_num;
+ if (once_num < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC)
+ goto l_end;
+ }
+ done_num +=
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num, nb_pkts);
+l_end:
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
* [PATCH v18 09/11] drivers: add data path for Rx and Tx
From: liujie5 @ 2026-05-19 14:48 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260519144810.3951202-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Implement receive and transmit burst functions for sxe2 PMD.
Add sxe2_recv_pkts and sxe2_xmit_pkts as the primary data path
interfaces.
The implementation includes:
- Efficient descriptor fetching and mbuf allocation for Rx.
- Descriptor setup and checksum offload handling for Tx.
- Buffer recycling and hardware tail pointer updates.
- Performance-oriented loop unrolling and prefetching where applicable.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_ioctl_chnl.c | 8 +-
drivers/net/sxe2/meson.build | 2 +
drivers/net/sxe2/sxe2_ethdev.c | 11 +-
drivers/net/sxe2/sxe2_txrx.c | 246 +++++++
drivers/net/sxe2/sxe2_txrx.h | 21 +
drivers/net/sxe2/sxe2_txrx_poll.c | 916 ++++++++++++++++++++++++++
6 files changed, 1199 insertions(+), 5 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx.c
create mode 100644 drivers/net/sxe2/sxe2_txrx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.c
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
index a40f9b8da2..c02e73f1f7 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl.c
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -177,13 +177,13 @@ void
goto l_err;
}
- PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=0x%zx, src=0x%"PRIx64", offset=0x%"PRIx64"",
+ PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=%"PRIu64", src=0x%"PRIx64", offset=0x%"PRIx64"",
bar_idx, cmd_fd, len, offset, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
virt = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_SHARED, cmd_fd, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
if (virt == MAP_FAILED) {
- PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=0x%zx, offset=0x%"PRIx64", err:%s",
+ PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=%"PRIu64", offset=0x%"PRIx64", err:%s",
cmd_fd, len, offset, strerror(errno));
goto l_err;
}
@@ -205,12 +205,12 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, uint64_t len)
goto l_end;
}
- PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%zx",
+ PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%"PRIx64"",
virt, len);
ret = munmap(virt, len);
if (ret < 0) {
- PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=0x%zx, err:%s",
+ PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=%"PRIu64", err:%s",
virt, len, strerror(errno));
ret = -errno;
goto l_end;
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 3dfe54903a..5645e3ad61 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -20,6 +20,8 @@ sources += files(
'sxe2_queue.c',
'sxe2_tx.c',
'sxe2_rx.c',
+ 'sxe2_txrx_poll.c',
+ 'sxe2_txrx.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 6abb4672f6..e47e788d78 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -26,6 +26,7 @@
#include "sxe2_cmd_chnl.h"
#include "sxe2_tx.h"
#include "sxe2_rx.h"
+#include "sxe2_txrx.h"
#include "sxe2_common.h"
#include "sxe2_common_log.h"
#include "sxe2_host_regs.h"
@@ -137,6 +138,9 @@ static int32_t sxe2_dev_start(struct rte_eth_dev *dev)
goto l_end;
}
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
+
ret = sxe2_queues_start(dev);
if (ret) {
PMD_LOG_ERR(INIT, "enable queues failed");
@@ -763,10 +767,15 @@ static int32_t sxe2_dev_init(struct rte_eth_dev *dev,
PMD_INIT_FUNC_TRACE();
+ sxe2_set_common_function(dev);
+
dev->dev_ops = &sxe2_eth_dev_ops;
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
goto l_end;
+ }
ret = sxe2_hw_init(dev);
if (ret) {
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
new file mode 100644
index 0000000000..0384c2091e
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -0,0 +1,246 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include <unistd.h>
+
+#include "sxe2_txrx.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_ethdev.h"
+
+#include "sxe2_common_log.h"
+#include "sxe2_osal.h"
+#include "sxe2_cmd_chnl.h"
+#if defined(RTE_ARCH_ARM64)
+#include <rte_cpuflags.h>
+#endif
+
+static int32_t sxe2_tx_descriptor_status(void *tx_queue, uint16_t offset)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ int32_t ret;
+ uint16_t desc_idx;
+
+ if (unlikely(offset >= txq->ring_depth)) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ desc_idx = txq->next_use + offset;
+ desc_idx = SXE2_DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
+ if (desc_idx >= txq->ring_depth) {
+ desc_idx -= txq->ring_depth;
+ if (desc_idx >= txq->ring_depth)
+ desc_idx -= txq->ring_depth;
+ }
+
+ if (desc_idx == 0)
+ desc_idx = txq->rs_thresh - 1;
+ else
+ desc_idx -= 1;
+
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
+ (txq->desc_ring[desc_idx].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
+ ret = RTE_ETH_TX_DESC_DONE;
+ else
+ ret = RTE_ETH_TX_DESC_FULL;
+
+l_end:
+ return ret;
+}
+
+static inline int32_t sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
+{
+ struct rte_mbuf *m_seg = mbuf;
+
+ while (m_seg != NULL) {
+ if (m_seg->data_len == 0)
+ return -EINVAL;
+ m_seg = m_seg->next;
+ }
+
+ return 0;
+}
+
+uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct rte_mbuf *mbuf;
+ uint64_t ol_flags = 0;
+ int32_t ret = 0;
+ int32_t i = 0;
+
+ for (i = 0; i < nb_pkts; i++) {
+ mbuf = tx_pkts[i];
+ if (!mbuf)
+ continue;
+ ol_flags = mbuf->ol_flags;
+ if (!(ol_flags & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))) {
+ if (mbuf->nb_segs > SXE2_TX_MTU_SEG_MAX ||
+ mbuf->pkt_len > SXE2_FRAME_SIZE_MAX) {
+ rte_errno = -EINVAL;
+ goto l_end;
+ }
+ } else if ((mbuf->tso_segsz < SXE2_MIN_TSO_MSS) ||
+ (mbuf->tso_segsz > SXE2_MAX_TSO_MSS) ||
+ (mbuf->nb_segs > txq->ring_depth) ||
+ (mbuf->pkt_len > SXE2_TX_TSO_PKTLEN_MAX)) {
+ rte_errno = -EINVAL;
+ goto l_end;
+ }
+
+ if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
+ rte_errno = -EINVAL;
+ goto l_end;
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_TX
+ ret = rte_validate_tx_offload(mbuf);
+ if (ret != 0) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+#endif
+ ret = rte_net_intel_cksum_prepare(mbuf);
+ if (ret != 0) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+
+ ret = sxe2_tx_mbuf_empty_check(mbuf);
+ if (ret != 0) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return i;
+}
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ uint32_t tx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
+ PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
+ tx_mode_flags, dev->data->port_id);
+}
+
+static int32_t sxe2_rx_descriptor_status(void *rx_queue, uint16_t offset)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ int32_t ret;
+
+ if (unlikely(offset >= rxq->ring_depth)) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (offset >= rxq->ring_depth - rxq->hold_num) {
+ ret = RTE_ETH_RX_DESC_UNAVAIL;
+ goto l_end;
+ }
+
+ if (rxq->processing_idx + offset >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
+ else
+ desc = &rxq->desc_ring[rxq->processing_idx + offset];
+
+ if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
+ ret = RTE_ETH_RX_DESC_DONE;
+ else
+ ret = RTE_ETH_RX_DESC_AVAIL;
+
+l_end:
+ PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
+ offset, ret, rxq->queue_id, rxq->port_id);
+ return ret;
+}
+
+static int32_t sxe2_rx_queue_count(void *rx_queue)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ uint16_t done_num = 0;
+
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ while ((done_num < rxq->ring_depth) &&
+ (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK)) {
+ done_num += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ if (rxq->processing_idx + done_num >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + done_num - rxq->ring_depth];
+ else
+ desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ }
+
+ PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
+ done_num, rxq->queue_id, rxq->port_id);
+
+ return done_num;
+}
+
+static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (0 != (rxq->offloads & offload)) {
+ en = true;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return en;
+}
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ uint32_t rx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
+ else
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
+
+ PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
+ rx_mode_flags, dev->data->port_id);
+ adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
+}
+
+void sxe2_set_common_function(struct rte_eth_dev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+
+ dev->rx_queue_count = sxe2_rx_queue_count;
+ dev->rx_descriptor_status = sxe2_rx_descriptor_status;
+
+ dev->tx_descriptor_status = sxe2_tx_descriptor_status;
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
new file mode 100644
index 0000000000..f6558e2189
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef SXE2_TXRX_H
+#define SXE2_TXRX_H
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+
+void sxe2_set_common_function(struct rte_eth_dev *dev);
+
+uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
+
+#endif /* __SXE2_TXRX_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.c b/drivers/net/sxe2/sxe2_txrx_poll.c
new file mode 100644
index 0000000000..ce56f9086f
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_poll.c
@@ -0,0 +1,916 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include "sxe2_osal.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_txrx.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+
+static __rte_always_inline int32_t
+sxe2_tx_bufs_free(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX];
+ int32_t ret;
+ uint32_t i;
+ uint16_t rs_thresh;
+ uint16_t free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - rs_thresh + 1];
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {
+ if (likely(rs_thresh <= SXE2_TX_FREE_BUFFER_SIZE_MAX)) {
+ mbuf = buffer[0].mbuf;
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = buffer[i].mbuf;
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ rte_mempool_put(buffer->mbuf->pool, buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline int32_t sxe2_tx_cleanup(struct sxe2_tx_queue *txq)
+{
+ int32_t ret = 0;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ struct sxe2_tx_buffer *buffer_ring = txq->buffer_ring;
+ uint16_t ring_depth = txq->ring_depth;
+ uint16_t next_clean = txq->next_clean;
+ uint16_t clean_last;
+ uint16_t clean_num;
+
+ clean_last = next_clean + txq->rs_thresh;
+ if (clean_last >= ring_depth)
+ clean_last = clean_last - ring_depth;
+
+ clean_last = buffer_ring[clean_last].last_id;
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) !=
+ (txq->desc_ring[clean_last].wb.dd & rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK))) {
+ PMD_LOG_DEBUG(TX, "desc[%u] is not done.port_id=%u queue_id=%u val=0x%" PRIx64,
+ clean_last, txq->port_id,
+ txq->queue_id, txq->desc_ring[clean_last].wb.dd);
+ ret = -1;
+ goto l_end;
+ }
+
+ if (clean_last > next_clean)
+ clean_num = clean_last - next_clean;
+ else
+ clean_num = ring_depth - next_clean + clean_last;
+
+ desc_ring[clean_last].wb.dd = 0;
+
+ txq->next_clean = clean_last;
+ txq->desc_free_num += clean_num;
+
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+static __rte_always_inline uint16_t
+sxe2_tx_pkt_data_desc_count(struct rte_mbuf *tx_pkt)
+{
+ struct rte_mbuf *m_seg = tx_pkt;
+ uint16_t count = 0;
+
+ while (m_seg != NULL) {
+ count += SXE2_DIV_ROUND_UP(m_seg->data_len,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC);
+ m_seg = m_seg->next;
+ }
+
+ return count;
+}
+
+static __rte_always_inline void
+sxe2_tx_desc_checksum_fill(uint64_t offloads, uint32_t *desc_cmd, uint32_t *desc_offset,
+ union sxe2_tx_offload_info ol_info)
+{
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ }
+
+ if (offloads & RTE_MBUF_F_TX_TCP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_UDP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ default:
+
+ break;
+ }
+
+l_end:
+ return;
+}
+
+static __rte_always_inline uint64_t
+sxe2_tx_data_desc_build_cobt(uint32_t cmd, uint32_t offset, uint16_t buf_size, uint16_t l2tag)
+{
+ return rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DATA |
+ (((uint64_t)cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT) |
+ (((uint64_t)offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT) |
+ (((uint64_t)buf_size) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT) |
+ (((uint64_t)l2tag) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT));
+}
+
+uint16_t sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct sxe2_tx_buffer *buffer_ring;
+ struct sxe2_tx_buffer *buffer;
+ struct sxe2_tx_buffer *next_buffer;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ volatile union sxe2_tx_data_desc *desc_ring;
+ volatile union sxe2_tx_data_desc *desc;
+ volatile struct sxe2_tx_context_desc *ctxt_desc;
+ union sxe2_tx_offload_info ol_info;
+ struct sxe2_vsi *vsi = txq->vsi;
+ rte_iova_t buf_dma_addr;
+ uint64_t offloads;
+ uint64_t desc_type_cmd_tso_mss;
+ uint32_t desc_cmd;
+ uint32_t desc_offset;
+ uint32_t desc_tag;
+ uint32_t desc_tunneling_params;
+ uint16_t ipsec_offset;
+ uint16_t ctxt_desc_num;
+ uint16_t desc_sum_num;
+ uint16_t tx_num;
+ uint16_t seg_len;
+ uint16_t next_use;
+ uint16_t last_use;
+ uint16_t desc_l2tag2;
+
+ buffer_ring = txq->buffer_ring;
+ desc_ring = txq->desc_ring;
+ next_use = txq->next_use;
+ buffer = &buffer_ring[next_use];
+
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_cleanup(txq);
+
+ for (tx_num = 0; tx_num < nb_pkts; tx_num++) {
+ tx_pkt = *tx_pkts++;
+ desc_cmd = 0;
+ desc_offset = 0;
+ desc_tag = 0;
+ desc_tunneling_params = 0;
+ ipsec_offset = 0;
+ offloads = tx_pkt->ol_flags;
+ ol_info.l2_len = tx_pkt->l2_len;
+ ol_info.l3_len = tx_pkt->l3_len;
+ ol_info.l4_len = tx_pkt->l4_len;
+ ol_info.tso_segsz = tx_pkt->tso_segsz;
+ ol_info.outer_l2_len = tx_pkt->outer_l2_len;
+ ol_info.outer_l3_len = tx_pkt->outer_l3_len;
+
+ ctxt_desc_num = (offloads &
+ SXE2_TX_OFFLOAD_CTXT_NEEDCK_MASK) ? 1 : 0;
+ if (unlikely(vsi->vsi_type == SXE2_VSI_T_DPDK_ESW))
+ ctxt_desc_num = 1;
+
+ if (offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))
+ desc_sum_num = sxe2_tx_pkt_data_desc_count(tx_pkt) + ctxt_desc_num;
+ else
+ desc_sum_num = tx_pkt->nb_segs + ctxt_desc_num;
+
+ last_use = next_use + desc_sum_num - 1;
+ if (last_use >= txq->ring_depth)
+ last_use = last_use - txq->ring_depth;
+
+ if (desc_sum_num > txq->desc_free_num) {
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_cleanup_exit;
+
+ if (unlikely(desc_sum_num > txq->rs_thresh)) {
+ while (desc_sum_num > txq->desc_free_num) {
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_cleanup_exit;
+ }
+ }
+ }
+
+ desc_offset |= SXE2_TX_DATA_DESC_MACLEN_VAL(ol_info.l2_len);
+
+ if (offloads & SXE2_TX_OFFLOAD_CKSUM_MASK) {
+ sxe2_tx_desc_checksum_fill(offloads, &desc_cmd,
+ &desc_offset, ol_info);
+ }
+
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ desc_tag = tx_pkt->vlan_tci;
+ }
+
+ if (ctxt_desc_num) {
+ ctxt_desc = (volatile struct sxe2_tx_context_desc *)
+ &desc_ring[next_use];
+ desc_l2tag2 = 0;
+ desc_type_cmd_tso_mss = SXE2_TX_DESC_DTYPE_CTXT;
+
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_QINQ) {
+ desc_l2tag2 = tx_pkt->vlan_tci_outer;
+ desc_type_cmd_tso_mss |= SXE2_TX_CTXT_DESC_CMD_IL2TAG2_MASK;
+ }
+
+ ctxt_desc->tunneling_params =
+ rte_cpu_to_le_32(desc_tunneling_params);
+ ctxt_desc->l2tag2 = rte_cpu_to_le_16(desc_l2tag2);
+ ctxt_desc->type_cmd_tso_mss = rte_cpu_to_le_64(desc_type_cmd_tso_mss);
+ ctxt_desc->ipsec_offset = rte_cpu_to_le_64(ipsec_offset);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ }
+
+ m_seg = tx_pkt;
+
+ do {
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ buffer->mbuf = m_seg;
+ seg_len = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ while ((offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG)) &&
+ unlikely(seg_len > SXE2_TX_MAX_DATA_NUM_PER_DESC)) {
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd, desc_offset,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC,
+ desc_tag);
+ buf_dma_addr += SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ seg_len -= SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ }
+
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd,
+ desc_offset, seg_len, desc_tag);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_EOP;
+ txq->desc_used_num += desc_sum_num;
+ txq->desc_free_num -= desc_sum_num;
+
+ if (txq->desc_used_num >= txq->rs_thresh) {
+ PMD_LOG_DEBUG(TX, "Tx pkts set RS bit."
+ "last_use=%u port_id=%u, queue_id=%u",
+ last_use, txq->port_id, txq->queue_id);
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_RS;
+ txq->desc_used_num = 0;
+ }
+
+ desc->read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(((uint64_t)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT);
+ }
+ goto l_end_of_tx;
+
+l_cleanup_exit:
+ if (tx_num == 0)
+ return 0;
+l_end_of_tx:
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, tx_num);
+
+ txq->next_use = next_use;
+ return tx_num;
+}
+
+static __rte_always_inline void
+sxe2_tx_data_desc_fill(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ uint32_t desc_offset;
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset, (*tx_pkts)->data_len, 0);
+}
+static __rte_always_inline void
+sxe2_tx_data_desc_fill_batch(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ uint32_t i;
+ uint32_t desc_offset;
+ for (i = 0; i < SXE2_TX_FILL_PER_LOOP; ++i, ++desc, ++tx_pkts) {
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset,
+ (*tx_pkts)->data_len,
+ 0);
+ }
+}
+
+static inline void sxe2_tx_ring_fill(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_buffer *buffer = &txq->buffer_ring[txq->next_use];
+ volatile union sxe2_tx_data_desc *desc = &txq->desc_ring[txq->next_use];
+ uint32_t i, j;
+ uint32_t mainpart;
+ uint32_t leftover;
+ mainpart = nb_pkts & ((uint32_t)~SXE2_TX_FILL_PER_LOOP_MASK);
+ leftover = nb_pkts & ((uint32_t)SXE2_TX_FILL_PER_LOOP_MASK);
+ for (i = 0; i < mainpart; i += SXE2_TX_FILL_PER_LOOP) {
+ for (j = 0; j < SXE2_TX_FILL_PER_LOOP; ++j)
+ (buffer + i + j)->mbuf = *(tx_pkts + i + j);
+ sxe2_tx_data_desc_fill_batch(desc + i, tx_pkts + i);
+ }
+ if (unlikely(leftover > 0)) {
+ for (i = 0; i < leftover; ++i) {
+ (buffer + mainpart + i)->mbuf = *(tx_pkts + mainpart + i);
+ sxe2_tx_data_desc_fill(desc + mainpart + i,
+ tx_pkts + mainpart + i);
+ }
+ }
+}
+
+static inline uint16_t sxe2_tx_pkts_batch(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ uint16_t res_num = 0;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ txq->desc_free_num -= nb_pkts;
+ if ((txq->next_use + nb_pkts) > txq->ring_depth) {
+ res_num = txq->ring_depth - txq->next_use;
+ sxe2_tx_ring_fill(txq, tx_pkts, res_num);
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs = txq->rs_thresh - 1;
+ txq->next_use = 0;
+ }
+ sxe2_tx_ring_fill(txq, tx_pkts + res_num, nb_pkts - res_num);
+ txq->next_use = txq->next_use + (nb_pkts - res_num);
+ if (txq->next_use > txq->next_rs) {
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ if (txq->next_rs >= txq->ring_depth)
+ txq->next_rs = txq->rs_thresh - 1;
+ }
+ if (txq->next_use >= txq->ring_depth)
+ txq->next_use = 0;
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, txq->next_use, nb_pkts);
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, txq->next_use);
+l_end:
+ return nb_pkts;
+}
+
+static inline void
+sxe2_update_rx_tail(struct sxe2_rx_queue *rxq, uint16_t hold_num, uint16_t rx_id)
+{
+ hold_num += rxq->hold_num;
+
+ if (hold_num > rxq->rx_free_thresh) {
+ rx_id = (uint16_t)((rx_id == 0) ? (rxq->ring_depth - 1) : (rx_id - 1));
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, rx_id);
+ hold_num = 0;
+ }
+ rxq->hold_num = hold_num;
+}
+
+static inline uint64_t
+sxe2_rx_desc_error_para(__rte_unused struct sxe2_rx_queue *rxq,
+ union sxe2_rx_desc *desc)
+{
+ uint64_t flags = 0;
+ uint64_t desc_qw1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (unlikely(0 == (desc_qw1 & SXE2_RX_DESC_STATUS_L3L4_P_MASK)))
+ goto l_end;
+
+ if (likely(0 == (desc->wb.rxdid_src & SXE2_RX_DESC_EUDPE_MASK)))
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+ else
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_QW1_ERRORS_MASK))) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD);
+ goto l_end;
+ }
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_IPE_MASK)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_L4_MASK)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+
+ if (unlikely(0 != (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_EIP_MASK)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+l_end:
+ return flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_mbuf_common_fields_fill(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ uint64_t qword1;
+ uint64_t pkt_flags;
+ qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+
+ mbuf->ol_flags = 0;
+ mbuf->packet_type = ptype_tbl[SXE2_RX_DESC_PTYPE_VAL_GET(qword1)];
+
+ pkt_flags = sxe2_rx_desc_error_para(rxq, rxd);
+
+ mbuf->ol_flags |= pkt_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_sw_stats_update(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ uint64_t qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ switch (SXE2_RX_DESC_STATUS_UMBCAST_VAL_GET(qword1)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+}
+
+uint16_t sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ uint64_t qword1;
+ uint16_t done_num;
+ uint16_t hold_num;
+ uint16_t cur_idx;
+ uint16_t pkt_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ PMD_LOG_INFO(RX, "Rx new_mbuf alloc failed port_id:%u "
+ "queue_id:%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+
+ rte_prefetch0(buffer_ring[cur_idx]);
+
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+
+ cur_mbuf = *cur_buffer;
+
+ *cur_buffer = new_mbuf;
+
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+
+ if (0 == (qword1 & SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len + pkt_len -
+ RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+ first_seg->port = rxq->port_id;
+
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
+
+uint16_t sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *cur_mbuf_pay;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *new_mbuf_pay = NULL;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ uint64_t qword1;
+ uint16_t done_num;
+ uint16_t hold_num;
+ uint16_t cur_idx;
+ uint16_t pkt_len;
+ uint16_t hdr_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ new_mbuf = NULL;
+
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) == 0 ||
+ first_seg == NULL) {
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ break;
+ }
+ }
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) {
+ new_mbuf_pay = rte_mbuf_raw_alloc(rxq->rx_seg[1].mp);
+ if (unlikely(new_mbuf_pay == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ if (new_mbuf != NULL)
+ rte_pktmbuf_free(new_mbuf);
+ new_mbuf = NULL;
+ break;
+ }
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+ rte_prefetch0(buffer_ring[cur_idx]);
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+ cur_mbuf = *cur_buffer;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ *cur_buffer = new_mbuf;
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ } else {
+ if (first_seg == NULL) {
+ *cur_buffer = new_mbuf;
+ new_mbuf->next = new_mbuf_pay;
+ new_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ } else {
+ cur_mbuf_pay = cur_mbuf->next;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ cur_mbuf->next = new_mbuf_pay;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(cur_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ cur_mbuf = cur_mbuf_pay;
+ }
+ }
+ new_mbuf = NULL;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ } else {
+ if (first_seg == NULL) {
+ cur_mbuf->nb_segs = 2;
+ cur_mbuf->next->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ hdr_len = SXE2_RX_DESC_HDR_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = hdr_len;
+ cur_mbuf->pkt_len = hdr_len + pkt_len;
+ cur_mbuf->next->data_len = pkt_len;
+ first_seg = cur_mbuf;
+ cur_mbuf = cur_mbuf->next;
+ last_seg = cur_mbuf;
+ } else {
+ cur_mbuf->nb_segs = 1;
+ cur_mbuf->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_RX
+
+ rte_pktmbuf_dump(stdout, first_seg, rte_pktmbuf_pkt_len(first_seg));
+#endif
+
+ if (0 == (rte_le_to_cpu_64(desc_tmp.wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len +
+ pkt_len - RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ first_seg->port = rxq->port_id;
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox