* Re: [PATCH v3 4/5] net/iavf: accept up to 32k unicast MAC addresses
From: Stephen Hemminger @ 2026-05-12 14:41 UTC (permalink / raw)
To: David Marchand; +Cc: dev, rjarry, cfontain, Vladimir Medvedkin
In-Reply-To: <20260510170306.3406045-5-david.marchand@redhat.com>
On Sun, 10 May 2026 19:03:04 +0200
David Marchand <david.marchand@redhat.com> wrote:
> E810 hardware provides 32k switch lookups.
> Thanks to this, it is possible to allow a lot more secondary mac
> addresses than what is possible today.
>
> In practice, the maximum number of macs available per port may be lower
> and depends on usage by other (trusted?) VFs on the same PF.
> There is no way to figure out this limit but to try adding a mac address
> and get an error from the PF driver.
>
> Mailbox exchanges are limited to IAVF_AQ_BUF_SZ, segment messages
> accordingly.
>
> Signed-off
AI review feedback; nothing major but worth a look
Series: ethdev: VMDq cleanup and iavf: 32k MACs / duplicate install fix
Tree: dpdk main (DPDK 26.07)
Patch 2 - ethdev: skip VMDq pools unless configured
Info
Doxygen of rte_eth_dev_mac_addr_add() is not updated to describe the
new constraint that pool must be 0 when VMDq is not configured. The
function-level @param documentation and @return list should mention
that -EINVAL is now returned when VMDq is disabled and pool != 0.
Subtle behavior change: when VMDq is not configured, mac_pool_sel[index]
is no longer set to RTE_BIT64(0) on add. Before this patch every added
MAC ended up with bit 0 set; afterwards the array stays at 0. This is
consistent with the new restore path (which now uses the VMDq flag
rather than walking pool_mask in the !vmdq case), but the change in
observable state of dev->data->mac_pool_sel[] could surprise an out-of-
tree consumer. Worth a sentence in the API change note.
Patch 4 - net/iavf: accept up to 32k unicast MAC addresses
Info
Release notes wording: "secondary MAC addresses from 64 to 32k" is
slightly imprecise. IAVF_NUM_MACADDR_MAX (64) was the total cap including
the primary MAC; IAVF_UC_MACADDR_MAX (32768) is also a total cap
including index 0. Either drop "secondary" or say "secondary MAC
addresses from 63 to 32767".
iavf_add_del_uc_addr_bulk() returns an int but the only caller
(iavf_add_del_all_mac_addr) discards it. The pre-existing function was
already void so this is no regression, but having the helper return an
error code that is unconditionally dropped is misleading. Either
propagate the error to the caller (and make iavf_add_del_all_mac_addr
return int) or make the helper void.
eth_dev_mac_restore() now iterates dev_info->max_mac_addrs which for
iavf becomes 32768. With patch 5 applied this path is skipped via
get_restore_flags, but any driver that increases max_mac_addrs into the
thousands without setting RTE_ETH_RESTORE_MAC_ADDR off will pay a 32k
rte_is_zero_ether_addr() scan on every dev_start. Not a bug here, but
worth keeping in mind.
sizeof(struct virtchnl_ether_addr_list) already accounts for the
embedded list[1] slot, so
in_args_size = sizeof(struct virtchnl_ether_addr_list) +
sizeof(struct virtchnl_ether_addr) * list->num_elements
overcounts by sizeof(struct virtchnl_ether_addr). This matches what
iavf_add_del_eth_addr() already does, and the buffer is sized to hold
it, so it's not a bug -- noting it for completeness.
Patch 5 - net/iavf: fix duplicate MAC addresses install
Info
Recovery edge case: if iavf_add_del_eth_addr() for the primary fails
during the first iavf_dev_start(), mac_primary_set stays false. On a
subsequent VF reset, iavf_dev_start() (called from iavf_handle_hw_reset)
will re-attempt the primary add, and the new
iavf_add_del_all_mac_addr() call right after will add the primary again
via VIRTCHNL_ETHER_ADDR_PRIMARY. This re-introduces a single-MAC
double-install in a narrow failure path. Could be avoided by skipping
the primary in iavf_add_del_all_mac_addr() when mac_primary_set is
already true, or by having iavf_dev_start() skip the primary add when
in_reset_recovery is set.
No release notes entry. With Fixes: + Cc: stable this is acceptable as
a bug fix, but the patch also introduces a new dev_op
(get_restore_flags) for the driver and changes when MC addresses are
re-pushed (only on VF reset, not on every dev_start). A one-line note
in the iavf section of the release notes would help users tracking
behaviour changes between stop/start cycles.
Patches 1 and 3 have no findings.
^ permalink raw reply
* Re: [PATCH v1 0/4] Introduce generic cache stash API and PCIe TPH implementation
From: Stephen Hemminger @ 2026-05-12 14:38 UTC (permalink / raw)
To: Chengwen Feng; +Cc: thomas, dev, wathsala.vithanage
In-Reply-To: <20260508092855.51987-1-fengchengwen@huawei.com>
On Fri, 8 May 2026 17:28:51 +0800
Chengwen Feng <fengchengwen@huawei.com> wrote:
> This patch set introduces a generic ethdev cache stash framework,
> provides PCIe TPH (Transaction Processing Hints) implementation
> on PCI/VFIO bus, enables TPH cache stash support for e1000/igb PMD,
> and adds testpmd command line configuration.
>
> The patchset is organized as follows:
> - Patch 1/4: bus/pci: introduce PCIe TPH support
> Add generic PCIe TPH APIs and VFIO backend implementation,
> with stub implementations for BSD and Windows.
>
> - Patch 2/4: ethdev: introduce generic cache stash API
> Define unified cache stash capability/operation/config abstraction,
> support device-level and per-queue stash management.
>
> - Patch 3/4: net/e1000: add cache stash support via TPH
> Implement ethdev cache stash ops for igb PMD based on PCIe TPH.
>
> - Patch 4/4: app/testpmd: add TPH stash objects configuration
> Add testpmd CLI option to configure stash objects and
> automatically apply stash enable/disable on forwarding start/stop.
>
> Note:
> 1. This implementation references the Linux kernel VFIO TPH uapi
> series (v8 version), which is not yet merged upstream:
> https://lore.kernel.org/all/20260508064053.37529-1-fengchengwen@huawei.com/
>
> 2. The API design and cache stash model learn from the prior work
> by Wathsala Vithanage: "An API for Cache Stashing with TPH".
>
> 3. The e1000/igb PMD is enabled as the first user of this API
> because it is the only PCIe TPH capable hardware available
> for testing at this time. The generic API design is extensible
> to other NICs and future cache stash mechanisms.
>
> Chengwen Feng (4):
> bus/pci: introduce PCIe TPH support
> ethdev: introduce generic cache stash API
> net/e1000: add cache stash support via TPH
> app/testpmd: add TPH stash objects configuration
>
> app/test-pmd/parameters.c | 20 ++
> app/test-pmd/testpmd.c | 81 +++++++
> app/test-pmd/testpmd.h | 1 +
> drivers/bus/pci/bsd/pci.c | 50 +++++
> drivers/bus/pci/linux/pci.c | 48 +++++
> drivers/bus/pci/linux/pci_init.h | 9 +
> drivers/bus/pci/linux/pci_vfio.c | 272 ++++++++++++++++++++++++
> drivers/bus/pci/rte_bus_pci.h | 112 ++++++++++
> drivers/bus/pci/windows/pci.c | 50 +++++
> drivers/net/intel/e1000/base/e1000_hw.h | 2 +
> drivers/net/intel/e1000/base/e1000_vf.h | 2 +
> drivers/net/intel/e1000/igb_ethdev.c | 221 +++++++++++++++++++
> lib/ethdev/ethdev_driver.h | 37 ++++
> lib/ethdev/rte_ethdev.c | 40 ++++
> lib/ethdev/rte_ethdev.h | 95 +++++++++
> 15 files changed, 1040 insertions(+)
>
The longer AI generated review
Subject: [v2 0/4] PCIe TPH and ethdev cache stash - review
Review of "PCIe TPH support" / "generic cache stash" series v2.
Patch 1 - bus/pci: introduce PCIe TPH support
=============================================
Error: pci_vfio_tph_query() extracts the ST Table Size with the wrong
field mask:
*st_table_sz = RTE_FIELD_GET32(PCI_TPH_LOC_MSIX, cap) + 1;
PCI_TPH_LOC_MSIX is 0x00000400 (bit 10, an ST Table Location bit). Per
PCIe Base Spec, ST Table Size is bits [26:16] of the TPH Requester
Capability Register; Linux defines this as PCI_TPH_CAP_ST_MASK
(0x07FF0000) with shift 16. The current code returns 1 or 2 instead of
the real table size and the result is meaningless. Add the proper
mask/shift, e.g.:
#define PCI_TPH_CAP_ST_MASK 0x07FF0000
#define PCI_TPH_CAP_ST_SHIFT 16
*st_table_sz = ((cap & PCI_TPH_CAP_ST_MASK) >> PCI_TPH_CAP_ST_SHIFT) + 1;
Additionally, per PCIe spec / Linux pci_get_tph_st_num_entries(), the
Size field is only meaningful when the table is located in capability
space. When the ST table is in the MSI-X table, the entry count comes
from the MSI-X table size, not from this field. The branch that returns
the same value for LOC_CAP and LOC_MSIX is incorrect for the MSI-X
case.
Error: pci_vfio_tph_st_get() and pci_vfio_tph_st_set() return raw -1 on
the vfio_dev_fd lookup failure and return the raw ioctl(2) return value
on ioctl error. ioctl() returns -1 with errno set; the function loses
errno and reports -1 to its callers. Return -errno (or a meaningful
-EXXX in the fd case):
if (vfio_dev_fd < 0)
return -ENODEV;
...
if (ioctl(vfio_dev_fd, VFIO_DEVICE_FEATURE, feature) < 0) {
ret = -errno;
free(feature);
return ret;
}
Error: kernel/linux/uapi/linux/vfio.h adds VFIO_DEVICE_FEATURE_TPH_ST
= 13 and the surrounding vfio_device_feature_tph_st structure. The
in-tree uapi is synced from a specific upstream kernel
(kernel/linux/uapi/version says v6.16, where the last
VFIO_DEVICE_FEATURE_* is BUS_MASTER = 10). Picking 13 ahead of an
accepted kernel change is risky - upstream may take 11/12/13 for other
features, creating a permanent ABI clash. DPDK uapi headers should only
mirror accepted kernel UAPI; please rebase once the kernel change is
upstream.
Warning: rte_pci_tph_st_set() doxygen uses "@index" rather than
"@param index", and the description "Array of entries with CPU IDs and
index indices to program" reads as a typo. Also rte_pci_tph_disable()
says "negative value on error" while the rest of the new API says
"negative errno value on error" - please make these consistent.
Patch 2 - ethdev: introduce generic cache stash API
===================================================
Error: RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_cache_stash_get, 26.03)
and ..._set use ABI version 26.03. The tree is currently 26.07.0-rc0
(and patch 1 correctly uses 26.07). 26.03 is a past release; new
symbols added in this cycle must use 26.07.
Warning: rte_eth_cache_stash_set() does not validate op against
RTE_ETH_CACHE_STASH_OP_BUTT. An out-of-range op is passed straight to
the driver, where the switch default just returns -ENOTSUP - validate
in the API:
if (op >= RTE_ETH_CACHE_STASH_OP_BUTT)
return -EINVAL;
Warning: the API uses the legacy dereference style for dev_ops
function pointers:
if (*dev->dev_ops->cache_stash_get == NULL)
return -ENOTSUP;
return eth_err(port_id, (*dev->dev_ops->cache_stash_get)(dev, capa));
Recent ethdev code uses dev->dev_ops->X == NULL / dev->dev_ops->X(...)
directly; please match.
Warning: RTE_ETH_CACHE_STASH_OP_BUTT is not a typical DPDK
sentinel name; existing code uses _MAX, _LAST or _NB. Same for the
enum: rte_eth_cache_stash_type uses RTE_BIT32(0) values (a bitmask)
while the cap field that consumes it is plain uint32_t supported_types
- either keep the type as a flag enum and store flags in a plain
uint32_t (and rename the enum so it does not read as a value type),
or use #define constants like the RTE_ETH_CACHE_STASH_OBJ_* set.
Warning: the queue-level config carries a single queue_id with a
bitmask of Rx and Tx objects in the same call. There is no way for a
caller to say "stash Rx queue 3" versus "stash Tx queue 3" - they share
the queue_id field. Either split into separate enable ops or document
that OBJ_RX_* targets queue.queue_id as an Rx queue and OBJ_TX_* as a
Tx queue (with the caller required to make two calls for both).
Warning: missing release notes entry for the new public API.
Patch 3 - net/e1000: add cache stash support via TPH
====================================================
Error: supported_objects advertises TX_DESC, TX_HEADER and TX_PAYLOAD,
but eth_igb_cache_stash_queue_enable() only reads/writes
E1000_RXCTL(queue_id) and never touches E1000_TXCTL. If a caller
requests Tx stashing the call silently succeeds and does nothing.
Either implement the TXCTL programming or drop the TX_* bits from
supported_objects.
Error: eth_igb_cache_stash_dev_clear() loops over
max(nb_rx_queues, nb_tx_queues) and writes E1000_RXCTL(i):
nb_queues = RTE_MAX(dev->data->nb_rx_queues, dev->data->nb_tx_queues);
for (i = 0; i < nb_queues; i++) {
...
reg = E1000_READ_REG(hw, E1000_RXCTL(i));
...
E1000_WRITE_REG(hw, E1000_RXCTL(i), reg);
}
If nb_tx_queues > nb_rx_queues this writes RXCTL entries for indices
beyond the configured Rx queues. Use nb_rx_queues for the RXCTL loop
(and a separate TXCTL loop with nb_tx_queues when Tx is implemented).
Error: eth_igb_cache_stash_queue_enable() and _queue_disable() do not
bounds-check config->queue.queue_id. E1000_RXCTL(queue_id) is just an
arithmetic register offset; a bogus queue_id writes to an arbitrary MMIO
address. Reject queue_id >= dev->data->nb_rx_queues.
Error: config->queue.lcore_id is passed directly to rte_pci_tph_st_get()
without validation. Use rte_lcore_is_enabled() (or at least
config->queue.lcore_id < RTE_MAX_LCORE) before calling.
Error: drivers/net/intel/e1000/base/e1000_hw.h and base/e1000_vf.h are
modified to add "u8 tph_mode". The base/ directory is a snapshot of
Intel's vendor base (see base/README) and DPDK-private state must not
be added there - it will be lost on the next resync. Put the field in
the DPDK-side adapter struct in drivers/net/intel/e1000/e1000_ethdev.h
instead. Also, the patch touches the VF copy of struct e1000_hw but the
implementation only lives in igb_ethdev.c (PF) - the VF addition is
dead state.
Warning: log uses %u for an int errno value:
PMD_DRV_LOG(ERR, "Failed to get device queue=%u st of cpu=%u ret=%u",
queue_id, config->queue.lcore_id, ret);
ret is int and is negative on failure; print with %d.
Warning: magic bit literals in RXCTL programming - RTE_BIT32(0/2/3),
RTE_SHIFT_VAL32(0xFF, 24) - should be named per the I350 register layout
(e.g. E1000_RXCTL_RX_DESC_STASH_EN, _HDR_STASH_EN, _PLD_STASH_EN,
_CPU_STAG_MASK/SHIFT).
Warning: comment "I350 must encoding st in high 8bit, and ST in
config-space is no needed!" - grammar aside, the intent is not clear;
please reword.
Warning: doc/guides/nics/features/igb.ini is not updated for the new
cache-stash capability.
Warning: missing release notes for the new e1000 capability.
Patch 4 - app/testpmd: add TPH stash objects configuration
==========================================================
Warning: start_tph_stash() returns early on the first port that fails
the capability check or DEV_ENABLE, but ports with index < i have
already had DEV_ENABLE applied. Those ports stay enabled with no queue
configuration and will not be disabled on stop until a full
stop_tph_stash() runs. Either roll back already-enabled ports on error,
or skip just the failing port and continue.
Warning: queue-level enable failures in the inner loop only print to
stderr and forwarding proceeds with a partial configuration. Worth at
least returning failure from start_tph_stash() and propagating so
start_packet_forwarding() can refuse to run.
Warning: no entry added to usage() in app/test-pmd/parameters.c for
--tph-stash-objects. Users have no way to discover the option or its
accepted values (rxdesc, rxheader, rxpayload, rxdata, rxall).
Warning: no documentation update (doc/guides/testpmd_app_ug/) covering
the new option.
Warning: user-visible message "don't support tph stash" -> "doesn't
support tph stash" (also in the second fprintf). Minor.
Warning: "lcoreid_t lc_id;" has a double space; "int i" should be
unsigned to match the portid_t/nb_fwd_ports loop bound type.
General
=======
The kernel-side dependency is the central blocker: until the matching
VFIO_DEVICE_FEATURE_TPH_ST is in an accepted Linux UAPI, the uapi
header import in patch 1 (and by extension all of patch 1's VFIO
ioctl plumbing) is speculative. Please post or reference the upstream
kernel patch so the feature number and structure layout are stable
before this lands in DPDK.
^ permalink raw reply
* Re: [PATCH v1 0/4] Introduce generic cache stash API and PCIe TPH implementation
From: Stephen Hemminger @ 2026-05-12 14:36 UTC (permalink / raw)
To: Chengwen Feng; +Cc: thomas, dev, wathsala.vithanage
In-Reply-To: <20260508092855.51987-1-fengchengwen@huawei.com>
On Fri, 8 May 2026 17:28:51 +0800
Chengwen Feng <fengchengwen@huawei.com> wrote:
> Note:
> 1. This implementation references the Linux kernel VFIO TPH uapi
> series (v8 version), which is not yet merged upstream:
> https://lore.kernel.org/all/20260508064053.37529-1-fengchengwen@huawei.com/
DPDK changes will not be accepted until it is merged upstream.
No speculative patches.
Also, the first patch should be a single patch that updates the sanitized
kernel headers in DPDK.
^ permalink raw reply
* DPDK 23.11.7 released
From: Shani Peretz @ 2026-05-12 13:02 UTC (permalink / raw)
To: announce
Hi all,
Here is a new stable release:
https://fast.dpdk.org/rel/dpdk-23.11.7.tar.xz
The git tree is at:
https://dpdk.org/browse/dpdk-stable/?h=23.11
Shani
---
.mailmap | 14 +-
VERSION | 2 +-
app/graph/conn.c | 134 +++---
app/pdump/main.c | 12 +-
app/proc-info/main.c | 2 +-
app/test-dma-perf/benchmark.c | 6 +-
app/test-dma-perf/main.h | 2 +-
app/test-pmd/cmdline_flow.c | 42 +-
app/test-pmd/config.c | 212 ++++++++-
app/test-pmd/parameters.c | 36 +-
app/test-pmd/testpmd.c | 10 +-
app/test-pmd/testpmd.h | 4 +
app/test/meson.build | 6 +-
app/test/suites/meson.build | 19 +-
app/test/test.h | 19 +
app/test/test_atomic.c | 66 +--
app/test/test_bpf.c | 644 ++++++++++++++++++++++++++
app/test/test_cryptodev.c | 46 +-
app/test/test_debug.c | 9 +-
app/test/test_event_dma_adapter.c | 2 +-
app/test/test_mbuf.c | 18 +-
app/test/test_mcslock.c | 9 +-
app/test/test_pcapng.c | 65 ++-
app/test/test_security_inline_proto.c | 4 +-
app/test/test_stack.c | 11 +-
app/test/test_table_acl.c | 30 +-
app/test/test_telemetry_data.c | 21 +
app/test/test_threads.c | 17 +-
app/test/test_timer_secondary.c | 48 +-
buildtools/get-test-suites.py | 3 +-
config/arm/arm32_armv8_linux_gcc | 3 +
doc/guides/nics/features/i40e.ini | 2 +-
doc/guides/nics/features/iavf.ini | 4 +-
doc/guides/nics/features/ice.ini | 6 +-
doc/guides/nics/i40e.rst | 4 +-
doc/guides/nics/igc.rst | 2 +-
doc/guides/rel_notes/release_23_11.rst | 282 +++++++++++
drivers/bus/fslmc/fslmc_bus.c | 2 +-
drivers/bus/ifpga/ifpga_bus.c | 4 +-
drivers/bus/pci/linux/pci.c | 1 -
drivers/bus/pci/linux/pci_vfio.c | 2 +-
drivers/bus/pci/windows/pci.c | 1 -
drivers/common/cnxk/roc_nix_inl.c | 2 +-
drivers/common/cnxk/roc_nix_inl_dev.c | 10 +-
drivers/common/idpf/idpf_common_rxtx.h | 2 +-
drivers/common/idpf/idpf_common_rxtx_avx512.c | 10 +-
drivers/common/mlx5/linux/mlx5_common_os.c | 85 +++-
drivers/common/mlx5/linux/mlx5_common_os.h | 9 +
drivers/common/mlx5/linux/mlx5_nl.c | 6 +-
drivers/common/mlx5/mlx5_devx_cmds.c | 18 +-
drivers/common/mlx5/mlx5_prm.h | 2 +-
drivers/common/mlx5/version.map | 1 +
drivers/common/mlx5/windows/mlx5_glue.h | 2 +-
drivers/common/sfc_efx/base/ef10_mac.c | 35 +-
drivers/crypto/caam_jr/caam_jr_uio.c | 12 +-
drivers/crypto/cnxk/cnxk_ae.h | 7 +-
drivers/crypto/openssl/rte_openssl_pmd_ops.c | 6 +
drivers/event/cnxk/cn10k_eventdev.c | 15 +-
drivers/event/cnxk/cn10k_worker.h | 6 +-
drivers/net/af_packet/rte_eth_af_packet.c | 2 +-
drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +-
drivers/net/axgbe/axgbe_common.h | 2 +-
drivers/net/axgbe/axgbe_dev.c | 3 +
drivers/net/axgbe/axgbe_ethdev.h | 4 +-
drivers/net/axgbe/axgbe_mdio.c | 4 +-
drivers/net/axgbe/axgbe_phy.h | 4 +-
drivers/net/axgbe/axgbe_phy_impl.c | 15 +
drivers/net/bnxt/tf_ulp/ulp_mapper.c | 5 +
drivers/net/bonding/rte_eth_bond_pmd.c | 11 +-
drivers/net/cnxk/cn10k_rx.h | 6 +-
drivers/net/cpfl/cpfl_flow_engine_fxp.c | 2 -
drivers/net/cpfl/cpfl_flow_parser.c | 6 +-
drivers/net/cpfl/cpfl_fxp_rule.h | 2 -
drivers/net/cpfl/cpfl_representor.h | 2 +-
drivers/net/cpfl/cpfl_rxtx.c | 1 +
drivers/net/dpaa2/dpaa2_ethdev.c | 106 ++---
drivers/net/dpaa2/dpaa2_rxtx.c | 36 +-
drivers/net/dpaa2/mc/fsl_dpni.h | 6 +
drivers/net/e1000/igb_rxtx.c | 3 +-
drivers/net/enetfec/enet_uio.c | 8 +-
drivers/net/hns3/hns3_rxtx.c | 3 +-
drivers/net/i40e/i40e_ethdev.c | 24 +-
drivers/net/i40e/i40e_ethdev.h | 10 +-
drivers/net/i40e/i40e_flow.c | 81 ++--
drivers/net/i40e/i40e_rxtx_vec_avx2.c | 4 +-
drivers/net/i40e/i40e_rxtx_vec_avx512.c | 2 +-
drivers/net/iavf/iavf.h | 1 +
drivers/net/iavf/iavf_ethdev.c | 40 +-
drivers/net/iavf/iavf_fsub.c | 4 +-
drivers/net/iavf/iavf_generic_flow.c | 37 +-
drivers/net/iavf/iavf_generic_flow.h | 1 +
drivers/net/iavf/iavf_ipsec_crypto.c | 2 +-
drivers/net/iavf/iavf_rxtx_vec_avx2.c | 4 +-
drivers/net/iavf/iavf_rxtx_vec_avx512.c | 6 +-
drivers/net/ice/base/ice_acl_ctrl.c | 6 +-
drivers/net/ice/base/ice_bitops.h | 14 +-
drivers/net/ice/base/ice_dcb.c | 8 +-
drivers/net/ice/base/ice_sched.c | 3 +-
drivers/net/ice/base/ice_switch.c | 2 +-
drivers/net/ice/ice_dcf_sched.c | 15 +-
drivers/net/ice/ice_ethdev.c | 21 +-
drivers/net/ice/ice_fdir_filter.c | 8 +-
drivers/net/ice/ice_rxtx_vec_avx2.c | 4 +-
drivers/net/ice/ice_rxtx_vec_avx512.c | 2 +-
drivers/net/idpf/idpf_rxtx.c | 1 +
drivers/net/igc/igc_ethdev.c | 19 +-
drivers/net/igc/igc_txrx.c | 229 +++++++--
drivers/net/igc/igc_txrx.h | 9 +
drivers/net/ixgbe/ixgbe_ethdev.h | 2 +
drivers/net/ixgbe/ixgbe_fdir.c | 7 +-
drivers/net/ixgbe/ixgbe_flow.c | 50 +-
drivers/net/ixgbe/ixgbe_ipsec.c | 18 +-
drivers/net/ixgbe/ixgbe_ipsec.h | 16 +-
drivers/net/ixgbe/ixgbe_rxtx.c | 3 +-
drivers/net/mana/mana.c | 14 +
drivers/net/mana/mp.c | 8 +
drivers/net/mana/tx.c | 6 +-
drivers/net/memif/rte_eth_memif.c | 30 +-
drivers/net/memif/rte_eth_memif.h | 1 +
drivers/net/mlx4/mlx4_mp.c | 6 +
drivers/net/mlx5/hws/mlx5dr_cmd.c | 4 +-
drivers/net/mlx5/hws/mlx5dr_internal.h | 6 +
drivers/net/mlx5/hws/mlx5dr_rule.c | 6 +-
drivers/net/mlx5/hws/mlx5dr_send.c | 4 +-
drivers/net/mlx5/linux/mlx5_ethdev_os.c | 21 +
drivers/net/mlx5/linux/mlx5_mp_os.c | 6 +
drivers/net/mlx5/linux/mlx5_os.c | 360 +++++++++-----
drivers/net/mlx5/mlx5.c | 35 +-
drivers/net/mlx5/mlx5.h | 6 +-
drivers/net/mlx5/mlx5_ethdev.c | 50 +-
drivers/net/mlx5/mlx5_flow.c | 7 +
drivers/net/mlx5/mlx5_flow.h | 2 +
drivers/net/mlx5/mlx5_flow_aso.c | 44 +-
drivers/net/mlx5/mlx5_flow_dv.c | 6 +
drivers/net/mlx5/mlx5_flow_flex.c | 4 +-
drivers/net/mlx5/mlx5_flow_hw.c | 82 ++--
drivers/net/mlx5/mlx5_rx.c | 7 +-
drivers/net/mlx5/mlx5_rx.h | 36 +-
drivers/net/mlx5/mlx5_rxq.c | 85 +++-
drivers/net/mlx5/mlx5_rxtx_vec.c | 2 +
drivers/net/mlx5/mlx5_trigger.c | 16 +-
drivers/net/mlx5/mlx5_txpp.c | 3 -
drivers/net/mlx5/windows/mlx5_os.c | 38 +-
drivers/net/mvpp2/mrvl_ethdev.c | 1 -
drivers/net/netvsc/hn_ethdev.c | 372 ++++++++++++++-
drivers/net/netvsc/hn_nvs.h | 6 +
drivers/net/netvsc/hn_rxtx.c | 40 +-
drivers/net/netvsc/hn_var.h | 1 +
drivers/net/netvsc/hn_vf.c | 213 ++++++---
drivers/net/nfb/nfb.h | 13 +
drivers/net/nfb/nfb_ethdev.c | 166 ++++---
drivers/net/nfb/nfb_rx.c | 2 +-
drivers/net/nfb/nfb_rxmode.c | 12 +-
drivers/net/nfb/nfb_stats.c | 57 ++-
drivers/net/nfb/nfb_tx.c | 2 +-
drivers/net/sfc/sfc.h | 2 +-
drivers/net/sfc/sfc_ethdev.c | 128 ++---
drivers/net/sfc/sfc_ev.c | 7 +-
drivers/net/sfc/sfc_port.c | 8 +-
drivers/net/sfc/sfc_repr.c | 2 +-
drivers/net/tap/rte_eth_tap.c | 51 +-
drivers/net/tap/rte_eth_tap.h | 4 +-
drivers/net/tap/tap_flow.c | 23 +-
examples/ethtool/ethtool-app/main.c | 8 +-
examples/fips_validation/main.c | 17 +-
examples/ipsec-secgw/parser.c | 4 +-
examples/ipsec-secgw/sa.c | 2 +-
examples/ipsec-secgw/sp4.c | 2 +-
examples/ipsec-secgw/sp6.c | 2 +-
examples/l2fwd-jobstats/main.c | 4 +-
examples/packet_ordering/main.c | 2 +-
examples/ptpclient/ptpclient.c | 2 +-
examples/vm_power_manager/channel_manager.c | 12 +-
examples/vm_power_manager/main.c | 2 +-
examples/vmdq_dcb/main.c | 31 +-
lib/bbdev/rte_bbdev.c | 2 +-
lib/bpf/bpf_jit_arm64.c | 162 +++----
lib/bpf/bpf_jit_x86.c | 15 +
lib/bpf/bpf_load.c | 2 +-
lib/bpf/bpf_validate.c | 48 +-
lib/cryptodev/rte_cryptodev.c | 37 +-
lib/eal/common/eal_common_options.c | 22 +-
lib/eal/common/eal_common_trace.c | 90 ++--
lib/eal/common/malloc_heap.c | 2 +-
lib/eal/freebsd/eal_interrupts.c | 6 +
lib/eal/include/rte_bitops.h | 66 +++
lib/eal/include/rte_common.h | 6 +-
lib/eal/include/rte_interrupts.h | 21 +
lib/eal/linux/eal_interrupts.c | 102 ++--
lib/eal/linux/eal_memalloc.c | 26 +-
lib/eal/version.map | 1 +
lib/eal/windows/eal_interrupts.c | 6 +
lib/eal/x86/rte_cycles.c | 2 +-
lib/ethdev/ethdev_driver.c | 6 +-
lib/ethdev/rte_ethdev.c | 1 -
lib/eventdev/rte_event_eth_rx_adapter.c | 17 +-
lib/eventdev/rte_event_eth_tx_adapter.c | 8 +-
lib/fib/dir24_8.c | 3 +-
lib/fib/trie.c | 4 +-
lib/hash/rte_cuckoo_hash.c | 46 +-
lib/hash/rte_thash_x86_gfni.h | 4 +-
lib/mbuf/rte_mbuf.c | 2 +-
lib/mbuf/rte_mbuf.h | 2 +-
lib/net/net_crc_avx512.c | 4 +-
lib/pcapng/rte_pcapng.c | 83 ++--
lib/pcapng/rte_pcapng.h | 10 +-
lib/pdcp/pdcp_process.c | 2 +
lib/pipeline/rte_swx_ctl.c | 11 +-
lib/pipeline/rte_swx_pipeline.c | 21 +-
lib/pipeline/rte_swx_pipeline_internal.h | 10 +-
lib/power/rte_power_pmd_mgmt.c | 6 +-
lib/rcu/rte_rcu_qsbr.h | 2 +-
lib/sched/rte_red.h | 9 +-
lib/table/rte_lru.h | 38 +-
lib/table/rte_lru_arm64.h | 14 +-
lib/table/rte_lru_x86.h | 4 +-
lib/table/rte_swx_table_em.c | 2 -
lib/table/rte_swx_table_learner.c | 4 +-
lib/table/rte_table_hash_key16.c | 4 +-
lib/table/rte_table_hash_key32.c | 4 +-
lib/table/rte_table_hash_key8.c | 4 +-
lib/telemetry/telemetry_data.c | 6 +-
lib/vhost/vduse.c | 5 +-
lib/vhost/vhost.h | 5 +-
lib/vhost/virtio_net_ctrl.c | 22 +-
usertools/dpdk-pmdinfo.py | 5 +-
226 files changed, 4412 insertions(+), 1605 deletions(-)
Aarnav JP (2):
examples/packet_ordering: fix format specifier for port ID
examples/ptpclient: fix format specifier for port ID
Anatoly Burakov (12):
net/ixgbe: fix memory leak in security flows
net/ixgbe: fix potential null dereference with IPsec config
net/ixgbe: fix potential null dereference in IPsec flow
net/ixgbe: fix pointer handling in IPsec
net/i40e: move filter config to flow create
net/i40e: fix IPv6 GTPU handling
net/iavf: fix memory leak on egress IPsec flows
net/iavf: fix memory leak on uninit
net/iavf: fix IPv4 flow subscription
net/ice: fix memory leak in DCF QoS bandwidth config
net/ice: fix memory leak in FDIR flow parsing
net/iavf: fix struct size in IPsec status get
Anurag Mandal (3):
app/testpmd: fix function names in logs
net/i40e: fix QinQ stripping
net/iavf: fix deletion of primary MAC address
Ashok Kumar Natarajan (5):
net/axgbe: fix MAC TCR speed select field width
net/axgbe: add 100 Mbps MAC speed select
net/axgbe: fix auto-negotiation capabilities
net/axgbe: fix SGMII auto-negotiation status bits
net/axgbe: fix 100M SGMII mode
Bruce Richardson (31):
net/iavf: revert check for PF Rx timestamp support
rcu: fix build with MSVC
test/red: fix some undefined behaviour
test/timer: fix hang on secondary process failure
buildtools/test: suppress empty output
eal: fix variable shadowing
bbdev: fix variable shadowing
ethdev: fix variable shadowing
eventdev: fix variable shadowing
net: fix variable shadowing
pcapng: fix variable shadowing
pipeline: fix variable shadowing
power: fix variable shadowing
table: fix variable shadowing
bus/pci: fix variable shadowing
net/cpfl: fix variable shadowing
net/e1000: fix variable shadowing
net/i40e: fix variable shadowing
net/ice: fix variable shadowing
net/ixgbe: fix variable shadowing
app/graph: fix variable shadowing
app/pdump: fix variable shadowing
app/testpmd: fix variable shadowing
config/arm: fix 32-bit build
hash: fix maybe-uninitialized warnings on build
net/intel: fix memory leak on Tx queue setup failure
test: fix dependencies on net null driver
hash: fix overflow of 32-bit offsets
test/security: skip inline protocol test if no HW support
net/bonding: clamp Rx free threshold for small rings
usertools/pmdinfo: fix search for PMD info string
Chengwen Feng (2):
net/hns3: fix outer UDP checksum with simple BD
eal: introduce more macros for bit definition
Ciara Loftus (7):
doc: fix TSO and checksum offload feature status for ice
doc: fix TSO feature status for iavf
doc: fix inline crypto feature status for iavf
doc: fix TSO feature status for i40e
net/i40e: validate raw flow items before dereferencing
doc: remove references to obsolete testpmd flag
net/i40e: fix raw flow item validation
Congjie Zhou (1):
eal/linux: fix fbarray name collision in containers
Dariusz Sosnowski (10):
app/testpmd: fix flow queue job leaks
net/mlx5: fix flow mark reading after reconfigure
net/mlx5: fix shared Rx queue limitations
common/mlx5: fix bonding check
net/mlx5: fix bonding check
net/mlx5: fix probing to allow BlueField Socket Direct
net/mlx5: fix VLAN strip info for CQE compression
net/mlx5: fix flex item capability check
common/mlx5: fix error logging for queue modify
net/mlx5: report share group and queue ID
David Marchand (9):
bus/ifpga: fix const pointer in device name parsing
crypto/caam_jr: fix const pointer in UIO filename parsing
net/enetfec: fix const pointer in UIO filename parsing
net/memif: fix const pointer in socket check
app/procinfo: fix const pointer in collectd format
net/tap: remove log when running without multiprocess
net/af_xdp: fix external mbuf transmit
common/mlx5: fix MAC deletion on Linux
examples/vm_power_manager: fix format specifier for port ID
Emma Finn (1):
examples/fips_validation: fix dangling pointers
Garvit Varshney (1):
crypto/cnxk: return decrypted data for RSA verify
Gregory Etelson (3):
net: fix packet type for stacked VLAN
net/mlx5: fix MPESW PF probe for any number of ports
net/mlx5: fix IPv6 SRH flex node header length
Itai Sharoni (1):
net/mlx5/windows: fix MAC address ownership tracking
Ivan Malov (4):
common/sfc_efx/base: fix flow control on legacy MCDI
net/sfc: rework capability check that is done on FEC set
net/sfc: drop AUTO from FEC capabilities and fix comment
net/sfc: fix reporting status of autonegotiation
Jacob Keller (1):
net/iavf: negotiate PTP before reporting Rx timestamping
John McNamara (1):
net/i40e: fix unused variable
Kai Ji (1):
crypto/openssl: fix SM2 public key buffer overflow
Kevin Traynor (5):
eal/linux: handle interrupt epoll events
interrupts: add interrupt event info
net/mlx5: check DevX disconnect/error interrupt events
examples/ipsec-secgw: fix build with glibc 2.43
examples/vm_power: check truncation of socket path
Liangxing Wang (1):
app/dma-perf: fix reversed CPU copy
Long Li (14):
net/netvsc: fix race conditions on VF add/remove events
net/netvsc: support multi-process VF device removal
net/mana: fix PD resource leak on device close
net/netvsc: fix devargs memory leak on hotplug
net/mana: fix fast-path ops setup in secondary process
net/mlx5: fix fast-path ops setup in secondary process
net/mlx4: fix fast-path ops setup in secondary process
net/netvsc: fix subchannel leak on device removal
net/netvsc: fix double-free of primary Rx queue on uninit
net/netvsc: fix resource leak on init failure
net/netvsc: fix event callback leak on Rx filter failure
net/netvsc: fix resource leaks on MTU change
net/mana: fix CQE suppression handling on error completions
net/netvsc: switch data path to synthetic on device stop
Luca Boccassi (1):
test/debug: fix IOVA mode on PPC64 without huge pages
Maayan Kashani (1):
net/mlx5: fix redundant control rules in promiscuous mode
Maciej Paczkowski (1):
net/ice/base: fix integer types in comparisons
Marat Khalili (6):
bpf: fix x86 call stack alignment for external calls
bpf: disallow empty program
bpf: fix add/subtract overflow
bpf: fix starting with conditional jump
bitops: allow variable as first argument of shift macros
bpf: fix signed shift overflows in ARM JIT
Martin Spinler (5):
net/nfb: use constant values for max Rx/Tx queues count
net/nfb: fix bad pointer access in queue stats
net/nfb: use process private variable for internal
net/nfb: fix resources release
net/nfb: stop only started queues in fail path
Maxime Coquelin (3):
vhost: fix virtqueue array size for control queue
vhost: fix descriptor chain bounds check in control queue
vhost: fix mmap error check in VDUSE IOTLB miss handler
Maxime Leroy (8):
net/dpaa2: warn on Rx descriptor limit in high perf mode
net/dpaa2: fix resource leak on soft parser failure
net/dpaa2: fix link after port stop/start
net/dpaa2: fix spurious VLAN insertion on non-VLAN packet
net/dpaa2: fix L4 packet type in slow parse path
net/dpaa2: fix L3/L4 checksum offload flags
net/dpaa2: fix burst mode info
net/dpaa2: add SG table walk upper bound in Rx
Morten Brørup (2):
eal: fix cache guard for pedantic compilation
mbuf: fix packet data read
Nithinsen Kaithakadan (1):
test/crypto: fix mbuf segment number
Pavan Nikhilesh (1):
event/cnxk: fix Rx offload flags
Piotr Krzewinski (1):
cryptodev: fix memory corruption in secondary process
Radu Nicolau (1):
pdcp: add digest physical address
Rahul Bhansali (1):
event/cnxk: fix crash on CN10K
Rakesh Kudurumalla (2):
common/cnxk: fix engine capabilities fetch logic
examples/l2fwd-jobstats: fix stats availability
Robin Jarry (3):
net/iavf: fix reported max Tx and Rx queues
telemetry: fix adding dict in container array
hash: avoid leaking entries on RCU defer queue failure
Rongwei Liu (1):
net/mlx5: fix job leak on indirect meter creation failure
Sergei Iashin (1):
eventdev/eth_rx: fix crash with telemetry
Shaiq Wani (1):
net/idpf: fix typo in CQ scan threshold macro name
Shani Peretz (10):
examples/ethtool: fix size of mempool name
net/mlx5/hws: fix stack alignment for ASan compatibility
net/mlx5/hws: fix null dereference in rule skip
net/mlx5: allow MTU mismatch for running shared queue
app/testpmd: fix memory leak in port flow configure
net/mlx5: fix meter ASO action leak on release to pool
version: 23.11.7-rc1
app/test: fix uninitialized event in DMA adapter test
Revert "net: fix packet type for stacked VLAN"
version: 23.11.7
Song Yoong Siang (3):
net/e1000: use device timestamp for clock read in igc
net/e1000: fix allocation of context desc for launch time
net/e1000: fix igc launch time calculation
Sriram Yagnaraman (2):
net/memif: fix descriptor Tx flags corruption
net/memif: fix multi-segment Rx corruption
Stephen Hemminger (26):
bus/fslmc: fix const pointer in device name parsing
common/cnxk: fix array out-of-bounds
examples/fips_validation: fix RSA memcpy
net/bnxt: fix build with GCC 16
test/table: avoid input line overflow
test/crypto: check for vdev args overflow
pcapng: document return values
pcapng: use malloc instead of fixed buffer size
pcapng: chain additional mbuf when comment exceeds tailroom
test/pcapng: skip test if null driver missing
net/mlx5: fix use-after-free in ASO management init
net/tap: use correct length for interface names
net/tap: fix resource leaks in secondary process probe
net/tap: free IPC reply buffer on queue count mismatch
net/tap: fix use-after-free on remote flow creation failure
net/tap: free remote flow when implicit rule already exists
test/pcapng: fix for Windows
test: add pause to synchronization spinloops
test/atomic: scale test based on core count
test/mcslock: scale test based on core count
test/stack: scale test based on core count
test/timer: scale test based on core count
test/timer: replace volatile with C11 atomics
test: add file-prefix for all fast-tests on Linux
test/trace: fix parallel execution with traces enabled
examples/vmdq_dcb: initialize all configuration structures
Talluri Chaitanyababu (1):
app/testpmd: fix DCB forwarding TC mismatch handling
Thierry Herbelot (1):
net/intel: update key length when getting RSS key
Thomas Monjalon (4):
common/mlx5: fix variable shadowing
net/mvpp2: fix variable shadowing
eal/x86: fix TSC frequency query
examples/ethtool: fix error message about ports limit
Venkatesh Vemula (1):
net/intel: fix comma operator warnings
Viacheslav Ovsiienko (2):
net/mlx5: fix flex parser creation length attribute
net/mlx5: fix send skew settings when using wait on time
Vipin Varghese (1):
app/dma-perf: fix buffer overflow with high core count
Vladimir Medvedkin (1):
fib: fix prefix addition handling
Xavier Guillaume (1):
net/af_packet: fix MTU set data size calculation
Xiaoyu Min (1):
net/mlx5: fix HW flow counter query
Yang Xu (1):
net/mlx5: fix port down in link detection failure
Yunjian Wang (1):
net/mlx5: fix memory leak after device spawn failure
^ permalink raw reply
* [PATCH v13 09/10] drivers: add data path for Rx and Tx
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Implement receive and transmit burst functions for sxe2 PMD.
Add sxe2_recv_pkts and sxe2_xmit_pkts as the primary data path
interfaces.
The implementation includes:
- Efficient descriptor fetching and mbuf allocation for Rx.
- Descriptor setup and checksum offload handling for Tx.
- Buffer recycling and hardware tail pointer updates.
- Performance-oriented loop unrolling and prefetching where applicable.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_common.c | 1 +
drivers/common/sxe2/sxe2_common_log.h | 1 -
drivers/common/sxe2/sxe2_errno.h | 3 -
drivers/common/sxe2/sxe2_ioctl_chnl.c | 8 +-
drivers/common/sxe2/sxe2_osal.h | 2 -
drivers/net/sxe2/meson.build | 2 +
drivers/net/sxe2/sxe2_ethdev.c | 22 +-
drivers/net/sxe2/sxe2_txrx.c | 247 +++++++
drivers/net/sxe2/sxe2_txrx.h | 21 +
drivers/net/sxe2/sxe2_txrx_poll.c | 945 ++++++++++++++++++++++++++
10 files changed, 1237 insertions(+), 15 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx.c
create mode 100644 drivers/net/sxe2/sxe2_txrx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.c
diff --git a/drivers/common/sxe2/sxe2_common.c b/drivers/common/sxe2/sxe2_common.c
index 63873afe4a..bfb54eec49 100644
--- a/drivers/common/sxe2/sxe2_common.c
+++ b/drivers/common/sxe2/sxe2_common.c
@@ -664,6 +664,7 @@ sxe2_common_init(void)
goto l_end;
pthread_mutex_init(&sxe2_common_devices_list_lock, NULL);
+
sxe2_common_pci_init();
sxe2_commoin_inited = true;
diff --git a/drivers/common/sxe2/sxe2_common_log.h b/drivers/common/sxe2/sxe2_common_log.h
index a7d2157610..cbb53263b5 100644
--- a/drivers/common/sxe2/sxe2_common_log.h
+++ b/drivers/common/sxe2/sxe2_common_log.h
@@ -81,4 +81,3 @@ extern s32 sxe2_log_hw;
#define PMD_INIT_FUNC_TRACE() PMD_LOG_DEBUG(INIT, " >>")
#endif /* __SXE2_COMMON_LOG_H__ */
-
diff --git a/drivers/common/sxe2/sxe2_errno.h b/drivers/common/sxe2/sxe2_errno.h
index 89a715eaef..1257319edf 100644
--- a/drivers/common/sxe2/sxe2_errno.h
+++ b/drivers/common/sxe2/sxe2_errno.h
@@ -50,9 +50,6 @@ enum sxe2_status {
SXE2_ERR_NOLCK = -ENOLCK,
SXE2_ERR_NOSYS = -ENOSYS,
SXE2_ERR_NOTEMPTY = -ENOTEMPTY,
- SXE2_ERR_ILSEQ = -EILSEQ,
- SXE2_ERR_NODATA = -ENODATA,
- SXE2_ERR_CANCELED = -ECANCELED,
SXE2_ERR_TIMEDOUT = -ETIMEDOUT,
SXE2_ERROR = -150,
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
index 4dfc4fd0fa..b9224cf197 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl.c
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -178,13 +178,13 @@ void
goto l_err;
}
- PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=0x%zx, src=0x%"PRIx64", offset=0x%"PRIx64"",
+ PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=%"PRIu64", src=0x%"PRIx64", offset=0x%"PRIx64"",
bar_idx, cmd_fd, len, offset, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
virt = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_SHARED, cmd_fd, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
if (virt == MAP_FAILED) {
- PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=0x%zx, offset=0x%"PRIx64", err:%s",
+ PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=%"PRIu64", offset=0x%"PRIx64", err:%s",
cmd_fd, len, offset, strerror(errno));
goto l_err;
}
@@ -206,12 +206,12 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
goto l_end;
}
- PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%zx",
+ PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%"PRIx64"",
virt, len);
ret = munmap(virt, len);
if (ret < 0) {
- PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=0x%zx, err:%s",
+ PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=%"PRIu64", err:%s",
virt, len, strerror(errno));
ret = -EIO;
goto l_end;
diff --git a/drivers/common/sxe2/sxe2_osal.h b/drivers/common/sxe2/sxe2_osal.h
index d77057e7ee..20d1accd5f 100644
--- a/drivers/common/sxe2/sxe2_osal.h
+++ b/drivers/common/sxe2/sxe2_osal.h
@@ -29,8 +29,6 @@
#define BIT_ULL(a) (1ULL << (a))
#endif
-#define MIN(a, b) ((a) < (b) ? (a) : (b))
-
#define BITS_PER_BYTE 8
#define IS_UNICAST_ETHER_ADDR(addr) \
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 8638244d80..b348dd71a1 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -18,6 +18,8 @@ sources += files(
'sxe2_queue.c',
'sxe2_tx.c',
'sxe2_rx.c',
+ 'sxe2_txrx_poll.c',
+ 'sxe2_txrx.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 2a07c211bf..7e9a842eb9 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -26,6 +26,7 @@
#include "sxe2_cmd_chnl.h"
#include "sxe2_tx.h"
#include "sxe2_rx.h"
+#include "sxe2_txrx.h"
#include "sxe2_common.h"
#include "sxe2_common_log.h"
#include "sxe2_host_regs.h"
@@ -131,6 +132,9 @@ static s32 sxe2_dev_start(struct rte_eth_dev *dev)
goto l_end;
}
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
+
ret = sxe2_queues_start(dev);
if (ret) {
PMD_LOG_ERR(INIT, "enable queues failed");
@@ -348,8 +352,8 @@ void __iomem *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
for (i = 0; i < bar_info->map_cnt; i++) {
seg_info = &bar_info->seg_info[i];
if (res_type == seg_info->type) {
- addr = (void __iomem *)((uintptr_t)seg_info->addr +
- seg_info->page_inner_offset + reg_width * idx_in_func);
+ addr = (uint8_t __iomem *)seg_info->addr +
+ seg_info->page_inner_offset + reg_width * idx_in_func;
goto l_end;
}
}
@@ -460,8 +464,9 @@ s32 sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
map_addr = sxe2_drv_dev_mmap(adapter->cdev, bar_info->bar_idx, aligned_len, aligned_offset);
if (!map_addr) {
- PMD_LOG_ERR(INIT, "Failed to mmap BAR space, type=%d, len=%zu, page_size=%zu",
- res_type, org_len, page_size);
+ PMD_LOG_ERR(INIT, "Failed to mmap BAR space, type=%d, len=%" PRIu64
+ ", offset=%" PRIu64 ", page_size=%zu",
+ res_type, org_len, org_offset, page_size);
ret = -EFAULT;
goto l_end;
}
@@ -745,10 +750,17 @@ static s32 sxe2_dev_init(struct rte_eth_dev *dev, struct sxe2_dev_kvargs_info *k
PMD_INIT_FUNC_TRACE();
+ sxe2_set_common_function(dev);
+
dev->dev_ops = &sxe2_eth_dev_ops;
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
+ if (ret != SXE2_SUCCESS)
+ PMD_LOG_ERR(INIT, "Failed to mp init (secondary), ret=%d", ret);
goto l_end;
+ }
ret = sxe2_hw_init(dev);
if (ret) {
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
new file mode 100644
index 0000000000..a7b94e8967
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -0,0 +1,247 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include <unistd.h>
+
+#include "sxe2_txrx.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_ethdev.h"
+
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+#include "sxe2_osal.h"
+#include "sxe2_cmd_chnl.h"
+#if defined(RTE_ARCH_ARM64)
+#include <rte_cpuflags.h>
+#endif
+
+static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ s32 ret;
+ u16 desc_idx;
+
+ if (unlikely(offset >= txq->ring_depth)) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+ desc_idx = txq->next_use + offset;
+ desc_idx = DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
+ if (desc_idx >= txq->ring_depth) {
+ desc_idx -= txq->ring_depth;
+ if (desc_idx >= txq->ring_depth)
+ desc_idx -= txq->ring_depth;
+ }
+
+ if (desc_idx == 0)
+ desc_idx = txq->rs_thresh - 1;
+ else
+ desc_idx -= 1;
+
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
+ (txq->desc_ring[desc_idx].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
+ ret = RTE_ETH_TX_DESC_DONE;
+ else
+ ret = RTE_ETH_TX_DESC_FULL;
+
+l_end:
+ return ret;
+}
+
+static inline s32 sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
+{
+ struct rte_mbuf *m_seg = mbuf;
+
+ while (m_seg != NULL) {
+ if (m_seg->data_len == 0)
+ return SXE2_ERR_INVAL;
+ m_seg = m_seg->next;
+ }
+
+ return SXE2_SUCCESS;
+}
+
+u16 sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct rte_mbuf *mbuf;
+ u64 ol_flags = 0;
+ s32 ret = SXE2_SUCCESS;
+ s32 i = 0;
+
+ for (i = 0; i < nb_pkts; i++) {
+ mbuf = tx_pkts[i];
+ if (!mbuf)
+ continue;
+ ol_flags = mbuf->ol_flags;
+ if (!(ol_flags & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))) {
+ if (mbuf->nb_segs > SXE2_TX_MTU_SEG_MAX ||
+ mbuf->pkt_len > SXE2_FRAME_SIZE_MAX) {
+ rte_errno = -SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ } else if ((mbuf->tso_segsz < SXE2_MIN_TSO_MSS) ||
+ (mbuf->tso_segsz > SXE2_MAX_TSO_MSS) ||
+ (mbuf->nb_segs > txq->ring_depth) ||
+ (mbuf->pkt_len > SXE2_TX_TSO_PKTLEN_MAX)) {
+ rte_errno = -SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+ if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
+ rte_errno = -SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_TX
+ ret = rte_validate_tx_offload(mbuf);
+ if (ret != SXE2_SUCCESS) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+#endif
+ ret = rte_net_intel_cksum_prepare(mbuf);
+ if (ret != SXE2_SUCCESS) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+
+ ret = sxe2_tx_mbuf_empty_check(mbuf);
+ if (ret != SXE2_SUCCESS) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return i;
+}
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ u32 tx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
+ PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
+ tx_mode_flags, dev->data->port_id);
+}
+
+static s32 sxe2_rx_desciptor_status(void *rx_queue, u16 offset)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ s32 ret;
+
+ if (unlikely(offset >= rxq->ring_depth)) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+ if (offset >= rxq->ring_depth - rxq->hold_num) {
+ ret = RTE_ETH_RX_DESC_UNAVAIL;
+ goto l_end;
+ }
+
+ if (rxq->processing_idx + offset >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
+ else
+ desc = &rxq->desc_ring[rxq->processing_idx + offset];
+
+ if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
+ ret = RTE_ETH_RX_DESC_DONE;
+ else
+ ret = RTE_ETH_RX_DESC_AVAIL;
+
+l_end:
+ PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
+ offset, ret, rxq->queue_id, rxq->port_id);
+ return ret;
+}
+
+static s32 sxe2_rx_queue_count(void *rx_queue)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ u16 done_num = 0;
+
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ while ((done_num < rxq->ring_depth) &&
+ (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK)) {
+ done_num += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ if (rxq->processing_idx + done_num >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + done_num - rxq->ring_depth];
+ else
+ desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ }
+
+ PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
+ done_num, rxq->queue_id, rxq->port_id);
+
+ return done_num;
+}
+
+static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ u16 i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (0 != (rxq->offloads & offload)) {
+ en = true;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return en;
+}
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ u32 rx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
+ else
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
+
+ PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
+ rx_mode_flags, dev->data->port_id);
+ adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
+}
+
+void sxe2_set_common_function(struct rte_eth_dev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+
+ dev->rx_queue_count = sxe2_rx_queue_count;
+ dev->rx_descriptor_status = sxe2_rx_desciptor_status;
+
+ dev->tx_descriptor_status = sxe2_tx_desciptor_status;
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
new file mode 100644
index 0000000000..e6f671e3dc
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef SXE2_TXRX_H
+#define SXE2_TXRX_H
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+
+void sxe2_set_common_function(struct rte_eth_dev *dev);
+
+u16 sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts);
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.c b/drivers/net/sxe2/sxe2_txrx_poll.c
new file mode 100644
index 0000000000..02533abfd5
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_poll.c
@@ -0,0 +1,945 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include <unistd.h>
+
+#include "sxe2_osal.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_txrx.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+
+static __rte_always_inline s32
+sxe2_tx_bufs_free(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX];
+ s32 ret;
+ u32 i;
+ u16 rs_thresh;
+ u16 free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - rs_thresh + 1];
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {
+ if (likely(rs_thresh <= SXE2_TX_FREE_BUFFER_SIZE_MAX)) {
+ mbuf = buffer[0].mbuf;
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = buffer[i].mbuf;
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ rte_mempool_put(buffer->mbuf->pool, buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline s32 sxe2_tx_cleanup(struct sxe2_tx_queue *txq)
+{
+ s32 ret = SXE2_SUCCESS;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ struct sxe2_tx_buffer *buffer_ring = txq->buffer_ring;
+ u16 ring_depth = txq->ring_depth;
+ u16 next_clean = txq->next_clean;
+ u16 clean_last;
+ u16 clean_num;
+
+ clean_last = next_clean + txq->rs_thresh;
+ if (clean_last >= ring_depth)
+ clean_last = clean_last - ring_depth;
+
+ clean_last = buffer_ring[clean_last].last_id;
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) !=
+ (txq->desc_ring[clean_last].wb.dd & rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK))) {
+ PMD_LOG_DEBUG(TX, "desc[%u] is not done.port_id=%u queue_id=%u val=0x%" PRIx64,
+ clean_last, txq->port_id,
+ txq->queue_id, txq->desc_ring[clean_last].wb.dd);
+ ret = SXE2_ERR_DESC_NO_DONE;
+ goto l_end;
+ }
+
+ if (clean_last > next_clean)
+ clean_num = clean_last - next_clean;
+ else
+ clean_num = ring_depth - next_clean + clean_last;
+
+ desc_ring[clean_last].wb.dd = 0;
+
+ txq->next_clean = clean_last;
+ txq->desc_free_num += clean_num;
+
+ ret = SXE2_SUCCESS;
+
+l_end:
+ return ret;
+}
+
+static __rte_always_inline u16
+sxe2_tx_pkt_data_desc_count(struct rte_mbuf *tx_pkt)
+{
+ struct rte_mbuf *m_seg = tx_pkt;
+ u16 count = 0;
+
+ while (m_seg != NULL) {
+ count += DIV_ROUND_UP(m_seg->data_len,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC);
+ m_seg = m_seg->next;
+ }
+
+ return count;
+}
+
+static __rte_always_inline void
+sxe2_tx_desc_checksum_fill(u64 offloads, u32 *desc_cmd, u32 *desc_offset,
+ union sxe2_tx_offload_info ol_info)
+{
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ }
+
+ if (offloads & RTE_MBUF_F_TX_TCP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_UDP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ default:
+
+ break;
+ }
+
+l_end:
+ return;
+}
+
+static __rte_always_inline u64
+sxe2_tx_data_desc_build_cobt(u32 cmd, u32 offset, u16 buf_size, u16 l2tag)
+{
+ return rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DATA |
+ (((u64)cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT) |
+ (((u64)offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT) |
+ (((u64)buf_size) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT) |
+ (((u64)l2tag) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT));
+}
+
+u16 sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct sxe2_tx_buffer *buffer_ring;
+ struct sxe2_tx_buffer *buffer;
+ struct sxe2_tx_buffer *next_buffer;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ volatile union sxe2_tx_data_desc *desc_ring;
+ volatile union sxe2_tx_data_desc *desc;
+ volatile struct sxe2_tx_context_desc *ctxt_desc;
+ union sxe2_tx_offload_info ol_info;
+ struct sxe2_vsi *vsi = txq->vsi;
+ rte_iova_t buf_dma_addr;
+ u64 offloads;
+ u64 desc_type_cmd_tso_mss;
+ u32 desc_cmd;
+ u32 desc_offset;
+ u32 desc_tag;
+ u32 desc_tunneling_params;
+ u16 ipsec_offset;
+ u16 ctxt_desc_num;
+ u16 desc_sum_num;
+ u16 tx_num;
+ u16 seg_len;
+ u16 next_use;
+ u16 last_use;
+ u16 desc_l2tag2;
+
+ buffer_ring = txq->buffer_ring;
+ desc_ring = txq->desc_ring;
+ next_use = txq->next_use;
+ buffer = &buffer_ring[next_use];
+
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_cleanup(txq);
+
+ for (tx_num = 0; tx_num < nb_pkts; tx_num++) {
+ tx_pkt = *tx_pkts++;
+ desc_cmd = 0;
+ desc_offset = 0;
+ desc_tag = 0;
+ desc_tunneling_params = 0;
+ ipsec_offset = 0;
+ offloads = tx_pkt->ol_flags;
+ ol_info.l2_len = tx_pkt->l2_len;
+ ol_info.l3_len = tx_pkt->l3_len;
+ ol_info.l4_len = tx_pkt->l4_len;
+ ol_info.tso_segsz = tx_pkt->tso_segsz;
+ ol_info.outer_l2_len = tx_pkt->outer_l2_len;
+ ol_info.outer_l3_len = tx_pkt->outer_l3_len;
+
+ ctxt_desc_num = (offloads &
+ SXE2_TX_OFFLOAD_CTXT_NEEDCK_MASK) ? 1 : 0;
+ if (unlikely(vsi->vsi_type == SXE2_VSI_T_DPDK_ESW))
+ ctxt_desc_num = 1;
+
+ if (offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))
+ desc_sum_num = sxe2_tx_pkt_data_desc_count(tx_pkt) + ctxt_desc_num;
+ else
+ desc_sum_num = tx_pkt->nb_segs + ctxt_desc_num;
+
+ last_use = next_use + desc_sum_num - 1;
+ if (last_use >= txq->ring_depth)
+ last_use = last_use - txq->ring_depth;
+
+ if (desc_sum_num > txq->desc_free_num) {
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_exit_logic;
+
+ if (unlikely(desc_sum_num > txq->rs_thresh)) {
+ while (desc_sum_num > txq->desc_free_num)
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_exit_logic;
+ }
+ }
+
+ desc_offset |= SXE2_TX_DATA_DESC_MACLEN_VAL(ol_info.l2_len);
+
+ if (offloads & SXE2_TX_OFFLOAD_CKSUM_MASK) {
+ sxe2_tx_desc_checksum_fill(offloads, &desc_cmd,
+ &desc_offset, ol_info);
+ }
+
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ desc_tag = tx_pkt->vlan_tci;
+ }
+
+ if (ctxt_desc_num) {
+ ctxt_desc = (volatile struct sxe2_tx_context_desc *)
+ &desc_ring[next_use];
+ desc_l2tag2 = 0;
+ desc_type_cmd_tso_mss = SXE2_TX_DESC_DTYPE_CTXT;
+
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_QINQ) {
+ desc_l2tag2 = tx_pkt->vlan_tci_outer;
+ desc_type_cmd_tso_mss |= SXE2_TX_CTXT_DESC_CMD_IL2TAG2_MASK;
+ }
+
+ ctxt_desc->tunneling_params =
+ rte_cpu_to_le_32(desc_tunneling_params);
+ ctxt_desc->l2tag2 = rte_cpu_to_le_16(desc_l2tag2);
+ ctxt_desc->type_cmd_tso_mss = rte_cpu_to_le_64(desc_type_cmd_tso_mss);
+ ctxt_desc->ipsec_offset = rte_cpu_to_le_64(ipsec_offset);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ }
+
+ m_seg = tx_pkt;
+
+ do {
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ buffer->mbuf = m_seg;
+ seg_len = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ while ((offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG)) &&
+ unlikely(seg_len > SXE2_TX_MAX_DATA_NUM_PER_DESC)) {
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd, desc_offset,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC,
+ desc_tag);
+ buf_dma_addr += SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ seg_len -= SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ }
+
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd,
+ desc_offset, seg_len, desc_tag);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_EOP;
+ txq->desc_used_num += desc_sum_num;
+ txq->desc_free_num -= desc_sum_num;
+
+ if (txq->desc_used_num >= txq->rs_thresh) {
+ PMD_LOG_DEBUG(TX, "Tx pkts set RS bit."
+ "last_use=%u port_id=%u, queue_id=%u",
+ last_use, txq->port_id, txq->queue_id);
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_RS;
+ txq->desc_used_num = 0;
+ }
+
+ desc->read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(((u64)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT);
+ }
+
+l_exit_logic:
+ if (tx_num == 0)
+ goto l_end;
+ goto l_end_of_tx;
+l_end_of_tx:
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, tx_num);
+
+ txq->next_use = next_use;
+
+l_end:
+ return tx_num;
+}
+
+static __rte_always_inline void
+sxe2_tx_data_desc_fill(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ u32 desc_offset;
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset, (*tx_pkts)->data_len, 0);
+}
+static __rte_always_inline void
+sxe2_tx_data_desc_fill_batch(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ u32 i;
+ u32 desc_offset;
+ for (i = 0; i < SXE2_TX_FILL_PER_LOOP; ++i, ++desc, ++tx_pkts) {
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset,
+ (*tx_pkts)->data_len,
+ 0);
+ }
+}
+
+static inline void sxe2_tx_ring_fill(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_buffer *buffer = &txq->buffer_ring[txq->next_use];
+ volatile union sxe2_tx_data_desc *desc = &txq->desc_ring[txq->next_use];
+ u32 i, j;
+ u32 mainpart;
+ u32 leftover;
+ mainpart = nb_pkts & ((u32)~SXE2_TX_FILL_PER_LOOP_MASK);
+ leftover = nb_pkts & ((u32)SXE2_TX_FILL_PER_LOOP_MASK);
+ for (i = 0; i < mainpart; i += SXE2_TX_FILL_PER_LOOP) {
+ for (j = 0; j < SXE2_TX_FILL_PER_LOOP; ++j)
+ (buffer + i + j)->mbuf = *(tx_pkts + i + j);
+ sxe2_tx_data_desc_fill_batch(desc + i, tx_pkts + i);
+ }
+ if (unlikely(leftover > 0)) {
+ for (i = 0; i < leftover; ++i) {
+ (buffer + mainpart + i)->mbuf = *(tx_pkts + mainpart + i);
+ sxe2_tx_data_desc_fill(desc + mainpart + i,
+ tx_pkts + mainpart + i);
+ }
+ }
+}
+
+static inline u16 sxe2_tx_pkts_batch(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ u16 res_num = 0;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ txq->desc_free_num -= nb_pkts;
+ if ((txq->next_use + nb_pkts) > txq->ring_depth) {
+ res_num = txq->ring_depth - txq->next_use;
+ sxe2_tx_ring_fill(txq, tx_pkts, res_num);
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs = txq->rs_thresh - 1;
+ txq->next_use = 0;
+ }
+ sxe2_tx_ring_fill(txq, tx_pkts + res_num, nb_pkts - res_num);
+ txq->next_use = txq->next_use + (nb_pkts - res_num);
+ if (txq->next_use > txq->next_rs) {
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ if (txq->next_rs >= txq->ring_depth)
+ txq->next_rs = txq->rs_thresh - 1;
+ }
+ if (txq->next_use >= txq->ring_depth)
+ txq->next_use = 0;
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, txq->next_use, nb_pkts);
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, txq->next_use);
+l_end:
+ return nb_pkts;
+}
+
+u16 sxe2_tx_pkts_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ u16 tx_done_num;
+ u16 tx_once_num;
+ u16 tx_need_num;
+ if (likely(nb_pkts <= SXE2_TX_PKTS_BURST_BATCH_NUM)) {
+ tx_done_num = sxe2_tx_pkts_batch(tx_queue,
+ tx_pkts, nb_pkts);
+ goto l_end;
+ }
+ tx_done_num = 0;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, SXE2_TX_PKTS_BURST_BATCH_NUM);
+ tx_once_num = sxe2_tx_pkts_batch(tx_queue,
+ &tx_pkts[tx_done_num], tx_need_num);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+l_end:
+ return tx_done_num;
+}
+
+static inline void
+sxe2_update_rx_tail(struct sxe2_rx_queue *rxq, u16 hold_num, u16 rx_id)
+{
+ hold_num += rxq->hold_num;
+
+ if (hold_num > rxq->rx_free_thresh) {
+ rx_id = (u16)((rx_id == 0) ? (rxq->ring_depth - 1) : (rx_id - 1));
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, rx_id);
+ hold_num = 0;
+ }
+ rxq->hold_num = hold_num;
+}
+
+static inline u64
+sxe2_rx_desc_error_para(__rte_unused struct sxe2_rx_queue *rxq,
+ union sxe2_rx_desc *desc)
+{
+ u64 flags = 0;
+ u64 desc_qw1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (unlikely(0 == (desc_qw1 & SXE2_RX_DESC_STATUS_L3L4_P_MASK)))
+ goto l_end;
+
+ if (likely(0 == (desc->wb.rxdid_src & SXE2_RX_DESC_EUDPE_MASK)))
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+ else
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_QW1_ERRORS_MASK))) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD);
+ goto l_end;
+ }
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_IPE_MASK)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_L4_MASK)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+
+ if (unlikely(0 != (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_EIP_MASK)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+l_end:
+ return flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_mbuf_common_fields_fill(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ u32 *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ u64 qword1;
+ u64 pkt_flags;
+ qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+
+ mbuf->ol_flags = 0;
+ mbuf->packet_type = ptype_tbl[SXE2_RX_DESC_PTYPE_VAL_GET(qword1)];
+
+ pkt_flags = sxe2_rx_desc_error_para(rxq, rxd);
+
+ mbuf->ol_flags |= pkt_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_sw_stats_update(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ u64 qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ switch (SXE2_RX_DESC_STATUS_UMBCAST_VAL_GET(qword1)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+}
+
+u16 sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ u64 qword1;
+ u16 done_num;
+ u16 hold_num;
+ u16 cur_idx;
+ u16 pkt_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ PMD_LOG_INFO(RX, "Rx new_mbuf alloc failed port_id:%u "
+ "queue_id:%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+
+ rte_prefetch0(buffer_ring[cur_idx]);
+
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+
+ cur_mbuf = *cur_buffer;
+
+ *cur_buffer = new_mbuf;
+
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+
+ if (0 == (qword1 & SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len + pkt_len -
+ RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+ first_seg->port = rxq->port_id;
+
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
+
+u16 sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *cur_mbuf_pay;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *new_mbuf_pay = NULL;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ u64 qword1;
+ u16 done_num;
+ u16 hold_num;
+ u16 cur_idx;
+ u16 pkt_len;
+ u16 hdr_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ new_mbuf = NULL;
+
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) == 0 ||
+ first_seg == NULL) {
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ break;
+ }
+ }
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) {
+ new_mbuf_pay = rte_mbuf_raw_alloc(rxq->rx_seg[1].mp);
+ if (unlikely(new_mbuf_pay == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ if (new_mbuf != NULL)
+ rte_pktmbuf_free(new_mbuf);
+ new_mbuf = NULL;
+ break;
+ }
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+ rte_prefetch0(buffer_ring[cur_idx]);
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+ cur_mbuf = *cur_buffer;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ *cur_buffer = new_mbuf;
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ } else {
+ if (first_seg == NULL) {
+ *cur_buffer = new_mbuf;
+ new_mbuf->next = new_mbuf_pay;
+ new_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ } else {
+ cur_mbuf_pay = cur_mbuf->next;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ cur_mbuf->next = new_mbuf_pay;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(cur_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ cur_mbuf = cur_mbuf_pay;
+ }
+ }
+ new_mbuf = NULL;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ } else {
+ if (first_seg == NULL) {
+ cur_mbuf->nb_segs = 2;
+ cur_mbuf->next->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ hdr_len = SXE2_RX_DESC_HDR_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = hdr_len;
+ cur_mbuf->pkt_len = hdr_len + pkt_len;
+ cur_mbuf->next->data_len = pkt_len;
+ first_seg = cur_mbuf;
+ cur_mbuf = cur_mbuf->next;
+ last_seg = cur_mbuf;
+ } else {
+ cur_mbuf->nb_segs = 1;
+ cur_mbuf->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_RX
+
+ rte_pktmbuf_dump(stdout, first_seg, rte_pktmbuf_pkt_len(first_seg));
+#endif
+
+ if (0 == (rte_le_to_cpu_64(desc_tmp.wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len +
+ pkt_len - RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ first_seg->port = rxq->port_id;
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
* [PATCH v13 05/10] drivers: add base driver probe skeleton
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Initialize the eth_dev_ops for the sxe2 PMD. This includes the
implementation of mandatory ethdev operations such as dev_configure,
dev_start, dev_stop, and dev_infos_get.
Set up the basic infrastructure for device initialization to allow
the driver to be recognized as a valid ethernet device within the
DPDK framework.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_common.h | 2 +-
drivers/common/sxe2/sxe2_ioctl_chnl.c | 27 +
drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 9 +
drivers/net/meson.build | 1 +
drivers/net/sxe2/meson.build | 21 +
drivers/net/sxe2/sxe2_cmd_chnl.c | 319 +++++++++++
drivers/net/sxe2/sxe2_cmd_chnl.h | 33 ++
drivers/net/sxe2/sxe2_drv_cmd.h | 398 ++++++++++++++
drivers/net/sxe2/sxe2_ethdev.c | 611 +++++++++++++++++++++
drivers/net/sxe2/sxe2_ethdev.h | 295 ++++++++++
drivers/net/sxe2/sxe2_irq.h | 49 ++
drivers/net/sxe2/sxe2_queue.c | 39 ++
drivers/net/sxe2/sxe2_queue.h | 191 +++++++
drivers/net/sxe2/sxe2_txrx_common.h | 541 ++++++++++++++++++
drivers/net/sxe2/sxe2_txrx_poll.h | 16 +
drivers/net/sxe2/sxe2_vsi.c | 212 +++++++
drivers/net/sxe2/sxe2_vsi.h | 205 +++++++
17 files changed, 2968 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/sxe2/meson.build
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.c
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.h
create mode 100644 drivers/net/sxe2/sxe2_drv_cmd.h
create mode 100644 drivers/net/sxe2/sxe2_ethdev.c
create mode 100644 drivers/net/sxe2/sxe2_ethdev.h
create mode 100644 drivers/net/sxe2/sxe2_irq.h
create mode 100644 drivers/net/sxe2/sxe2_queue.c
create mode 100644 drivers/net/sxe2/sxe2_queue.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.h
create mode 100644 drivers/net/sxe2/sxe2_vsi.c
create mode 100644 drivers/net/sxe2/sxe2_vsi.h
diff --git a/drivers/common/sxe2/sxe2_common.h b/drivers/common/sxe2/sxe2_common.h
index d02d281a70..090b643548 100644
--- a/drivers/common/sxe2/sxe2_common.h
+++ b/drivers/common/sxe2/sxe2_common.h
@@ -57,7 +57,7 @@ typedef s32 (sxe2_class_driver_remove_t)(struct sxe2_common_device *scdev);
struct sxe2_class_driver {
TAILQ_ENTRY(sxe2_class_driver) next;
enum sxe2_class_type drv_class;
- const s8 *name;
+ const char *name;
sxe2_class_driver_probe_t *probe;
sxe2_class_driver_remove_t *remove;
const struct rte_pci_id *id_table;
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
index 0d300e0f81..4b041765de 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl.c
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -159,3 +159,30 @@ sxe2_drv_dev_handshark(struct sxe2_common_device *cdev)
l_end:
return ret;
}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_munmap)
+s32
+sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
+{
+ s32 ret = SXE2_SUCCESS;
+
+ if (cdev->config.kernel_reset) {
+ ret = SXE2_ERR_PERM;
+ PMD_LOG_WARN(COM, "kernel reset, need restart app.");
+ goto l_end;
+ }
+
+ PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%zx",
+ virt, len);
+
+ ret = munmap(virt, len);
+ if (ret < 0) {
+ PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=0x%zx, err:%s",
+ virt, len, strerror(errno));
+ ret = SXE2_ERR_IO;
+ goto l_end;
+ }
+
+l_end:
+ return ret;
+}
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl_func.h b/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
index 0c3cb9caea..376c5e3ac7 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
@@ -38,6 +38,15 @@ __rte_internal
s32
sxe2_drv_dev_handshark(struct sxe2_common_device *cdev);
+__rte_internal
+void
+*sxe2_drv_dev_mmap(struct sxe2_common_device *cdev, u8 bar_idx,
+ u64 len, u64 offset);
+
+__rte_internal
+s32
+sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len);
+
#ifdef __cplusplus
}
#endif
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index c7dae4ad27..4e8ccb945f 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -58,6 +58,7 @@ drivers = [
'rnp',
'sfc',
'softnic',
+ 'sxe2',
'tap',
'thunderx',
'txgbe',
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
new file mode 100644
index 0000000000..6c9a86423a
--- /dev/null
+++ b/drivers/net/sxe2/meson.build
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+
+if is_windows
+ build = false
+ reason = 'only supported on Linux'
+ subdir_done()
+endif
+
+cflags += ['-g']
+
+deps += ['common_sxe2', 'hash','cryptodev','security']
+
+sources += files(
+ 'sxe2_ethdev.c',
+ 'sxe2_cmd_chnl.c',
+ 'sxe2_vsi.c',
+ 'sxe2_queue.c',
+)
+
+allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_cmd_chnl.c b/drivers/net/sxe2/sxe2_cmd_chnl.c
new file mode 100644
index 0000000000..78e2a30614
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_cmd_chnl.c
@@ -0,0 +1,319 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include "sxe2_ioctl_chnl_func.h"
+#include "sxe2_drv_cmd.h"
+#include "sxe2_cmd_chnl.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+
+static union sxe2_drv_trace_info sxe2_drv_trace_id;
+
+static void sxe2_drv_trace_id_alloc(u64 *trace_id)
+{
+ union sxe2_drv_trace_info *trace = NULL;
+ u64 trace_id_count = 0;
+
+ trace = &sxe2_drv_trace_id;
+
+ trace_id_count = trace->sxe2_drv_trace_id_param.count;
+ ++trace_id_count;
+ trace->sxe2_drv_trace_id_param.count =
+ (trace_id_count & SXE2_DRV_TRACE_ID_COUNT_MASK);
+
+ *trace_id = trace->id;
+}
+
+static void __sxe2_drv_cmd_params_fill(struct sxe2_adapter *adapter,
+ struct sxe2_drv_cmd_params *cmd, u32 opc, const char *opc_str,
+ void *in_data, u32 in_len, void *out_data, u32 out_len)
+{
+ PMD_DEV_LOG_DEBUG(adapter, DRV, "cmd opcode:%s", opc_str);
+ cmd->timeout = SXE2_DRV_CMD_DFLT_TIMEOUT;
+ cmd->opcode = opc;
+ cmd->vsi_id = adapter->vsi_ctxt.dpdk_vsi_id;
+ cmd->repr_id = (adapter->repr_priv_data != NULL) ?
+ adapter->repr_priv_data->repr_id : 0xFFFF;
+ cmd->req_len = in_len;
+ cmd->req_data = in_data;
+ cmd->resp_len = out_len;
+ cmd->resp_data = out_data;
+
+ sxe2_drv_trace_id_alloc(&cmd->trace_id);
+}
+
+#define sxe2_drv_cmd_params_fill(adapter, cmd, opc, in_data, in_len, out_data, out_len) \
+ __sxe2_drv_cmd_params_fill(adapter, cmd, opc, #opc, in_data, in_len, out_data, out_len)
+
+
+s32 sxe2_drv_dev_caps_get(struct sxe2_adapter *adapter, struct sxe2_drv_dev_caps_resp *dev_caps)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_DEV_GET_CAPS,
+ NULL, 0, dev_caps,
+ sizeof(struct sxe2_drv_dev_caps_resp));
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret)
+ PMD_DEV_LOG_ERR(adapter, DRV, "get dev caps failed, ret=%d", ret);
+
+ return ret;
+}
+
+s32 sxe2_drv_dev_info_get(struct sxe2_adapter *adapter,
+ struct sxe2_drv_dev_info_resp *dev_info_resp)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_DEV_GET_INFO,
+ NULL, 0, dev_info_resp,
+ sizeof(struct sxe2_drv_dev_info_resp));
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret)
+ PMD_DEV_LOG_ERR(adapter, DRV, "get dev info failed, ret=%d", ret);
+
+ return ret;
+}
+
+s32 sxe2_drv_dev_fw_info_get(struct sxe2_adapter *adapter,
+ struct sxe2_drv_dev_fw_info_resp *dev_fw_info_resp)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_DEV_GET_FW_INFO,
+ NULL, 0, dev_fw_info_resp,
+ sizeof(struct sxe2_drv_dev_fw_info_resp));
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret)
+ PMD_DEV_LOG_ERR(adapter, DRV, "get dev fw info failed, ret=%d", ret);
+
+ return ret;
+}
+
+s32 sxe2_drv_vsi_add(struct sxe2_adapter *adapter, struct sxe2_vsi *vsi)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+ struct sxe2_drv_vsi_create_req_resp vsi_req = {0};
+ struct sxe2_drv_vsi_create_req_resp vsi_resp = {0};
+
+ vsi_req.vsi_id = vsi->vsi_id;
+
+ vsi_req.used_queues.queues_cnt = RTE_MIN(vsi->txqs.q_cnt, vsi->rxqs.q_cnt);
+ vsi_req.used_queues.base_idx_in_pf = vsi->txqs.base_idx_in_func;
+ vsi_req.used_msix.msix_vectors_cnt = vsi->irqs.avail_cnt;
+ vsi_req.used_msix.base_idx_in_func = vsi->irqs.base_idx_in_pf;
+
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_VSI_CREATE,
+ &vsi_req, sizeof(struct sxe2_drv_vsi_create_req_resp),
+ &vsi_resp, sizeof(struct sxe2_drv_vsi_create_req_resp));
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret) {
+ PMD_DEV_LOG_ERR(adapter, DRV, "dev add vsi failed, ret=%d", ret);
+ goto l_end;
+ }
+
+ vsi->vsi_id = vsi_resp.vsi_id;
+ vsi->vsi_type = vsi_resp.vsi_type;
+
+l_end:
+ return ret;
+}
+
+s32 sxe2_drv_vsi_del(struct sxe2_adapter *adapter, struct sxe2_vsi *vsi)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+ struct sxe2_drv_vsi_free_req vsi_req = {0};
+
+ vsi_req.vsi_id = vsi->vsi_id;
+
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_VSI_FREE,
+ &vsi_req, sizeof(struct sxe2_drv_vsi_free_req),
+ NULL, 0);
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret)
+ PMD_DEV_LOG_ERR(adapter, DRV, "dev del vsi failed, ret=%d", ret);
+
+ return ret;
+}
+
+#define SXE2_RXQ_CTXT_CFG_BUF_LEN_ALIGN (1 << 7)
+#define SXE2_RX_HDR_SIZE 256
+
+static s32 sxe2_rxq_ctxt_cfg_fill(struct sxe2_rx_queue *rxq,
+ struct sxe2_drv_rxq_cfg_req *req, u16 rxq_cnt)
+{
+ struct sxe2_adapter *adapter = rxq->vsi->adapter;
+ struct sxe2_drv_rxq_ctxt *ctxt = req->cfg;
+ struct rte_eth_dev_data *dev_data = adapter->dev_info.dev_data;
+ s32 ret = SXE2_SUCCESS;
+
+ req->vsi_id = adapter->vsi_ctxt.main_vsi->vsi_id;
+ req->q_cnt = rxq_cnt;
+ req->max_frame_size = dev_data->mtu + SXE2_ETH_OVERHEAD;
+
+ ctxt->queue_id = rxq->queue_id;
+ ctxt->depth = rxq->ring_depth;
+ ctxt->buf_len = RTE_ALIGN(rxq->rx_buf_len, SXE2_RXQ_CTXT_CFG_BUF_LEN_ALIGN);
+ ctxt->dma_addr = rxq->base_addr;
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO) {
+ ctxt->lro_en = 1;
+ ctxt->max_lro_size = dev_data->dev_conf.rxmode.max_lro_pkt_size;
+ } else {
+ ctxt->lro_en = 0;
+ }
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ ctxt->keep_crc_en = 1;
+ else
+ ctxt->keep_crc_en = 0;
+
+ ctxt->desc_size = sizeof(union sxe2_rx_desc);
+ return ret;
+}
+
+s32 sxe2_drv_rxq_ctxt_cfg(struct sxe2_adapter *adapter, struct sxe2_rx_queue *rxq, u16 rxq_cnt)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+ struct sxe2_drv_rxq_cfg_req *req = NULL;
+ u16 len = 0;
+
+ len = sizeof(*req) + rxq_cnt * sizeof(struct sxe2_drv_rxq_ctxt);
+ req = rte_zmalloc("sxe2_rxq_cfg", len, 0);
+ if (req == NULL) {
+ PMD_LOG_ERR(RX, "rxq cfg mem alloc failed");
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ ret = sxe2_rxq_ctxt_cfg_fill(rxq, req, rxq_cnt);
+ if (ret) {
+ PMD_DEV_LOG_ERR(adapter, DRV, "rxq cfg failed, ret=%d", ret);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_RXQ_CFG_ENABLE,
+ req, len, NULL, 0);
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret)
+ PMD_DEV_LOG_ERR(adapter, DRV, "rxq cfg failed, ret=%d", ret);
+
+l_end:
+ if (req)
+ rte_free(req);
+ return ret;
+}
+
+static void sxe2_txq_ctxt_cfg_fill(struct sxe2_tx_queue *txq,
+ struct sxe2_drv_txq_cfg_req *req, u16 txq_cnt)
+{
+ struct sxe2_drv_txq_ctxt *ctxt = req->cfg;
+ u16 q_idx = 0;
+
+ req->vsi_id = txq->vsi->vsi_id;
+ req->q_cnt = txq_cnt;
+
+ for (q_idx = 0; q_idx < txq_cnt; q_idx++) {
+ ctxt = &req->cfg[q_idx];
+ ctxt->depth = txq[q_idx].ring_depth;
+ ctxt->dma_addr = txq[q_idx].base_addr;
+ ctxt->queue_id = txq[q_idx].queue_id;
+ }
+}
+
+s32 sxe2_drv_txq_ctxt_cfg(struct sxe2_adapter *adapter, struct sxe2_tx_queue *txq, u16 txq_cnt)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+ struct sxe2_drv_txq_cfg_req *req;
+ u16 len = 0;
+
+ len = sizeof(*req) + txq_cnt * sizeof(struct sxe2_drv_txq_ctxt);
+ req = rte_zmalloc("sxe2_txq_cfg", len, 0);
+ if (req == NULL) {
+ PMD_LOG_ERR(TX, "txq cfg mem alloc failed");
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ sxe2_txq_ctxt_cfg_fill(txq, req, txq_cnt);
+
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_TXQ_CFG_ENABLE,
+ req, len, NULL, 0);
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret)
+ PMD_DEV_LOG_ERR(adapter, DRV, "txq cfg failed, ret=%d", ret);
+
+l_end:
+ if (req)
+ rte_free(req);
+ return ret;
+}
+
+s32 sxe2_drv_rxq_switch(struct sxe2_adapter *adapter, struct sxe2_rx_queue *rxq, bool enable)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+ struct sxe2_drv_q_switch_req req;
+
+ req.vsi_id = rte_cpu_to_le_16(rxq->vsi->vsi_id);
+ req.q_idx = rxq->queue_id;
+
+ req.is_enable = (u8)enable;
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_RXQ_DISABLE,
+ &req, sizeof(req), NULL, 0);
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret)
+ PMD_DEV_LOG_ERR(adapter, DRV, "rxq switch failed, enable: %d, ret:%d",
+ enable, ret);
+
+ return ret;
+}
+
+s32 sxe2_drv_txq_switch(struct sxe2_adapter *adapter, struct sxe2_tx_queue *txq, bool enable)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_common_device *cdev = adapter->cdev;
+ struct sxe2_drv_cmd_params param = {0};
+ struct sxe2_drv_q_switch_req req;
+
+ req.vsi_id = rte_cpu_to_le_16(txq->vsi->vsi_id);
+ req.q_idx = txq->queue_id;
+
+ req.is_enable = (u8)enable;
+ sxe2_drv_cmd_params_fill(adapter, ¶m, SXE2_DRV_CMD_TXQ_DISABLE,
+ &req, sizeof(req), NULL, 0);
+
+ ret = sxe2_drv_cmd_exec(cdev, ¶m);
+ if (ret) {
+ PMD_DEV_LOG_ERR(adapter, DRV, "txq switch failed, enable: %d, ret:%d",
+ enable, ret);
+ }
+
+ return ret;
+}
diff --git a/drivers/net/sxe2/sxe2_cmd_chnl.h b/drivers/net/sxe2/sxe2_cmd_chnl.h
new file mode 100644
index 0000000000..200fe0be00
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_cmd_chnl.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_CMD_CHNL_H__
+#define __SXE2_CMD_CHNL_H__
+
+#include "sxe2_ethdev.h"
+#include "sxe2_drv_cmd.h"
+#include "sxe2_ioctl_chnl_func.h"
+
+s32 sxe2_drv_dev_caps_get(struct sxe2_adapter *adapter,
+ struct sxe2_drv_dev_caps_resp *dev_caps);
+
+s32 sxe2_drv_dev_info_get(struct sxe2_adapter *adapter,
+ struct sxe2_drv_dev_info_resp *dev_info_resp);
+
+s32 sxe2_drv_dev_fw_info_get(struct sxe2_adapter *adapter,
+ struct sxe2_drv_dev_fw_info_resp *dev_fw_info_resp);
+
+s32 sxe2_drv_vsi_add(struct sxe2_adapter *adapter, struct sxe2_vsi *vsi);
+
+s32 sxe2_drv_vsi_del(struct sxe2_adapter *adapter, struct sxe2_vsi *vsi);
+
+s32 sxe2_drv_rxq_switch(struct sxe2_adapter *adapter, struct sxe2_rx_queue *rxq, bool enable);
+
+s32 sxe2_drv_txq_switch(struct sxe2_adapter *adapter, struct sxe2_tx_queue *txq, bool enable);
+
+s32 sxe2_drv_rxq_ctxt_cfg(struct sxe2_adapter *adapter, struct sxe2_rx_queue *rxq, u16 rxq_cnt);
+
+s32 sxe2_drv_txq_ctxt_cfg(struct sxe2_adapter *adapter, struct sxe2_tx_queue *txq, u16 txq_cnt);
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_drv_cmd.h b/drivers/net/sxe2/sxe2_drv_cmd.h
new file mode 100644
index 0000000000..4094442077
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_drv_cmd.h
@@ -0,0 +1,398 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_DRV_CMD_H__
+#define __SXE2_DRV_CMD_H__
+
+#ifdef SXE2_DPDK_DRIVER
+#include "sxe2_type.h"
+#define SXE2_DPDK_RESOURCE_INSUFFICIENT
+#endif
+
+#ifdef SXE2_LINUX_DRIVER
+#ifdef __KERNEL__
+#include <linux/types.h>
+#include <linux/if_ether.h>
+#endif
+#endif
+
+#define SXE2_DRV_CMD_MODULE_S (16)
+#define SXE2_MK_DRV_CMD(module, cmd) (((module) << SXE2_DRV_CMD_MODULE_S) | ((cmd) & 0xFFFF))
+
+#define SXE2_DEV_CAPS_OFFLOAD_L2 BIT(0)
+#define SXE2_DEV_CAPS_OFFLOAD_VLAN BIT(1)
+#define SXE2_DEV_CAPS_OFFLOAD_RSS BIT(2)
+#define SXE2_DEV_CAPS_OFFLOAD_IPSEC BIT(3)
+#define SXE2_DEV_CAPS_OFFLOAD_FNAV BIT(4)
+#define SXE2_DEV_CAPS_OFFLOAD_TM BIT(5)
+#define SXE2_DEV_CAPS_OFFLOAD_PTP BIT(6)
+#define SXE2_DEV_CAPS_OFFLOAD_Q_MAP BIT(7)
+#define SXE2_DEV_CAPS_OFFLOAD_FC_STATE BIT(8)
+
+#define SXE2_TXQ_STATS_MAP_MAX_NUM 16
+#define SXE2_RXQ_STATS_MAP_MAX_NUM 4
+#define SXE2_RXQ_MAP_Q_MAX_NUM 256
+
+#define SXE2_STAT_MAP_INVALID_QID 0xFFFF
+
+#define SXE2_SCHED_MODE_DEFAULT 0
+#define SXE2_SCHED_MODE_TM 1
+#define SXE2_SCHED_MODE_HIGH_PERFORMANCE 2
+#define SXE2_SCHED_MODE_INVALID 3
+
+#define SXE2_SRCVSI_PRUNE_MAX_NUM 2
+
+#define SXE2_PTYPE_UNKNOWN BIT(0)
+#define SXE2_PTYPE_L2_ETHER BIT(1)
+#define SXE2_PTYPE_L3_IPV4 BIT(2)
+#define SXE2_PTYPE_L3_IPV6 BIT(4)
+#define SXE2_PTYPE_L4_TCP BIT(6)
+#define SXE2_PTYPE_L4_UDP BIT(7)
+#define SXE2_PTYPE_L4_SCTP BIT(8)
+#define SXE2_PTYPE_INNER_L2_ETHER BIT(9)
+#define SXE2_PTYPE_INNER_L3_IPV4 BIT(10)
+#define SXE2_PTYPE_INNER_L3_IPV6 BIT(12)
+#define SXE2_PTYPE_INNER_L4_TCP BIT(14)
+#define SXE2_PTYPE_INNER_L4_UDP BIT(15)
+#define SXE2_PTYPE_INNER_L4_SCTP BIT(16)
+#define SXE2_PTYPE_TUNNEL_GRENAT BIT(17)
+
+#define SXE2_PTYPE_L2_MASK (SXE2_PTYPE_L2_ETHER)
+#define SXE2_PTYPE_L3_MASK (SXE2_PTYPE_L3_IPV4 | SXE2_PTYPE_L3_IPV6)
+#define SXE2_PTYPE_L4_MASK (SXE2_PTYPE_L4_TCP | SXE2_PTYPE_L4_UDP | \
+ SXE2_PTYPE_L4_SCTP)
+#define SXE2_PTYPE_INNER_L2_MASK (SXE2_PTYPE_INNER_L2_ETHER)
+#define SXE2_PTYPE_INNER_L3_MASK (SXE2_PTYPE_INNER_L3_IPV4 | \
+ SXE2_PTYPE_INNER_L3_IPV6)
+#define SXE2_PTYPE_INNER_L4_MASK (SXE2_PTYPE_INNER_L4_TCP | \
+ SXE2_PTYPE_INNER_L4_UDP | \
+ SXE2_PTYPE_INNER_L4_SCTP)
+#define SXE2_PTYPE_TUNNEL_MASK (SXE2_PTYPE_TUNNEL_GRENAT)
+
+enum sxe2_dev_type {
+ SXE2_DEV_T_PF = 0,
+ SXE2_DEV_T_VF,
+ SXE2_DEV_T_PF_BOND,
+ SXE2_DEV_T_MAX,
+};
+
+struct sxe2_drv_queue_caps {
+ __le16 queues_cnt;
+ __le16 base_idx_in_pf;
+};
+
+struct sxe2_drv_msix_caps {
+ __le16 msix_vectors_cnt;
+ __le16 base_idx_in_func;
+};
+
+struct sxe2_drv_rss_hash_caps {
+ __le16 hash_key_size;
+ __le16 lut_key_size;
+};
+
+enum sxe2_vf_vsi_valid {
+ SXE2_VF_VSI_BOTH = 0,
+ SXE2_VF_VSI_ONLY_DPDK,
+ SXE2_VF_VSI_ONLY_KERNEL,
+ SXE2_VF_VSI_MAX,
+};
+
+struct sxe2_drv_vsi_caps {
+ __le16 func_id;
+ __le16 dpdk_vsi_id;
+ __le16 kernel_vsi_id;
+ __le16 vsi_type;
+};
+
+struct sxe2_drv_representor_caps {
+ __le16 cnt_repr_vf;
+ u8 rsv[2];
+ struct sxe2_drv_vsi_caps repr_vf_id[256];
+};
+
+enum sxe2_phys_port_name_type {
+ SXE2_PHYS_PORT_NAME_TYPE_NOTSET = 0,
+ SXE2_PHYS_PORT_NAME_TYPE_LEGACY,
+ SXE2_PHYS_PORT_NAME_TYPE_UPLINK,
+ SXE2_PHYS_PORT_NAME_TYPE_PFVF,
+
+ SXE2_PHYS_PORT_NAME_TYPE_UNKNOWN,
+};
+
+struct sxe2_switchdev_mode_info {
+ u8 pf_id;
+ u8 is_switchdev;
+ u8 rsv[2];
+};
+
+struct sxe2_switchdev_cpvsi_info {
+ __le16 cp_vsi_id;
+ u8 rsv[2];
+};
+
+struct sxe2_txsch_caps {
+ u8 layer_cap;
+ u8 tm_mid_node_num;
+ u8 prio_num;
+ u8 rev;
+};
+
+struct sxe2_drv_dev_caps_resp {
+ struct sxe2_drv_queue_caps queue_caps;
+ struct sxe2_drv_msix_caps msix_caps;
+ struct sxe2_drv_rss_hash_caps rss_hash_caps;
+ struct sxe2_drv_vsi_caps vsi_caps;
+ struct sxe2_txsch_caps txsch_caps;
+ struct sxe2_drv_representor_caps repr_caps;
+ u8 port_idx;
+ u8 pf_idx;
+ u8 dev_type;
+ u8 rev;
+ __le32 cap_flags;
+};
+
+struct sxe2_drv_dev_info_resp {
+ __le64 dsn;
+ __le16 vsi_id;
+ u8 rsv[2];
+ u8 mac_addr[ETH_ALEN];
+ u8 rsv2[2];
+};
+
+struct sxe2_drv_dev_fw_info_resp {
+ u8 main_version_id;
+ u8 sub_version_id;
+ u8 fix_version_id;
+ u8 build_id;
+};
+
+struct sxe2_drv_rxq_ctxt {
+ __le64 dma_addr;
+ __le32 max_lro_size;
+ __le32 split_type_mask;
+ __le16 hdr_len;
+ __le16 buf_len;
+ __le16 depth;
+ __le16 queue_id;
+ u8 lro_en;
+ u8 keep_crc_en;
+ u8 split_en;
+ u8 desc_size;
+};
+
+struct sxe2_drv_rxq_cfg_req {
+ __le16 q_cnt;
+ __le16 vsi_id;
+ __le16 max_frame_size;
+ u8 rsv[2];
+ struct sxe2_drv_rxq_ctxt cfg[];
+};
+
+struct sxe2_drv_txq_ctxt {
+ __le64 dma_addr;
+ __le32 sched_mode;
+ __le16 queue_id;
+ __le16 depth;
+ __le16 vsi_id;
+ u8 rsv[2];
+};
+
+struct sxe2_drv_txq_cfg_req {
+ __le16 q_cnt;
+ __le16 vsi_id;
+ struct sxe2_drv_txq_ctxt cfg[];
+};
+
+struct sxe2_drv_q_switch_req {
+ __le16 q_idx;
+ __le16 vsi_id;
+ u8 is_enable;
+ u8 sched_mode;
+ u8 rsv[2];
+};
+
+struct sxe2_drv_vsi_create_req_resp {
+ __le16 vsi_id;
+ __le16 vsi_type;
+ struct sxe2_drv_queue_caps used_queues;
+ struct sxe2_drv_msix_caps used_msix;
+};
+
+struct sxe2_drv_vsi_free_req {
+ __le16 vsi_id;
+ u8 rsv[2];
+};
+
+struct sxe2_drv_vsi_info_get_req {
+ __le16 vsi_id;
+ u8 rsv[2];
+};
+
+struct sxe2_drv_vsi_info_get_resp {
+ __le16 vsi_id;
+ __le16 vsi_type;
+ struct sxe2_drv_queue_caps used_queues;
+ struct sxe2_drv_msix_caps used_msix;
+};
+
+enum sxe2_drv_cmd_module {
+ SXE2_DRV_CMD_MODULE_HANDSHAKE = 0,
+ SXE2_DRV_CMD_MODULE_DEV = 1,
+ SXE2_DRV_CMD_MODULE_VSI = 2,
+ SXE2_DRV_CMD_MODULE_QUEUE = 3,
+ SXE2_DRV_CMD_MODULE_STATS = 4,
+ SXE2_DRV_CMD_MODULE_SUBSCRIBE = 5,
+ SXE2_DRV_CMD_MODULE_RSS = 6,
+ SXE2_DRV_CMD_MODULE_FLOW = 7,
+ SXE2_DRV_CMD_MODULE_TM = 8,
+ SXE2_DRV_CMD_MODULE_IPSEC = 9,
+ SXE2_DRV_CMD_MODULE_PTP = 10,
+
+ SXE2_DRV_CMD_MODULE_VLAN = 11,
+ SXE2_DRV_CMD_MODULE_RDMA = 12,
+ SXE2_DRV_CMD_MODULE_LINK = 13,
+ SXE2_DRV_CMD_MODULE_MACADDR = 14,
+ SXE2_DRV_CMD_MODULE_PROMISC = 15,
+
+ SXE2_DRV_CMD_MODULE_LED = 16,
+ SXE2_DEV_CMD_MODULE_OPT = 17,
+ SXE2_DEV_CMD_MODULE_SWITCH = 18,
+ SXE2_DRV_CMD_MODULE_ACL = 19,
+ SXE2_DRV_CMD_MODULE_UDPTUNEEL = 20,
+ SXE2_DRV_CMD_MODULE_QUEUE_MAP = 21,
+
+ SXE2_DRV_CMD_MODULE_SCHED = 22,
+
+ SXE2_DRV_CMD_MODULE_IRQ = 23,
+
+ SXE2_DRV_CMD_MODULE_OPT = 24,
+};
+
+enum sxe2_drv_cmd_code {
+ SXE2_DRV_CMD_HANDSHAKE_ENABLE =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_HANDSHAKE, 1),
+ SXE2_DRV_CMD_HANDSHAKE_DISABLE,
+
+ SXE2_DRV_CMD_DEV_GET_CAPS =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_DEV, 1),
+ SXE2_DRV_CMD_DEV_GET_INFO,
+ SXE2_DRV_CMD_DEV_GET_FW_INFO,
+ SXE2_DRV_CMD_DEV_RESET,
+ SXE2_DRV_CMD_DEV_GET_SWITCHDEV_INFO,
+
+ SXE2_DRV_CMD_VSI_CREATE =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_VSI, 1),
+ SXE2_DRV_CMD_VSI_FREE,
+ SXE2_DRV_CMD_VSI_INFO_GET,
+ SXE2_DRV_CMD_VSI_SRCVSI_PRUNE,
+ SXE2_DRV_CMD_VSI_FC_GET,
+
+ SXE2_DRV_CMD_RX_MAP_SET =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_QUEUE_MAP, 1),
+ SXE2_DRV_CMD_TX_MAP_SET,
+ SXE2_DRV_CMD_TX_RX_MAP_GET,
+ SXE2_DRV_CMD_TX_RX_MAP_RESET,
+ SXE2_DRV_CMD_TX_RX_MAP_INFO_CLEAR,
+
+ SXE2_DRV_CMD_SCHED_ROOT_TREE_ALLOC =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_SCHED, 1),
+ SXE2_DRV_CMD_SCHED_ROOT_TREE_RELEASE,
+ SXE2_DRV_CMD_SCHED_ROOT_CHILDREN_DELETE,
+ SXE2_DRV_CMD_SCHED_TM_ADD_MID_NODE,
+ SXE2_DRV_CMD_SCHED_TM_ADD_QUEUE_NODE,
+
+ SXE2_DRV_CMD_RXQ_CFG_ENABLE =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_QUEUE, 1),
+ SXE2_DRV_CMD_TXQ_CFG_ENABLE,
+ SXE2_DRV_CMD_RXQ_DISABLE,
+ SXE2_DRV_CMD_TXQ_DISABLE,
+
+ SXE2_DRV_CMD_VSI_STATS_GET =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_STATS, 1),
+ SXE2_DRV_CMD_VSI_STATS_CLEAR,
+ SXE2_DRV_CMD_MAC_STATS_GET,
+ SXE2_DRV_CMD_MAC_STATS_CLEAR,
+
+ SXE2_DRV_CMD_RSS_KEY_SET =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_RSS, 1),
+ SXE2_DRV_CMD_RSS_LUT_SET,
+ SXE2_DRV_CMD_RSS_FUNC_SET,
+ SXE2_DRV_CMD_RSS_HF_ADD,
+ SXE2_DRV_CMD_RSS_HF_DEL,
+ SXE2_DRV_CMD_RSS_HF_CLEAR,
+
+ SXE2_DRV_CMD_FLOW_FILTER_ADD =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_FLOW, 1),
+ SXE2_DRV_CMD_FLOW_FILTER_DEL,
+ SXE2_DRV_CMD_FLOW_FILTER_CLEAR,
+ SXE2_DRV_CMD_FLOW_FNAV_STAT_ALLOC,
+ SXE2_DRV_CMD_FLOW_FNAV_STAT_FREE,
+ SXE2_DRV_CMD_FLOW_FNAV_STAT_QUERY,
+
+ SXE2_DRV_CMD_DEL_TM_ROOT =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_TM, 1),
+ SXE2_DRV_CMD_ADD_TM_ROOT,
+ SXE2_DRV_CMD_ADD_TM_NODE,
+ SXE2_DRV_CMD_ADD_TM_QUEUE,
+
+ SXE2_DRV_CMD_GET_PTP_CLOCK =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_PTP, 1),
+
+ SXE2_DRV_CMD_VLAN_FILTER_ADD_DEL =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_VLAN, 1),
+ SXE2_DRV_CMD_VLAN_FILTER_SWITCH,
+ SXE2_DRV_CMD_VLAN_OFFLOAD_CFG,
+ SXE2_DRV_CMD_VLAN_PORTVLAN_CFG,
+ SXE2_DRV_CMD_VLAN_CFG_QUERY,
+
+ SXE2_DRV_CMD_RDMA_DUMP_PCAP =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_RDMA, 1),
+
+ SXE2_DRV_CMD_LINK_STATUS_GET =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_LINK, 1),
+
+ SXE2_DRV_CMD_MAC_ADDR_UC =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_MACADDR, 1),
+ SXE2_DRV_CMD_MAC_ADDR_MC,
+
+ SXE2_DRV_CMD_PROMISC_CFG =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_PROMISC, 1),
+ SXE2_DRV_CMD_ALLMULTI_CFG,
+
+ SXE2_DRV_CMD_LED_CTRL =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_LED, 1),
+
+ SXE2_DRV_CMD_OPT_EEP =
+ SXE2_MK_DRV_CMD(SXE2_DEV_CMD_MODULE_OPT, 1),
+
+ SXE2_DRV_CMD_SWITCH =
+ SXE2_MK_DRV_CMD(SXE2_DEV_CMD_MODULE_SWITCH, 1),
+ SXE2_DRV_CMD_SWITCH_UPLINK,
+ SXE2_DRV_CMD_SWITCH_REPR,
+ SXE2_DRV_CMD_SWITCH_MODE,
+ SXE2_DRV_CMD_SWITCH_CPVSI,
+
+ SXE2_DRV_CMD_UDPTUNNEL_ADD =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_UDPTUNEEL, 1),
+ SXE2_DRV_CMD_UDPTUNNEL_DEL,
+ SXE2_DRV_CMD_UDPTUNNEL_GET,
+
+ SXE2_DRV_CMD_IPSEC_CAP_GET =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_IPSEC, 1),
+ SXE2_DRV_CMD_IPSEC_TXSA_ADD,
+ SXE2_DRV_CMD_IPSEC_RXSA_ADD,
+ SXE2_DRV_CMD_IPSEC_TXSA_DEL,
+ SXE2_DRV_CMD_IPSEC_RXSA_DEL,
+ SXE2_DRV_CMD_IPSEC_RESOURCE_CLEAR,
+
+ SXE2_DRV_CMD_EVT_IRQ_BAND_RXQ =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_IRQ, 1),
+
+ SXE2_DRV_CMD_OPT_EEP_GET =
+ SXE2_MK_DRV_CMD(SXE2_DRV_CMD_MODULE_OPT, 1),
+
+};
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
new file mode 100644
index 0000000000..a6cb51789e
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -0,0 +1,611 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_string_fns.h>
+#include <ethdev_pci.h>
+#include <ctype.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <rte_tailq.h>
+#include <rte_version.h>
+#include <bus_pci_driver.h>
+#include <dev_driver.h>
+#include <ethdev_driver.h>
+#include <rte_ethdev.h>
+#include <rte_alarm.h>
+#include <rte_dev_info.h>
+#include <rte_pci.h>
+#include <rte_mbuf_dyn.h>
+#include <rte_cycles.h>
+#include <rte_eal_paging.h>
+
+#include "sxe2_ethdev.h"
+#include "sxe2_drv_cmd.h"
+#include "sxe2_cmd_chnl.h"
+#include "sxe2_common.h"
+#include "sxe2_common_log.h"
+#include "sxe2_host_regs.h"
+#include "sxe2_ioctl_chnl_func.h"
+
+#define SXE2_PCI_VENDOR_ID_1 0x1ff2
+#define SXE2_PCI_DEVICE_ID_PF_1 0x10b1
+#define SXE2_PCI_DEVICE_ID_VF_1 0x10b2
+
+#define SXE2_PCI_VENDOR_ID_2 0x1d94
+#define SXE2_PCI_DEVICE_ID_PF_2 0x1260
+#define SXE2_PCI_DEVICE_ID_VF_2 0x126f
+
+#define SXE2_PCI_DEVICE_ID_PF_3 0x10b3
+#define SXE2_PCI_DEVICE_ID_VF_3 0x10b4
+
+#define SXE2_PCI_VENDOR_ID_206F 0x206f
+
+static const struct rte_pci_id pci_id_sxe2_tbl[] = {
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_1, SXE2_PCI_DEVICE_ID_PF_1)},
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_1, SXE2_PCI_DEVICE_ID_VF_1)},
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_2, SXE2_PCI_DEVICE_ID_PF_2)},
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_2, SXE2_PCI_DEVICE_ID_VF_2)},
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_1, SXE2_PCI_DEVICE_ID_PF_3)},
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_1, SXE2_PCI_DEVICE_ID_VF_3)},
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_206F, SXE2_PCI_DEVICE_ID_PF_1)},
+ { RTE_PCI_DEVICE(SXE2_PCI_VENDOR_ID_206F, SXE2_PCI_DEVICE_ID_VF_1)},
+ { .vendor_id = 0, },
+};
+
+static s32 sxe2_dev_configure(struct rte_eth_dev *dev)
+{
+ s32 ret = SXE2_SUCCESS;
+ PMD_INIT_FUNC_TRACE();
+
+ if (dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS_FLAG)
+ dev->data->dev_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_RSS_HASH;
+
+ return ret;
+}
+
+static void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev __rte_unused)
+{
+}
+
+static void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev __rte_unused)
+{
+}
+
+static s32 sxe2_dev_stop(struct rte_eth_dev *dev)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ PMD_INIT_FUNC_TRACE();
+
+ if (adapter->started == 0)
+ goto l_end;
+
+ sxe2_txqs_all_stop(dev);
+ sxe2_rxqs_all_stop(dev);
+
+ dev->data->dev_started = 0;
+ adapter->started = 0;
+l_end:
+ return ret;
+}
+
+static s32 __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev __rte_unused)
+{
+ return 0;
+}
+
+static s32 __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev __rte_unused)
+{
+ return 0;
+}
+
+static s32 sxe2_queues_start(struct rte_eth_dev *dev)
+{
+ s32 ret = SXE2_SUCCESS;
+ ret = sxe2_txqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start tx queue.");
+ goto l_end;
+ }
+
+ ret = sxe2_rxqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start rx queue.");
+ sxe2_txqs_all_stop(dev);
+ }
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_dev_start(struct rte_eth_dev *dev)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ PMD_INIT_FUNC_TRACE();
+
+ ret = sxe2_queues_init(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to init queues.");
+ goto l_end;
+ }
+
+ ret = sxe2_queues_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "enable queues failed");
+ goto l_end;
+ }
+
+ dev->data->dev_started = 1;
+ adapter->started = 1;
+ goto l_end;
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_dev_close(struct rte_eth_dev *dev)
+{
+ (void)sxe2_dev_stop(dev);
+
+ sxe2_vsi_uninit(dev);
+
+ return SXE2_SUCCESS;
+}
+
+static s32 sxe2_dev_infos_get(struct rte_eth_dev *dev,
+ struct rte_eth_dev_info *dev_info)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+
+ dev_info->max_rx_queues = vsi->rxqs.q_cnt;
+ dev_info->max_tx_queues = vsi->txqs.q_cnt;
+ dev_info->min_rx_bufsize = SXE2_MIN_BUF_SIZE;
+ dev_info->max_rx_pktlen = SXE2_FRAME_SIZE_MAX;
+ dev_info->max_lro_pkt_size = SXE2_FRAME_SIZE_MAX * SXE2_RX_LRO_DESC_MAX_NUM;
+ dev_info->max_mtu = dev_info->max_rx_pktlen - SXE2_ETH_OVERHEAD;
+ dev_info->min_mtu = RTE_ETHER_MIN_MTU;
+
+ dev_info->rx_offload_capa =
+ RTE_ETH_RX_OFFLOAD_KEEP_CRC |
+ RTE_ETH_RX_OFFLOAD_SCATTER |
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT |
+ RTE_ETH_RX_OFFLOAD_TCP_LRO;
+
+ dev_info->tx_offload_capa =
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE |
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_UDP_TSO |
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
+
+ dev_info->rx_queue_offload_capa =
+ RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT |
+ RTE_ETH_RX_OFFLOAD_KEEP_CRC |
+ RTE_ETH_RX_OFFLOAD_SCATTER |
+ RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM |
+ RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_RX_OFFLOAD_TCP_LRO;
+ dev_info->tx_queue_offload_capa =
+ RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE |
+ RTE_ETH_TX_OFFLOAD_TCP_TSO |
+ RTE_ETH_TX_OFFLOAD_UDP_TSO |
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS |
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO |
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
+
+ dev_info->default_rxconf = (struct rte_eth_rxconf) {
+ .rx_thresh = {
+ .pthresh = SXE2_DEFAULT_RX_PTHRESH,
+ .hthresh = SXE2_DEFAULT_RX_HTHRESH,
+ .wthresh = SXE2_DEFAULT_RX_WTHRESH,
+ },
+ .rx_free_thresh = SXE2_DEFAULT_RX_FREE_THRESH,
+ .rx_drop_en = 0,
+ .offloads = 0,
+ };
+
+ dev_info->default_txconf = (struct rte_eth_txconf) {
+ .tx_thresh = {
+ .pthresh = SXE2_DEFAULT_TX_PTHRESH,
+ .hthresh = SXE2_DEFAULT_TX_HTHRESH,
+ .wthresh = SXE2_DEFAULT_TX_WTHRESH,
+ },
+ .tx_free_thresh = SXE2_DEFAULT_TX_FREE_THRESH,
+ .tx_rs_thresh = SXE2_DEFAULT_TX_RSBIT_THRESH,
+ .offloads = 0,
+ };
+
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = SXE2_MAX_RING_DESC,
+ .nb_min = SXE2_MIN_RING_DESC,
+ .nb_align = SXE2_ALIGN,
+ };
+
+ dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = SXE2_MAX_RING_DESC,
+ .nb_min = SXE2_MIN_RING_DESC,
+ .nb_align = SXE2_ALIGN,
+ .nb_mtu_seg_max = SXE2_TX_MTU_SEG_MAX,
+ .nb_seg_max = SXE2_MAX_RING_DESC,
+ };
+
+ dev_info->speed_capa = RTE_ETH_LINK_SPEED_10G | RTE_ETH_LINK_SPEED_25G |
+ RTE_ETH_LINK_SPEED_50G | RTE_ETH_LINK_SPEED_100G;
+
+ dev_info->default_rxportconf.burst_size = SXE2_RX_MAX_BURST;
+ dev_info->default_txportconf.burst_size = SXE2_TX_MAX_BURST;
+ dev_info->default_rxportconf.nb_queues = 1;
+ dev_info->default_txportconf.nb_queues = 1;
+ dev_info->default_rxportconf.ring_size = SXE2_RING_SIZE_MIN;
+ dev_info->default_txportconf.ring_size = SXE2_RING_SIZE_MIN;
+
+ dev_info->rx_seg_capa.max_nseg = SXE2_RX_MAX_NSEG;
+
+ dev_info->rx_seg_capa.multi_pools = true;
+
+ dev_info->rx_seg_capa.offset_allowed = false;
+
+ dev_info->rx_seg_capa.offset_align_log2 = false;
+
+ return SXE2_SUCCESS;
+}
+
+static const struct eth_dev_ops sxe2_eth_dev_ops = {
+ .dev_configure = sxe2_dev_configure,
+ .dev_start = sxe2_dev_start,
+ .dev_stop = sxe2_dev_stop,
+ .dev_close = sxe2_dev_close,
+ .dev_infos_get = sxe2_dev_infos_get,
+};
+
+static void sxe2_drv_dev_caps_set(struct sxe2_adapter *adapter,
+ struct sxe2_drv_dev_caps_resp *dev_caps)
+{
+ adapter->port_idx = dev_caps->port_idx;
+
+ adapter->cap_flags = 0;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_L2)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_L2;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_VLAN)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_VLAN;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_RSS)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_RSS;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_IPSEC)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_IPSEC;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_FNAV)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_FNAV;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_TM)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_TM;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_PTP)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_PTP;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_Q_MAP)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_Q_MAP;
+
+ if (dev_caps->cap_flags & SXE2_DEV_CAPS_OFFLOAD_FC_STATE)
+ adapter->cap_flags |= SXE2_DEV_CAPS_OFFLOAD_FC_STATE;
+}
+
+static s32 sxe2_func_caps_get(struct sxe2_adapter *adapter)
+{
+ s32 ret = SXE2_ERROR;
+ struct sxe2_drv_dev_caps_resp dev_caps = {0};
+
+ ret = sxe2_drv_dev_caps_get(adapter, &dev_caps);
+ if (ret)
+ goto l_end;
+
+ adapter->dev_type = dev_caps.dev_type;
+
+ sxe2_drv_dev_caps_set(adapter, &dev_caps);
+
+ sxe2_sw_queue_ctx_hw_cap_set(adapter, &dev_caps.queue_caps);
+
+ sxe2_sw_vsi_ctx_hw_cap_set(adapter, &dev_caps.vsi_caps);
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_dev_caps_get(struct sxe2_adapter *adapter)
+{
+ s32 ret = SXE2_ERROR;
+
+ ret = sxe2_func_caps_get(adapter);
+ if (ret)
+ PMD_LOG_ERR(INIT, "get function caps failed, ret=%d", ret);
+
+ return ret;
+}
+
+static s32 sxe2_hw_init(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ s32 ret = SXE2_ERROR;
+
+ PMD_INIT_FUNC_TRACE();
+
+ ret = sxe2_dev_caps_get(adapter);
+ if (ret)
+ PMD_LOG_ERR(INIT, "Failed to get device caps, ret=[%d]", ret);
+
+ return ret;
+}
+
+static s32 sxe2_dev_info_init(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter =
+ SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev->device);
+ struct sxe2_dev_info *dev_info = &adapter->dev_info;
+ struct sxe2_drv_dev_info_resp dev_info_resp = {0};
+ struct sxe2_drv_dev_fw_info_resp dev_fw_info_resp = {0};
+ s32 ret = SXE2_SUCCESS;
+
+ dev_info->pci.bus_devid = pci_dev->addr.devid;
+ dev_info->pci.bus_function = pci_dev->addr.function;
+
+ ret = sxe2_drv_dev_info_get(adapter, &dev_info_resp);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to get device info, ret=[%d]", ret);
+ goto l_end;
+ }
+ dev_info->pci.serial_number = dev_info_resp.dsn;
+
+ ret = sxe2_drv_dev_fw_info_get(adapter, &dev_fw_info_resp);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to get device fw info, ret=[%d]", ret);
+ goto l_end;
+ }
+ dev_info->fw.build_id = dev_fw_info_resp.build_id;
+ dev_info->fw.fix_version_id = dev_fw_info_resp.fix_version_id;
+ dev_info->fw.sub_version_id = dev_fw_info_resp.sub_version_id;
+ dev_info->fw.main_version_id = dev_fw_info_resp.main_version_id;
+
+ if (rte_is_valid_assigned_ether_addr((struct rte_ether_addr *)dev_info_resp.mac_addr))
+ rte_ether_addr_copy((struct rte_ether_addr *)dev_info_resp.mac_addr,
+ (struct rte_ether_addr *)dev_info->mac.perm_addr);
+ else
+ rte_eth_random_addr(dev_info->mac.perm_addr);
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_dev_init(struct rte_eth_dev *dev, struct sxe2_dev_kvargs_info *kvargs __rte_unused)
+{
+ s32 ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->dev_ops = &sxe2_eth_dev_ops;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ goto l_end;
+
+ ret = sxe2_hw_init(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to initialize hw, ret=[%d]", ret);
+ goto l_end;
+ }
+
+ ret = sxe2_vsi_init(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "create main vsi failed, ret=%d", ret);
+ goto init_vsi_err;
+ }
+
+ ret = sxe2_dev_info_init(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to get device info, ret=[%d]", ret);
+ goto init_dev_info_err;
+ }
+
+ goto l_end;
+
+init_dev_info_err:
+ sxe2_vsi_uninit(dev);
+init_vsi_err:
+l_end:
+ return ret;
+}
+
+static s32 sxe2_dev_uninit(struct rte_eth_dev *dev)
+{
+ s32 ret = 0;
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ goto l_end;
+
+ ret = sxe2_dev_close(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Sxe2 dev close failed, ret=%d", ret);
+ goto l_end;
+ }
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_eth_pmd_remove(struct sxe2_common_device *cdev)
+{
+ struct rte_eth_dev *eth_dev;
+ struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(cdev->dev);
+ s32 ret = SXE2_SUCCESS;
+
+ eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
+ if (!eth_dev) {
+ PMD_LOG_INFO(INIT, "Sxe2 dev allocated failed");
+ goto l_end;
+ }
+
+ ret = sxe2_dev_uninit(eth_dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Sxe2 dev uninit failed, ret=%d", ret);
+ goto l_end;
+ }
+ (void)rte_eth_dev_release_port(eth_dev);
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_eth_pmd_probe_pf(struct sxe2_common_device *cdev,
+ struct rte_eth_devargs *req_eth_da __rte_unused,
+ u16 owner_id __rte_unused,
+ struct sxe2_dev_kvargs_info *kvargs)
+{
+ struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(cdev->dev);
+ struct rte_eth_dev *eth_dev = NULL;
+ struct sxe2_adapter *adapter = NULL;
+ s32 ret = SXE2_SUCCESS;
+
+ if (!cdev) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ eth_dev = rte_eth_dev_pci_allocate(pci_dev, sizeof(struct sxe2_adapter));
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ if (eth_dev == NULL) {
+ PMD_LOG_ERR(INIT, "Can not allocate ethdev");
+ ret = -ENOMEM;
+ goto l_end;
+ }
+ } else {
+ if (!eth_dev) {
+ PMD_LOG_DEBUG(INIT, "Can not attach secondary ethdev");
+ ret = -EINVAL;
+ goto l_end;
+ }
+ }
+
+ adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(eth_dev);
+ adapter->dev_port_id = eth_dev->data->port_id;
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ adapter->cdev = cdev;
+
+ ret = sxe2_dev_init(eth_dev, kvargs);
+ if (ret != SXE2_SUCCESS) {
+ PMD_DEV_LOG_ERR(adapter, INIT, "Sxe2 dev init failed, ret=%d", ret);
+ goto l_release_port;
+ }
+
+ rte_eth_dev_probing_finish(eth_dev);
+ PMD_DEV_LOG_DEBUG(adapter, INIT, "Sxe2 eth pmd probe successful!");
+ goto l_end;
+
+l_release_port:
+ (void)rte_eth_dev_release_port(eth_dev);
+l_end:
+ return ret;
+}
+
+static s32 sxe2_parse_eth_devargs(struct rte_device *dev,
+ struct rte_eth_devargs *eth_da)
+{
+ int ret = 0;
+
+ if (dev->devargs == NULL)
+ return 0;
+
+ memset(eth_da, 0, sizeof(*eth_da));
+
+ if (dev->devargs->cls_str) {
+ ret = rte_eth_devargs_parse(dev->devargs->cls_str, eth_da, 1);
+ if (ret != 0) {
+ PMD_LOG_ERR(INIT, "Failed to parse device arguments: %s",
+ dev->devargs->cls_str);
+ return -rte_errno;
+ }
+ }
+
+ if (eth_da->type == RTE_ETH_REPRESENTOR_NONE && dev->devargs->args) {
+ ret = rte_eth_devargs_parse(dev->devargs->args, eth_da, 1);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to parse device arguments: %s",
+ dev->devargs->args);
+ return -rte_errno;
+ }
+ }
+
+ return 0;
+}
+
+static s32 sxe2_eth_pmd_probe(struct sxe2_common_device *cdev, struct sxe2_dev_kvargs_info *kvargs)
+{
+ struct rte_eth_devargs eth_da = { .nb_ports = 0 };
+ s32 ret = SXE2_SUCCESS;
+
+ ret = sxe2_parse_eth_devargs(cdev->dev, ð_da);
+ if (ret != 0) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ ret = sxe2_eth_pmd_probe_pf(cdev, ð_da, 0, kvargs);
+
+l_end:
+ return ret;
+}
+
+static struct sxe2_class_driver sxe2_eth_pmd = {
+ .drv_class = SXE2_CLASS_TYPE_ETH,
+ .name = "SXE2_ETH_PMD_DRIVER_NAME",
+ .probe = sxe2_eth_pmd_probe,
+ .remove = sxe2_eth_pmd_remove,
+ .id_table = pci_id_sxe2_tbl,
+ .intr_lsc = 1,
+ .intr_rmv = 1,
+};
+
+RTE_INIT(rte_sxe2_pmd_init)
+{
+ sxe2_common_init();
+ sxe2_class_driver_register(&sxe2_eth_pmd);
+}
+
+RTE_PMD_EXPORT_NAME(net_sxe2);
+RTE_PMD_REGISTER_PCI_TABLE(net_sxe2, pci_id_sxe2_tbl);
+RTE_PMD_REGISTER_KMOD_DEP(net_sxe2, "* sxe2");
+
+RTE_LOG_REGISTER_SUFFIX(sxe2_log_init, init, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(sxe2_log_driver, driver, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(sxe2_log_rx, rx, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(sxe2_log_tx, tx, NOTICE);
+RTE_LOG_REGISTER_SUFFIX(sxe2_log_hw, hw, NOTICE);
diff --git a/drivers/net/sxe2/sxe2_ethdev.h b/drivers/net/sxe2/sxe2_ethdev.h
new file mode 100644
index 0000000000..412f5d2b14
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_ethdev.h
@@ -0,0 +1,295 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+#ifndef __SXE2_ETHDEV_H__
+#define __SXE2_ETHDEV_H__
+#include <rte_compat.h>
+#include <rte_kvargs.h>
+#include <rte_time.h>
+#include <ethdev_driver.h>
+#include <ethdev_pci.h>
+#include <rte_tm_driver.h>
+#include <rte_io.h>
+
+#include "sxe2_common.h"
+#include "sxe2_errno.h"
+#include "sxe2_type.h"
+#include "sxe2_vsi.h"
+#include "sxe2_queue.h"
+#include "sxe2_irq.h"
+#include "sxe2_osal.h"
+
+struct sxe2_link_msg {
+ __le32 speed;
+ u8 status;
+};
+
+enum sxe2_fnav_tunnel_flag_type {
+ SXE2_FNAV_TUN_FLAG_NO_TUNNEL,
+ SXE2_FNAV_TUN_FLAG_TUNNEL,
+ SXE2_FNAV_TUN_FLAG_ANY,
+};
+
+#define SXE2_VF_MAX_NUM 256
+#define SXE2_VSI_MAX_NUM 768
+#define SXE2_FRAME_SIZE_MAX 9832
+#define SXE2_VLAN_TAG_SIZE 4
+#define SXE2_ETH_OVERHEAD \
+ (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + SXE2_VLAN_TAG_SIZE)
+#define SXE2_ETH_MAX_LEN (RTE_ETHER_MTU + SXE2_ETH_OVERHEAD)
+
+#ifdef SXE2_TEST
+#define SXE2_RESET_ACTIVE_WAIT_COUNT (5)
+#else
+#define SXE2_RESET_ACTIVE_WAIT_COUNT (10000)
+#endif
+#define SXE2_NO_ACTIVE_CNT (10)
+
+#define SXE2_WOKER_DELAY_5MS (5)
+#define SXE2_WOKER_DELAY_10MS (10)
+#define SXE2_WOKER_DELAY_20MS (20)
+#define SXE2_WOKER_DELAY_30MS (30)
+
+#define SXE2_RESET_DETEC_WAIT_COUNT (100)
+#define SXE2_RESET_DONE_WAIT_COUNT (250)
+#define SXE2_RESET_WAIT_MS (10)
+
+#define SXE2_RESET_WAIT_MIN (10)
+#define SXE2_RESET_WAIT_MAX (20)
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)((n) & 0xffffffff))
+
+#define SXE2_I2C_EEPROM_DEV_ADDR 0xA0
+#define SXE2_I2C_EEPROM_DEV_ADDR2 0xA2
+#define SXE2_MODULE_TYPE_SFP 0x03
+#define SXE2_MODULE_TYPE_QSFP_PLUS 0x0D
+#define SXE2_MODULE_TYPE_QSFP28 0x11
+#define SXE2_MODULE_SFF_ADDR_MODE 0x04
+#define SXE2_MODULE_SFF_DIAG_CAPAB 0x40
+#define SXE2_MODULE_REVISION_ADDR 0x01
+#define SXE2_MODULE_SFF_8472_COMP 0x5E
+#define SXE2_MODULE_SFF_8472_SWAP 0x5C
+#define SXE2_MODULE_QSFP_MAX_LEN 640
+#define SXE2_MODULE_SFF_8472_UNSUP 0x0
+#define SXE2_MODULE_SFF_DDM_IMPLEMENTED 0x40
+#define SXE2_MODULE_SFF_SFP_TYPE 0x03
+#define SXE2_MODULE_TYPE_QSFP_PLUS 0x0D
+#define SXE2_MODULE_TYPE_QSFP28 0x11
+
+#define SXE2_MODULE_SFF_8079 0x1
+#define SXE2_MODULE_SFF_8079_LEN 256
+#define SXE2_MODULE_SFF_8472 0x2
+#define SXE2_MODULE_SFF_8472_LEN 512
+#define SXE2_MODULE_SFF_8636 0x3
+#define SXE2_MODULE_SFF_8636_LEN 256
+#define SXE2_MODULE_SFF_8636_MAX_LEN 640
+#define SXE2_MODULE_SFF_8436 0x4
+#define SXE2_MODULE_SFF_8436_LEN 256
+#define SXE2_MODULE_SFF_8436_MAX_LEN 640
+
+enum sxe2_wk_type {
+ SXE2_WK_MONITOR,
+ SXE2_WK_MONITOR_IM,
+ SXE2_WK_POST,
+ SXE2_WK_MBX,
+};
+
+enum {
+ SXE2_FLAG_LEGACY_RX_ENABLE = 0,
+ SXE2_FLAG_LRO_ENABLE = 1,
+ SXE2_FLAG_RXQ_DISABLED = 2,
+ SXE2_FLAG_TXQ_DISABLED = 3,
+ SXE2_FLAG_DRV_REMOVING = 4,
+ SXE2_FLAG_RESET_DETECTED = 5,
+ SXE2_FLAG_CORE_RESET_DONE = 6,
+ SXE2_FLAG_RESET_ACTIVED = 7,
+ SXE2_FLAG_RESET_PENDING = 8,
+ SXE2_FLAG_RESET_REQUEST = 9,
+ SXE2_FLAGS_RESET_PROCESS_DONE = 10,
+ SXE2_FLAG_RESET_FAILED = 11,
+ SXE2_FLAG_DRV_PROBE_DONE = 12,
+ SXE2_FLAG_NETDEV_REGISTED = 13,
+ SXE2_FLAG_DRV_UP = 15,
+ SXE2_FLAG_DCB_ENABLE = 16,
+ SXE2_FLAG_FLTR_SYNC = 17,
+
+ SXE2_FLAG_EVENT_IRQ_DISABLED = 18,
+ SXE2_FLAG_SUSPEND = 19,
+ SXE2_FLAG_FNAV_ENABLE = 20,
+
+ SXE2_FLAGS_NBITS
+};
+
+struct sxe2_link_context {
+ rte_spinlock_t link_lock;
+ bool link_up;
+ u32 speed;
+};
+
+struct sxe2_devargs {
+ u8 flow_dup_pattern_mode;
+ u8 func_flow_direct_en;
+ u8 fnav_stat_type;
+ u8 high_performance_mode;
+ u8 sched_layer_mode;
+ u8 sw_stats_en;
+ u8 rx_low_latency;
+};
+
+#define SXE2_PCI_MAP_BAR_INVALID ((u8)0xff)
+#define SXE2_PCI_MAP_INVALID_VAL ((u32)0xffffffff)
+
+enum sxe2_pci_map_resource {
+ SXE2_PCI_MAP_RES_INVALID = 0,
+ SXE2_PCI_MAP_RES_DOORBELL_TX,
+ SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL,
+ SXE2_PCI_MAP_RES_IRQ_DYN,
+ SXE2_PCI_MAP_RES_IRQ_ITR,
+ SXE2_PCI_MAP_RES_IRQ_MSIX,
+ SXE2_PCI_MAP_RES_PTP,
+ SXE2_PCI_MAP_RES_MAX_COUNT,
+};
+
+enum sxe2_udp_tunnel_protocol {
+ SXE2_UDP_TUNNEL_PROTOCOL_VXLAN = 0,
+ SXE2_UDP_TUNNEL_PROTOCOL_VXLAN_GPE,
+ SXE2_UDP_TUNNEL_PROTOCOL_GENEVE,
+ SXE2_UDP_TUNNEL_PROTOCOL_GTP_C = 4,
+ SXE2_UDP_TUNNEL_PROTOCOL_GTP_U,
+ SXE2_UDP_TUNNEL_PROTOCOL_PFCP,
+ SXE2_UDP_TUNNEL_PROTOCOL_ECPRI,
+ SXE2_UDP_TUNNEL_PROTOCOL_MPLS,
+ SXE2_UDP_TUNNEL_PROTOCOL_NVGRE = 10,
+ SXE2_UDP_TUNNEL_PROTOCOL_L2TP,
+ SXE2_UDP_TUNNEL_PROTOCOL_TEREDO,
+ SXE2_UDP_TUNNEL_MAX,
+};
+
+struct sxe2_pci_map_addr_info {
+ u64 addr_base;
+ u8 bar_idx;
+ u8 reg_width;
+};
+
+struct sxe2_pci_map_segment_info {
+ enum sxe2_pci_map_resource type;
+ void __iomem *addr;
+ resource_size_t page_inner_offset;
+ resource_size_t len;
+};
+
+struct sxe2_pci_map_bar_info {
+ u8 bar_idx;
+ u8 map_cnt;
+ struct sxe2_pci_map_segment_info *seg_info;
+};
+
+struct sxe2_pci_map_context {
+ u8 bar_cnt;
+ struct sxe2_pci_map_bar_info *bar_info;
+ struct sxe2_pci_map_addr_info *addr_info;
+};
+
+struct sxe2_dev_mac_info {
+ u8 perm_addr[ETH_ALEN];
+};
+
+struct sxe2_pci_info {
+ u64 serial_number;
+ u8 bus_devid;
+ u8 bus_function;
+ u16 max_vfs;
+};
+
+struct sxe2_fw_info {
+ u8 main_version_id;
+ u8 sub_version_id;
+ u8 fix_version_id;
+ u8 build_id;
+};
+
+struct sxe2_dev_info {
+ struct rte_eth_dev_data *dev_data;
+ struct sxe2_pci_info pci;
+ struct sxe2_fw_info fw;
+ struct sxe2_dev_mac_info mac;
+};
+
+enum sxe2_udp_tunnel_status {
+ SXE2_UDP_TUNNEL_DISABLE = 0x0,
+ SXE2_UDP_TUNNEL_ENABLE,
+};
+
+struct sxe2_udp_tunnel_cfg {
+ u8 protocol;
+ u8 dev_status;
+ u16 dev_port;
+ u16 dev_ref_cnt;
+
+ u16 fw_port;
+ u8 fw_status;
+ u8 fw_dst_en;
+ u8 fw_src_en;
+ u8 fw_used;
+};
+
+struct sxe2_udp_tunnel_ctx {
+ struct sxe2_udp_tunnel_cfg tunnel_conf[SXE2_UDP_TUNNEL_MAX];
+ rte_spinlock_t lock;
+};
+
+struct sxe2_repr_context {
+ u16 nb_vf;
+ u16 nb_repr_vf;
+ struct rte_eth_dev **vf_rep_eth_dev;
+ struct sxe2_drv_vsi_caps repr_vf_id[SXE2_VF_MAX_NUM];
+};
+
+struct sxe2_repr_private_data {
+ struct rte_eth_dev *rep_eth_dev;
+ struct sxe2_adapter *parent_adapter;
+
+ struct sxe2_vsi *cp_vsi;
+ u16 repr_q_id;
+
+ u16 repr_id;
+ u16 repr_pf_id;
+ u16 repr_vf_id;
+ u16 repr_vf_vsi_id;
+ u16 repr_vf_k_vsi_id;
+ u16 repr_vf_u_vsi_id;
+};
+
+struct sxe2_sched_hw_cap {
+ u32 tm_layers;
+ u8 root_max_children;
+ u8 prio_max;
+ u8 adj_lvl;
+};
+
+struct sxe2_adapter {
+ struct sxe2_common_device *cdev;
+ struct sxe2_dev_info dev_info;
+ struct rte_pci_device *pci_dev;
+ struct sxe2_repr_private_data *repr_priv_data;
+ struct sxe2_pci_map_context map_ctxt;
+ struct sxe2_irq_context irq_ctxt;
+ struct sxe2_queue_context q_ctxt;
+ struct sxe2_vsi_context vsi_ctxt;
+ struct sxe2_devargs devargs;
+ u16 dev_port_id;
+ u64 cap_flags;
+ enum sxe2_dev_type dev_type;
+ u32 ptype_tbl[SXE2_MAX_PTYPE_NUM];
+ struct rte_ether_addr mac_addr;
+ u8 port_idx;
+ u8 pf_idx;
+ u32 tx_mode_flags;
+ u32 rx_mode_flags;
+ u8 started;
+};
+
+#define SXE2_DEV_PRIVATE_TO_ADAPTER(dev) \
+ ((struct sxe2_adapter *)(dev)->data->dev_private)
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_irq.h b/drivers/net/sxe2/sxe2_irq.h
new file mode 100644
index 0000000000..7695a0206f
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_irq.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_IRQ_H__
+#define __SXE2_IRQ_H__
+
+#include <ethdev_driver.h>
+
+#include "sxe2_type.h"
+#include "sxe2_drv_cmd.h"
+
+#define SXE2_IRQ_MAX_CNT 2048
+
+#define SXE2_LAN_MSIX_MIN_CNT 1
+
+#define SXE2_EVENT_IRQ_IDX 0
+
+#define SXE2_MAX_INTR_QUEUE_NUM 256
+
+#define SXE2_IRQ_NAME_MAX_LEN (IFNAMSIZ + 16)
+
+#define SXE2_ITR_1000K 1
+#define SXE2_ITR_500K 2
+#define SXE2_ITR_50K 20
+
+#define SXE2_ITR_INTERVAL_NORMAL (SXE2_ITR_50K)
+#define SXE2_ITR_INTERVAL_LOW (SXE2_ITR_1000K)
+
+struct sxe2_fwc_msix_caps;
+struct sxe2_adapter;
+
+struct sxe2_irq_context {
+ struct rte_intr_handle *reset_handle;
+ s32 reset_event_fd;
+ s32 other_event_fd;
+
+ u16 max_cnt_hw;
+ u16 base_idx_in_func;
+
+ u16 rxq_avail_cnt;
+ u16 rxq_base_idx_in_pf;
+
+ u16 rxq_irq_cnt;
+ u32 *rxq_msix_idx;
+ s32 *rxq_event_fd;
+};
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_queue.c b/drivers/net/sxe2/sxe2_queue.c
new file mode 100644
index 0000000000..98343679f6
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_queue.c
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include "sxe2_ethdev.h"
+#include "sxe2_queue.h"
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+
+void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
+ struct sxe2_drv_queue_caps *q_caps)
+{
+ adapter->q_ctxt.qp_cnt_assign = q_caps->queues_cnt;
+ adapter->q_ctxt.base_idx_in_pf = q_caps->base_idx_in_pf;
+}
+
+s32 sxe2_queues_init(struct rte_eth_dev *dev)
+{
+ s32 ret = SXE2_SUCCESS;
+ u16 buf_size;
+ u16 frame_size;
+ struct sxe2_rx_queue *rxq;
+ u16 nb_rxq;
+
+ frame_size = dev->data->mtu + SXE2_ETH_OVERHEAD;
+ for (nb_rxq = 0; nb_rxq < dev->data->nb_rx_queues; nb_rxq++) {
+ rxq = dev->data->rx_queues[nb_rxq];
+ if (!rxq)
+ continue;
+
+ buf_size = rte_pktmbuf_data_room_size(rxq->mb_pool) - RTE_PKTMBUF_HEADROOM;
+ rxq->rx_buf_len = RTE_ALIGN_FLOOR(buf_size, (1 << SXE2_RXQ_CTX_DBUFF_SHIFT));
+ rxq->rx_buf_len = RTE_MIN(rxq->rx_buf_len, SXE2_RX_MAX_DATA_BUF_SIZE);
+ if (frame_size > rxq->rx_buf_len)
+ dev->data->scattered_rx = 1;
+ }
+
+ return ret;
+}
diff --git a/drivers/net/sxe2/sxe2_queue.h b/drivers/net/sxe2/sxe2_queue.h
new file mode 100644
index 0000000000..7fa22e2820
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_queue.h
@@ -0,0 +1,191 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_QUEUE_H__
+#define __SXE2_QUEUE_H__
+#include <rte_ethdev.h>
+#include <rte_io.h>
+#include <rte_stdatomic.h>
+#include <ethdev_driver.h>
+
+#include "sxe2_drv_cmd.h"
+#include "sxe2_txrx_common.h"
+
+#define SXE2_PCI_REG_READ(reg) \
+ rte_read32(reg)
+#define SXE2_PCI_REG_WRITE_WC(reg, value) \
+ rte_write32_wc((rte_cpu_to_le_32(value)), reg)
+#define SXE2_PCI_REG_WRITE_WC_RELAXED(reg, value) \
+ rte_write32_wc_relaxed((rte_cpu_to_le_32(value)), reg)
+
+struct sxe2_queue_context {
+ u16 qp_cnt_assign;
+ u16 base_idx_in_pf;
+
+ u32 tx_mode_flags;
+ u32 rx_mode_flags;
+};
+
+struct sxe2_tx_buffer {
+ struct rte_mbuf *mbuf;
+
+ u16 next_id;
+ u16 last_id;
+};
+
+struct sxe2_tx_buffer_vec {
+ struct rte_mbuf *mbuf;
+};
+
+struct sxe2_txq_stats {
+ u64 tx_restart;
+ u64 tx_busy;
+
+ u64 tx_linearize;
+ u64 tx_tso_linearize_chk;
+ u64 tx_vlan_insert;
+ u64 tx_tso_packets;
+ u64 tx_tso_bytes;
+ u64 tx_csum_none;
+ u64 tx_csum_partial;
+ u64 tx_csum_partial_inner;
+ u64 tx_queue_dropped;
+ u64 tx_xmit_more;
+ u64 tx_pkts_num;
+ u64 tx_desc_not_done;
+};
+
+struct sxe2_tx_queue;
+struct sxe2_txq_ops {
+ void (*queue_reset)(struct sxe2_tx_queue *txq);
+ void (*mbufs_release)(struct sxe2_tx_queue *txq);
+ void (*buffer_ring_free)(struct sxe2_tx_queue *txq);
+};
+struct sxe2_tx_queue {
+ volatile union sxe2_tx_data_desc *desc_ring;
+ struct sxe2_tx_buffer *buffer_ring;
+ volatile u32 *tdt_reg_addr;
+
+ u64 offloads;
+ u16 ring_depth;
+ u16 desc_free_num;
+
+ u16 free_thresh;
+
+ u16 rs_thresh;
+ u16 next_use;
+ u16 next_clean;
+
+ u16 desc_used_num;
+ u16 next_dd;
+ u16 next_rs;
+ u16 ipsec_pkt_md_offset;
+
+ u16 port_id;
+ u16 queue_id;
+ u16 idx_in_func;
+ bool tx_deferred_start;
+ u8 pthresh;
+ u8 hthresh;
+ u8 wthresh;
+ u16 reg_idx;
+ u64 base_addr;
+ struct sxe2_vsi *vsi;
+ const struct rte_memzone *mz;
+ struct sxe2_txq_ops ops;
+ u8 vlan_flag;
+ u8 use_ctx:1,
+ res:7;
+};
+struct sxe2_rx_queue;
+struct sxe2_rxq_ops {
+ void (*queue_reset)(struct sxe2_rx_queue *rxq);
+ void (*mbufs_release)(struct sxe2_rx_queue *txq);
+};
+struct sxe2_rxq_stats {
+ u64 rx_pkts_num;
+ u64 rx_rss_pkt_num;
+ u64 rx_fnav_pkt_num;
+ u64 rx_ptp_pkt_num;
+ u32 rx_vec_align_drop;
+
+ u32 rxdid_1588_err;
+ u32 ip_csum_err;
+ u32 l4_csum_err;
+ u32 outer_ip_csum_err;
+ u32 outer_l4_csum_err;
+ u32 macsec_err;
+ u32 ipsec_err;
+
+ u64 ptype_pkts[SXE2_MAX_PTYPE_NUM];
+};
+
+struct sxe2_rxq_sw_stats {
+ RTE_ATOMIC(uint64_t)pkts;
+ RTE_ATOMIC(uint64_t)bytes;
+ RTE_ATOMIC(uint64_t)drop_pkts;
+ RTE_ATOMIC(uint64_t)drop_bytes;
+ RTE_ATOMIC(uint64_t)unicast_pkts;
+ RTE_ATOMIC(uint64_t)multicast_pkts;
+ RTE_ATOMIC(uint64_t)broadcast_pkts;
+};
+
+struct sxe2_rx_queue {
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile u32 *rdt_reg_addr;
+ struct rte_mempool *mb_pool;
+ struct rte_mbuf **buffer_ring;
+ struct sxe2_vsi *vsi;
+
+ u64 offloads;
+ u16 ring_depth;
+ u16 rx_free_thresh;
+ u16 processing_idx;
+ u16 hold_num;
+ u16 next_ret_pkt;
+ u16 batch_alloc_trigger;
+ u16 completed_pkts_num;
+ u64 update_time;
+ u32 desc_ts;
+ u64 ts_high;
+ u32 ts_low;
+ u32 ts_need_update;
+ u8 crc_len;
+ bool fnav_enable;
+
+ struct rte_eth_rxseg_split rx_seg[SXE2_RX_SEG_NUM];
+
+ struct rte_mbuf *completed_buf[SXE2_RX_PKTS_BURST_BATCH_NUM * 2];
+ struct rte_mbuf *pkt_first_seg;
+ struct rte_mbuf *pkt_last_seg;
+ u64 mbuf_init_value;
+ u16 realloc_num;
+ u16 realloc_start;
+ struct rte_mbuf fake_mbuf;
+
+ const struct rte_memzone *mz;
+ struct sxe2_rxq_ops ops;
+ rte_iova_t base_addr;
+ u16 reg_idx;
+ u32 low_desc_waterline : 16;
+ u32 ldw_event_pending : 1;
+ struct sxe2_rxq_sw_stats sw_stats;
+ u16 port_id;
+ u16 queue_id;
+ u16 idx_in_func;
+ u16 rx_buf_len;
+ u16 rx_hdr_len;
+ u16 max_pkt_len;
+ bool rx_deferred_start;
+ u8 drop_en;
+};
+
+struct sxe2_adapter;
+
+void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
+ struct sxe2_drv_queue_caps *q_caps);
+
+s32 sxe2_queues_init(struct rte_eth_dev *dev);
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_common.h b/drivers/net/sxe2/sxe2_txrx_common.h
new file mode 100644
index 0000000000..7284cea4b6
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_common.h
@@ -0,0 +1,541 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef _SXE2_TXRX_COMMON_H_
+#define _SXE2_TXRX_COMMON_H_
+#include <stdbool.h>
+#include "sxe2_type.h"
+
+#define SXE2_ALIGN_RING_DESC 32
+#define SXE2_MIN_RING_DESC 64
+#define SXE2_MAX_RING_DESC 4096
+
+#define SXE2_VECTOR_PATH 0
+#define SXE2_VECTOR_OFFLOAD_PATH 1
+#define SXE2_VECTOR_CTX_OFFLOAD_PATH 2
+
+#define SXE2_MAX_PTYPE_NUM 1024
+#define SXE2_MIN_BUF_SIZE 1024
+
+#define SXE2_ALIGN 32
+#define SXE2_DESC_ADDR_ALIGN 128
+
+#define SXE2_MIN_TSO_MSS 88
+#define SXE2_MAX_TSO_MSS 9728
+
+#define SXE2_TX_MTU_SEG_MAX 15
+
+#define SXE2_TX_MIN_PKT_LEN 17
+#define SXE2_TX_MAX_BURST 32
+#define SXE2_TX_MAX_FREE_BUF 64
+#define SXE2_TX_TSO_PKTLEN_MAX (256ULL * 1024)
+
+#define DEFAULT_TX_RS_THRESH 32
+#define DEFAULT_TX_FREE_THRESH 32
+
+#define SXE2_TX_FLAGS_VLAN_TAG_LOC_L2TAG1 BIT(0)
+#define SXE2_TX_FLAGS_VLAN_TAG_LOC_L2TAG2 BIT(1)
+
+#define SXE2_TX_PKTS_BURST_BATCH_NUM 32
+
+union sxe2_tx_offload_info {
+ u64 data;
+ struct {
+ u64 l2_len:7;
+ u64 l3_len:9;
+ u64 l4_len:8;
+ u64 tso_segsz:16;
+ u64 outer_l2_len:8;
+ u64 outer_l3_len:16;
+ };
+};
+
+#define SXE2_TX_OFFLOAD_CTXT_NEEDCK_MASK (RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_MBUF_F_TX_UDP_SEG | \
+ RTE_MBUF_F_TX_QINQ | \
+ RTE_MBUF_F_TX_OUTER_IP_CKSUM | \
+ RTE_MBUF_F_TX_OUTER_UDP_CKSUM | \
+ RTE_MBUF_F_TX_SEC_OFFLOAD | \
+ RTE_MBUF_F_TX_IEEE1588_TMST)
+
+#define SXE2_TX_OFFLOAD_CKSUM_MASK (RTE_MBUF_F_TX_IP_CKSUM | \
+ RTE_MBUF_F_TX_L4_MASK | \
+ RTE_MBUF_F_TX_TCP_SEG | \
+ RTE_MBUF_F_TX_UDP_SEG | \
+ RTE_MBUF_F_TX_OUTER_UDP_CKSUM | \
+ RTE_MBUF_F_TX_OUTER_IP_CKSUM)
+
+struct sxe2_tx_context_desc {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 ipsec_offset;
+ __le64 type_cmd_tso_mss;
+};
+
+#define SXE2_TX_CTXT_DESC_EIPLEN_SHIFT 2
+#define SXE2_TX_CTXT_DESC_L4TUNT_SHIFT 9
+#define SXE2_TX_CTXT_DESC_NATLEN_SHIFT 12
+#define SXE2_TX_CTXT_DESC_L4T_CS_SHIFT 23
+
+#define SXE2_TX_CTXT_DESC_CMD_SHIFT 4
+#define SXE2_TX_CTXT_DESC_IPSEC_MODE_SHIFT 11
+#define SXE2_TX_CTXT_DESC_IPSEC_EN_SHIFT 12
+#define SXE2_TX_CTXT_DESC_IPSEC_ENGINE_SHIFT 13
+#define SXE2_TX_CTXT_DESC_IPSEC_SA_SHIFT 16
+#define SXE2_TX_CTXT_DESC_TSO_LEN_SHIFT 30
+#define SXE2_TX_CTXT_DESC_MSS_SHIFT 50
+#define SXE2_TX_CTXT_DESC_VSI_SHIFT 50
+
+#define SXE2_TX_CTXT_DESC_L4T_CS_MASK RTE_BIT64(SXE2_TX_CTXT_DESC_L4T_CS_SHIFT)
+
+#define SXE2_TX_CTXT_DESC_EIPLEN_VAL(val) \
+ (((val) >> 2) << SXE2_TX_CTXT_DESC_EIPLEN_SHIFT)
+#define SXE2_TX_CTXT_DESC_NATLEN_VAL(val) \
+ (((val) >> 1) << SXE2_TX_CTXT_DESC_NATLEN_SHIFT)
+
+enum sxe2_tx_ctxt_desc_eipt_bits {
+ SXE2_TX_CTXT_DESC_EIPT_NONE = 0x0,
+ SXE2_TX_CTXT_DESC_EIPT_IPV6 = 0x1,
+ SXE2_TX_CTXT_DESC_EIPT_IPV4_NO_CSUM = 0x2,
+ SXE2_TX_CTXT_DESC_EIPT_IPV4 = 0x3,
+};
+
+enum sxe2_tx_ctxt_desc_l4tunt_bits {
+ SXE2_TX_CTXT_DESC_UDP_TUNNE = 0x1 << SXE2_TX_CTXT_DESC_L4TUNT_SHIFT,
+ SXE2_TX_CTXT_DESC_GRE_TUNNE = 0x2 << SXE2_TX_CTXT_DESC_L4TUNT_SHIFT,
+};
+
+enum sxe2_tx_ctxt_desc_cmd_bits {
+ SXE2_TX_CTXT_DESC_CMD_TSO = 0x01,
+ SXE2_TX_CTXT_DESC_CMD_TSYN = 0x02,
+ SXE2_TX_CTXT_DESC_CMD_IL2TAG2 = 0x04,
+ SXE2_TX_CTXT_DESC_CMD_IL2TAG2_IL2H = 0x08,
+ SXE2_TX_CTXT_DESC_CMD_SWTCH_NOTAG = 0x00,
+ SXE2_TX_CTXT_DESC_CMD_SWTCH_UPLINK = 0x10,
+ SXE2_TX_CTXT_DESC_CMD_SWTCH_LOCAL = 0x20,
+ SXE2_TX_CTXT_DESC_CMD_SWTCH_VSI = 0x30,
+ SXE2_TX_CTXT_DESC_CMD_RESERVED = 0x40
+};
+#define SXE2_TX_CTXT_DESC_IPSEC_MODE RTE_BIT64(SXE2_TX_CTXT_DESC_IPSEC_MODE_SHIFT)
+#define SXE2_TX_CTXT_DESC_IPSEC_EN RTE_BIT64(SXE2_TX_CTXT_DESC_IPSEC_EN_SHIFT)
+#define SXE2_TX_CTXT_DESC_IPSEC_ENGINE RTE_BIT64(SXE2_TX_CTXT_DESC_IPSEC_ENGINE_SHIFT)
+#define SXE2_TX_CTXT_DESC_CMD_TSYN_MASK \
+ (((u64)SXE2_TX_CTXT_DESC_CMD_TSYN) << SXE2_TX_CTXT_DESC_CMD_SHIFT)
+#define SXE2_TX_CTXT_DESC_CMD_IL2TAG2_MASK \
+ (((u64)SXE2_TX_CTXT_DESC_CMD_IL2TAG2) << SXE2_TX_CTXT_DESC_CMD_SHIFT)
+
+union sxe2_tx_data_desc {
+ struct {
+ __le64 buf_addr;
+ __le64 type_cmd_off_bsz_l2t;
+ } read;
+ struct {
+ __le64 rsvd;
+ __le64 dd;
+ } wb;
+};
+
+#define SXE2_TX_DATA_DESC_CMD_SHIFT 4
+#define SXE2_TX_DATA_DESC_OFFSET_SHIFT 16
+#define SXE2_TX_DATA_DESC_BUF_SZ_SHIFT 34
+#define SXE2_TX_DATA_DESC_L2TAG1_SHIFT 48
+
+#define SXE2_TX_DATA_DESC_CMD_MASK \
+ (0xFFFULL << SXE2_TX_DATA_DESC_CMD_SHIFT)
+#define SXE2_TX_DATA_DESC_OFFSET_MASK \
+ (0x3FFFFULL << SXE2_TX_DATA_DESC_OFFSET_SHIFT)
+#define SXE2_TX_DATA_DESC_BUF_SZ_MASK \
+ (0x3FFFULL << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT)
+#define SXE2_TX_DATA_DESC_L2TAG1_MASK \
+ (0xFFFFULL << SXE2_TX_DATA_DESC_L2TAG1_SHIFT)
+
+#define SXE2_TX_DESC_LENGTH_MACLEN_SHIFT (0)
+#define SXE2_TX_DESC_LENGTH_IPLEN_SHIFT (7)
+#define SXE2_TX_DESC_LENGTH_L4_FC_LEN_SHIFT (14)
+
+#define SXE2_TX_DESC_DTYPE_MASK 0xF
+#define SXE2_TX_DATA_DESC_MACLEN_MASK \
+ (0x7FULL << SXE2_TX_DESC_LENGTH_MACLEN_SHIFT)
+#define SXE2_TX_DATA_DESC_IPLEN_MASK \
+ (0x7FULL << SXE2_TX_DESC_LENGTH_IPLEN_SHIFT)
+#define SXE2_TX_DATA_DESC_L4LEN_MASK \
+ (0xFULL << SXE2_TX_DESC_LENGTH_L4_FC_LEN_SHIFT)
+
+#define SXE2_TX_DATA_DESC_MACLEN_VAL(val) \
+ (((val) >> 1) << SXE2_TX_DESC_LENGTH_MACLEN_SHIFT)
+#define SXE2_TX_DATA_DESC_IPLEN_VAL(val) \
+ (((val) >> 2) << SXE2_TX_DESC_LENGTH_IPLEN_SHIFT)
+#define SXE2_TX_DATA_DESC_L4LEN_VAL(val) \
+ (((val) >> 2) << SXE2_TX_DESC_LENGTH_L4_FC_LEN_SHIFT)
+
+enum sxe2_tx_desc_type {
+ SXE2_TX_DESC_DTYPE_DATA = 0x0,
+ SXE2_TX_DESC_DTYPE_CTXT = 0x1,
+ SXE2_TX_DESC_DTYPE_FLTR_PROG = 0x8,
+ SXE2_TX_DESC_DTYPE_DESC_DONE = 0xF,
+};
+
+enum sxe2_tx_data_desc_cmd_bits {
+ SXE2_TX_DATA_DESC_CMD_EOP = 0x0001,
+ SXE2_TX_DATA_DESC_CMD_RS = 0x0002,
+ SXE2_TX_DATA_DESC_CMD_MACSEC = 0x0004,
+ SXE2_TX_DATA_DESC_CMD_IL2TAG1 = 0x0008,
+ SXE2_TX_DATA_DESC_CMD_DUMMY = 0x0010,
+ SXE2_TX_DATA_DESC_CMD_IIPT_IPV6 = 0x0020,
+ SXE2_TX_DATA_DESC_CMD_IIPT_IPV4 = 0x0040,
+ SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM = 0x0060,
+ SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP = 0x0100,
+ SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP = 0x0200,
+ SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP = 0x0300,
+ SXE2_TX_DATA_DESC_CMD_RE = 0x0400
+};
+#define SXE2_TX_DATA_DESC_CMD_RS_MASK \
+ (((u64)SXE2_TX_DATA_DESC_CMD_RS) << SXE2_TX_DATA_DESC_CMD_SHIFT)
+
+#define SXE2_TX_MAX_DATA_NUM_PER_DESC 0X3FFFUL
+
+#define SXE2_TX_DESC_RING_ALIGN \
+ (SXE2_ALIGN_RING_DESC / sizeof(union sxe2_tx_data_desc))
+
+#define SXE2_TX_DESC_DTYPE_DESC_MASK 0xF
+
+#define SXE2_TX_FILL_PER_LOOP 4
+#define SXE2_TX_FILL_PER_LOOP_MASK (SXE2_TX_FILL_PER_LOOP - 1)
+#define SXE2_TX_FREE_BUFFER_SIZE_MAX (64)
+
+#define SXE2_RX_MAX_BURST 32
+#define SXE2_RING_SIZE_MIN 1024
+#define SXE2_RX_MAX_NSEG 2
+
+#define SXE2_RX_PKTS_BURST_BATCH_NUM SXE2_RX_MAX_BURST
+#define SXE2_VPMD_RX_MAX_BURST SXE2_RX_MAX_BURST
+
+#define SXE2_RXQ_CTX_DBUFF_SHIFT 7
+
+#define SXE2_RX_NUM_PER_LOOP 8
+
+#define SXE2_RX_FLEX_DESC_PTYPE_S (16)
+#define SXE2_RX_FLEX_DESC_PTYPE_M (0x3FFULL)
+
+#define SXE2_RX_HBUF_LEN_UNIT 6
+#define SXE2_RX_LDW_LEN_UNIT 6
+#define SXE2_RX_DBUF_LEN_UNIT 7
+#define SXE2_RX_DBUF_LEN_MASK (~0x7F)
+
+#define SXE2_RX_PKTS_TS_TIMEOUT_VAL 200
+
+#define SXE2_RX_VECTOR_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_CHECKSUM | \
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_VLAN | \
+ RTE_ETH_RX_OFFLOAD_RSS_HASH | \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP)
+
+#define SXE2_DEFAULT_RX_FREE_THRESH 32
+#define SXE2_DEFAULT_RX_PTHRESH 8
+#define SXE2_DEFAULT_RX_HTHRESH 8
+#define SXE2_DEFAULT_RX_WTHRESH 0
+
+#define SXE2_DEFAULT_TX_FREE_THRESH 32
+#define SXE2_DEFAULT_TX_PTHRESH 32
+#define SXE2_DEFAULT_TX_HTHRESH 0
+#define SXE2_DEFAULT_TX_WTHRESH 0
+#define SXE2_DEFAULT_TX_RSBIT_THRESH 32
+
+#define SXE2_RX_SEG_NUM 2
+
+#ifdef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+#define sxe2_rx_desc sxe2_rx_16b_desc
+#else
+#define sxe2_rx_desc sxe2_rx_32b_desc
+#endif
+
+union sxe2_rx_16b_desc {
+ struct {
+ __le64 pkt_addr;
+ __le64 hdr_addr;
+ } read;
+ struct {
+ u8 rxdid_src;
+ u8 mirror;
+ __le16 l2tag1;
+ __le32 filter_status;
+
+ __le64 status_err_ptype_len;
+ } wb;
+};
+
+union sxe2_rx_32b_desc {
+ struct {
+ __le64 pkt_addr;
+ __le64 hdr_addr;
+ __le64 rsvd1;
+ __le64 rsvd2;
+ } read;
+ struct {
+ u8 rxdid_src;
+ u8 mirror;
+ __le16 l2tag1;
+ __le32 filter_status;
+
+ __le64 status_err_ptype_len;
+
+ __le32 status_lrocnt_fdpf_id;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ u8 acl_pf_id;
+ u8 sw_pf_id;
+ __le16 flow_id;
+
+ __le32 fd_filter_id;
+
+ } wb;
+ struct {
+ u8 rxdid_src_fd_eudpe;
+ u8 mirror;
+ __le16 l2_tag1;
+ __le32 filter_status;
+
+ __le64 status_err_ptype_len;
+
+ __le32 ext_status_ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ __le32 ts_h;
+ __le32 fd_filter_id;
+
+ } wb_ts;
+};
+
+enum sxe2_rx_lro_desc_max_num {
+ SXE2_RX_LRO_DESC_MAX_1 = 1,
+ SXE2_RX_LRO_DESC_MAX_4 = 4,
+ SXE2_RX_LRO_DESC_MAX_8 = 8,
+ SXE2_RX_LRO_DESC_MAX_16 = 16,
+ SXE2_RX_LRO_DESC_MAX_32 = 32,
+ SXE2_RX_LRO_DESC_MAX_48 = 48,
+ SXE2_RX_LRO_DESC_MAX_64 = 64,
+ SXE2_RX_LRO_DESC_MAX_NUM = SXE2_RX_LRO_DESC_MAX_64,
+};
+
+enum sxe2_rx_desc_rxdid {
+ SXE2_RX_DESC_RXDID_16B = 0,
+ SXE2_RX_DESC_RXDID_32B,
+ SXE2_RX_DESC_RXDID_1588,
+ SXE2_RX_DESC_RXDID_FD,
+};
+
+#define SXE2_RX_DESC_RXDID_SHIFT (0)
+#define SXE2_RX_DESC_RXDID_MASK (0x7 << SXE2_RX_DESC_RXDID_SHIFT)
+#define SXE2_RX_DESC_RXDID_VAL_GET(rxdid_src) \
+ (((rxdid_src) & SXE2_RX_DESC_RXDID_MASK) >> SXE2_RX_DESC_RXDID_SHIFT)
+
+#define SXE2_RX_DESC_PKT_SRC_SHIFT (3)
+#define SXE2_RX_DESC_PKT_SRC_MASK (0x3 << SXE2_RX_DESC_PKT_SRC_SHIFT)
+#define SXE2_RX_DESC_PKT_SRC_VAL_GET(rxdid_src) \
+ (((rxdid_src) & SXE2_RX_DESC_PKT_SRC_MASK) >> SXE2_RX_DESC_PKT_SRC_SHIFT)
+
+#define SXE2_RX_DESC_FD_VLD_SHIFT (5)
+#define SXE2_RX_DESC_FD_VLD_MASK (0x1 << SXE2_RX_DESC_FD_VLD_SHIFT)
+#define SXE2_RX_DESC_FD_VLD_VAL_GET(rxdid_src) \
+ (((rxdid_src) & SXE2_RX_DESC_FD_VLD_MASK) >> SXE2_RX_DESC_FD_VLD_SHIFT)
+
+#define SXE2_RX_DESC_EUDPE_SHIFT (6)
+#define SXE2_RX_DESC_EUDPE_MASK (0x1 << SXE2_RX_DESC_EUDPE_SHIFT)
+#define SXE2_RX_DESC_EUDPE_VAL_GET(rxdid_src) \
+ (((rxdid_src) & SXE2_RX_DESC_EUDPE_MASK) >> SXE2_RX_DESC_EUDPE_SHIFT)
+
+#define SXE2_RX_DESC_UDP_NET_SHIFT (7)
+#define SXE2_RX_DESC_UDP_NET_MASK (0x1 << SXE2_RX_DESC_UDP_NET_SHIFT)
+#define SXE2_RX_DESC_UDP_NET_VAL_GET(rxdid_src) \
+ (((rxdid_src) & SXE2_RX_DESC_UDP_NET_MASK) >> SXE2_RX_DESC_UDP_NET_SHIFT)
+
+#define SXE2_RX_DESC_MIRR_ID_SHIFT (0)
+#define SXE2_RX_DESC_MIRR_ID_MASK (0x3F << SXE2_RX_DESC_MIRR_ID_SHIFT)
+#define SXE2_RX_DESC_MIRR_ID_VAL_GET(mirr) \
+ (((mirr) & SXE2_RX_DESC_MIRR_ID_MASK) >> SXE2_RX_DESC_MIRR_ID_SHIFT)
+
+#define SXE2_RX_DESC_MIRR_TYPE_SHIFT (6)
+#define SXE2_RX_DESC_MIRR_TYPE_MASK (0x3 << SXE2_RX_DESC_MIRR_TYPE_SHIFT)
+#define SXE2_RX_DESC_MIRR_TYPE_VAL_GET(mirr) \
+ (((mirr) & SXE2_RX_DESC_MIRR_TYPE_MASK) >> SXE2_RX_DESC_MIRR_TYPE_SHIFT)
+
+#define SXE2_RX_DESC_PKT_LEN_SHIFT (32)
+#define SXE2_RX_DESC_PKT_LEN_MASK (0x3FFFULL << SXE2_RX_DESC_PKT_LEN_SHIFT)
+#define SXE2_RX_DESC_PKT_LEN_VAL_GET(qw1) \
+ (((qw1) & SXE2_RX_DESC_PKT_LEN_MASK) >> SXE2_RX_DESC_PKT_LEN_SHIFT)
+
+#define SXE2_RX_DESC_HDR_LEN_SHIFT (46)
+#define SXE2_RX_DESC_HDR_LEN_MASK (0x7FFULL << SXE2_RX_DESC_HDR_LEN_SHIFT)
+#define SXE2_RX_DESC_HDR_LEN_VAL_GET(qw1) \
+ (((qw1) & SXE2_RX_DESC_HDR_LEN_MASK) >> SXE2_RX_DESC_HDR_LEN_SHIFT)
+
+#define SXE2_RX_DESC_SPH_SHIFT (57)
+#define SXE2_RX_DESC_SPH_MASK (0x1ULL << SXE2_RX_DESC_SPH_SHIFT)
+#define SXE2_RX_DESC_SPH_VAL_GET(qw1) \
+ (((qw1) & SXE2_RX_DESC_SPH_MASK) >> SXE2_RX_DESC_SPH_SHIFT)
+
+#define SXE2_RX_DESC_PTYPE_SHIFT (16)
+#define SXE2_RX_DESC_PTYPE_MASK (0x3FFULL << SXE2_RX_DESC_PTYPE_SHIFT)
+#define SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT (0x3FFULL)
+#define SXE2_RX_DESC_PTYPE_VAL_GET(qw1) \
+ (((qw1) & SXE2_RX_DESC_PTYPE_MASK) >> SXE2_RX_DESC_PTYPE_SHIFT)
+
+#define SXE2_RX_DESC_FILTER_STATUS_SHIFT (32)
+#define SXE2_RX_DESC_FILTER_STATUS_MASK (0xFFFFUL)
+
+#define SXE2_RX_DESC_LROCNT_SHIFT (0)
+#define SXE2_RX_DESC_LROCNT_MASK (0xF)
+
+enum sxe2_rx_desc_status_shift {
+ SXE2_RX_DESC_STATUS_DD_SHIFT = 0,
+ SXE2_RX_DESC_STATUS_EOP_SHIFT = 1,
+ SXE2_RX_DESC_STATUS_L2TAG1_P_SHIFT = 2,
+
+ SXE2_RX_DESC_STATUS_L3L4_P_SHIFT = 3,
+ SXE2_RX_DESC_STATUS_CRCP_SHIFT = 4,
+ SXE2_RX_DESC_STATUS_SECP_SHIFT = 5,
+ SXE2_RX_DESC_STATUS_SECTAG_SHIFT = 6,
+ SXE2_RX_DESC_STATUS_SECE_SHIFT = 26,
+ SXE2_RX_DESC_STATUS_EXT_UDP_0_SHIFT = 27,
+ SXE2_RX_DESC_STATUS_UMBCAST_SHIFT = 28,
+ SXE2_RX_DESC_STATUS_PHY_PORT_SHIFT = 30,
+ SXE2_RX_DESC_STATUS_LPBK_SHIFT = 59,
+ SXE2_RX_DESC_STATUS_IPV6_EXADD_SHIFT = 60,
+ SXE2_RX_DESC_STATUS_RSS_VLD_SHIFT = 61,
+ SXE2_RX_DESC_STATUS_ACL_HIT_SHIFT = 62,
+ SXE2_RX_DESC_STATUS_INT_UDP_0_SHIFT = 63,
+};
+
+#define SXE2_RX_DESC_STATUS_DD_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_DD_SHIFT)
+#define SXE2_RX_DESC_STATUS_EOP_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_EOP_SHIFT)
+#define SXE2_RX_DESC_STATUS_L2TAG1_P_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_L2TAG1_P_SHIFT)
+#define SXE2_RX_DESC_STATUS_L3L4_P_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_L3L4_P_SHIFT)
+#define SXE2_RX_DESC_STATUS_CRCP_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_CRCP_SHIFT)
+#define SXE2_RX_DESC_STATUS_SECP_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_SECP_SHIFT)
+#define SXE2_RX_DESC_STATUS_SECTAG_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_SECTAG_SHIFT)
+#define SXE2_RX_DESC_STATUS_SECE_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_SECE_SHIFT)
+#define SXE2_RX_DESC_STATUS_EXT_UDP_0_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_EXT_UDP_0_SHIFT)
+#define SXE2_RX_DESC_STATUS_UMBCAST_MASK \
+ (0x3ULL << SXE2_RX_DESC_STATUS_UMBCAST_SHIFT)
+#define SXE2_RX_DESC_STATUS_PHY_PORT_MASK \
+ (0x3ULL << SXE2_RX_DESC_STATUS_PHY_PORT_SHIFT)
+#define SXE2_RX_DESC_STATUS_LPBK_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_LPBK_SHIFT)
+#define SXE2_RX_DESC_STATUS_IPV6_EXADD_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_IPV6_EXADD_SHIFT)
+#define SXE2_RX_DESC_STATUS_RSS_VLD_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_RSS_VLD_SHIFT)
+#define SXE2_RX_DESC_STATUS_ACL_HIT_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_ACL_HIT_SHIFT)
+#define SXE2_RX_DESC_STATUS_INT_UDP_0_MASK \
+ (0x1ULL << SXE2_RX_DESC_STATUS_INT_UDP_0_SHIFT)
+
+enum sxe2_rx_desc_umbcast_val {
+ SXE2_RX_DESC_STATUS_UNICAST = 0,
+ SXE2_RX_DESC_STATUS_MUTICAST = 1,
+ SXE2_RX_DESC_STATUS_BOARDCAST = 2,
+};
+
+#define SXE2_RX_DESC_STATUS_UMBCAST_VAL_GET(qw1) \
+ (((qw1) & SXE2_RX_DESC_STATUS_UMBCAST_MASK) >> SXE2_RX_DESC_STATUS_UMBCAST_SHIFT)
+
+enum sxe2_rx_desc_error_shift {
+ SXE2_RX_DESC_ERROR_RXE_SHIFT = 7,
+ SXE2_RX_DESC_ERROR_PKT_ECC_SHIFT = 8,
+ SXE2_RX_DESC_ERROR_PKT_HBO_SHIFT = 9,
+
+ SXE2_RX_DESC_ERROR_CSUM_IPE_SHIFT = 10,
+
+ SXE2_RX_DESC_ERROR_CSUM_L4_SHIFT = 11,
+
+ SXE2_RX_DESC_ERROR_CSUM_EIP_SHIFT = 12,
+ SXE2_RX_DESC_ERROR_OVERSIZE_SHIFT = 13,
+ SXE2_RX_DESC_ERROR_SEC_ERR_SHIFT = 14,
+};
+
+#define SXE2_RX_DESC_ERROR_RXE_MASK \
+ (0x1ULL << SXE2_RX_DESC_ERROR_RXE_SHIFT)
+#define SXE2_RX_DESC_ERROR_PKT_ECC_MASK \
+ (0x1ULL << SXE2_RX_DESC_ERROR_PKT_ECC_SHIFT)
+#define SXE2_RX_DESC_ERROR_PKT_HBO_MASK \
+ (0x1ULL << SXE2_RX_DESC_ERROR_PKT_HBO_SHIFT)
+#define SXE2_RX_DESC_ERROR_CSUM_IPE_MASK \
+ (0x1ULL << SXE2_RX_DESC_ERROR_CSUM_IPE_SHIFT)
+#define SXE2_RX_DESC_ERROR_CSUM_L4_MASK \
+ (0x1ULL << SXE2_RX_DESC_ERROR_CSUM_L4_SHIFT)
+#define SXE2_RX_DESC_ERROR_CSUM_EIP_MASK \
+ (0x1ULL << SXE2_RX_DESC_ERROR_CSUM_EIP_SHIFT)
+#define SXE2_RX_DESC_ERROR_OVERSIZE_MASK \
+ (0x1ULL << SXE2_RX_DESC_ERROR_OVERSIZE_SHIFT)
+
+#define SXE2_RX_DESC_QW1_ERRORS_MASK \
+ (SXE2_RX_DESC_ERROR_CSUM_IPE_MASK | \
+ SXE2_RX_DESC_ERROR_CSUM_L4_MASK | \
+ SXE2_RX_DESC_ERROR_CSUM_EIP_MASK)
+
+enum sxe2_rx_desc_ext_status_shift {
+ SXE2_RX_DESC_EXT_STATUS_L2TAG2P_SHIFT = 4,
+ SXE2_RX_DESC_EXT_STATUS_RSVD = 5,
+ SXE2_RX_DESC_EXT_STATUS_PKT_REE_SHIFT = 7,
+ SXE2_RX_DESC_EXT_STATUS_ROCE_SHIFT = 13,
+};
+#define SXE2_RX_DESC_EXT_STATUS_L2TAG2P_MASK \
+ (0x1ULL << SXE2_RX_DESC_EXT_STATUS_L2TAG2P_SHIFT)
+#define SXE2_RX_DESC_EXT_STATUS_PKT_REE_MASK \
+ (0x3FULL << SXE2_RX_DESC_EXT_STATUS_PKT_REE_SHIFT)
+#define SXE2_RX_DESC_EXT_STATUS_ROCE_MASK \
+ (0x1ULL << SXE2_RX_DESC_EXT_STATUS_ROCE_SHIFT)
+
+enum sxe2_rx_desc_ipsec_shift {
+ SXE2_RX_DESC_IPSEC_PKT_S = 21,
+ SXE2_RX_DESC_IPSEC_ENGINE_S = 22,
+ SXE2_RX_DESC_IPSEC_MODE_S = 23,
+ SXE2_RX_DESC_IPSEC_STATUS_S = 24,
+
+ SXE2_RX_DESC_IPSEC_LAST
+};
+
+enum sxe2_rx_desc_ipsec_status {
+ SXE2_RX_DESC_IPSEC_STATUS_SUCCESS = 0x0,
+ SXE2_RX_DESC_IPSEC_STATUS_PKG_OVER_2K = 0x1,
+ SXE2_RX_DESC_IPSEC_STATUS_SPI_IP_INVALID = 0x2,
+ SXE2_RX_DESC_IPSEC_STATUS_SA_INVALID = 0x3,
+ SXE2_RX_DESC_IPSEC_STATUS_NOT_ALIGN = 0x4,
+ SXE2_RX_DESC_IPSEC_STATUS_ICV_ERROR = 0x5,
+ SXE2_RX_DESC_IPSEC_STATUS_BY_PASSH = 0x6,
+ SXE2_RX_DESC_IPSEC_STATUS_MAC_BY_PASSH = 0x7,
+};
+
+#define SXE2_RX_DESC_IPSEC_PKT_MASK \
+ (0x1ULL << SXE2_RX_DESC_IPSEC_PKT_S)
+#define SXE2_RX_DESC_IPSEC_STATUS_MASK (0x7)
+#define SXE2_RX_DESC_IPSEC_STATUS_VAL_GET(qw2) \
+ (((qw2) >> SXE2_RX_DESC_IPSEC_STATUS_S) & \
+ SXE2_RX_DESC_IPSEC_STATUS_MASK)
+
+#define SXE2_RX_ERR_BITS 0x3f
+
+#define SXE2_RX_QUEUE_CHECK_INTERVAL_NUM 4
+
+#define SXE2_RX_DESC_RING_ALIGN \
+ (SXE2_ALIGN / sizeof(union sxe2_rx_desc))
+
+#define SXE2_RX_RING_SIZE \
+ ((SXE2_MAX_RING_DESC + SXE2_RX_PKTS_BURST_BATCH_NUM) * sizeof(union sxe2_rx_desc))
+
+#define SXE2_RX_MAX_DATA_BUF_SIZE (16 * 1024 - 128)
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.h b/drivers/net/sxe2/sxe2_txrx_poll.h
new file mode 100644
index 0000000000..4924b0f41f
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_poll.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef SXE2_TXRX_POLL_H
+#define SXE2_TXRX_POLL_H
+
+#include "sxe2_queue.h"
+
+u16 sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
+
+u16 sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts);
+
+u16 sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts);
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_vsi.c b/drivers/net/sxe2/sxe2_vsi.c
new file mode 100644
index 0000000000..e1e0e279cd
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_vsi.c
@@ -0,0 +1,212 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_os.h>
+#include <rte_tailq.h>
+#include <rte_malloc.h>
+#include "sxe2_ethdev.h"
+#include "sxe2_vsi.h"
+#include "sxe2_common_log.h"
+#include "sxe2_cmd_chnl.h"
+
+void sxe2_sw_vsi_ctx_hw_cap_set(struct sxe2_adapter *adapter,
+ struct sxe2_drv_vsi_caps *vsi_caps)
+{
+ adapter->vsi_ctxt.dpdk_vsi_id = vsi_caps->dpdk_vsi_id;
+ adapter->vsi_ctxt.kernel_vsi_id = vsi_caps->kernel_vsi_id;
+ adapter->vsi_ctxt.vsi_type = vsi_caps->vsi_type;
+}
+
+static struct sxe2_vsi *
+sxe2_vsi_node_alloc(struct sxe2_adapter *adapter, u16 vsi_id, u16 vsi_type)
+{
+ struct sxe2_vsi *vsi = NULL;
+ vsi = rte_zmalloc("sxe2_vsi", sizeof(*vsi), 0);
+ if (vsi == NULL) {
+ PMD_LOG_ERR(DRV, "Failed to malloc vf vsi struct.");
+ goto l_end;
+ }
+ vsi->adapter = adapter;
+
+ vsi->vsi_id = vsi_id;
+ vsi->vsi_type = vsi_type;
+
+l_end:
+ return vsi;
+}
+
+static void sxe2_vsi_queues_num_set(struct sxe2_vsi *vsi, u16 num_queues, u16 base_idx)
+{
+ vsi->txqs.q_cnt = num_queues;
+ vsi->rxqs.q_cnt = num_queues;
+ vsi->txqs.base_idx_in_func = base_idx;
+ vsi->rxqs.base_idx_in_func = base_idx;
+}
+
+static void sxe2_vsi_queues_cfg(struct sxe2_vsi *vsi)
+{
+ vsi->txqs.depth = vsi->txqs.depth ? : SXE2_DFLT_NUM_TX_DESC;
+ vsi->rxqs.depth = vsi->rxqs.depth ? : SXE2_DFLT_NUM_RX_DESC;
+
+ PMD_LOG_INFO(DRV, "vsi:%u queue_cnt:%u txq_depth:%u rxq_depth:%u.",
+ vsi->vsi_id, vsi->txqs.q_cnt,
+ vsi->txqs.depth, vsi->rxqs.depth);
+}
+
+static void sxe2_vsi_irqs_cfg(struct sxe2_vsi *vsi, u16 num_irqs, u16 base_idx)
+{
+ vsi->irqs.avail_cnt = num_irqs;
+ vsi->irqs.base_idx_in_pf = base_idx;
+}
+
+static struct sxe2_vsi *sxe2_vsi_node_create(struct sxe2_adapter *adapter, u16 vsi_id, u16 vsi_type)
+{
+ struct sxe2_vsi *vsi = NULL;
+ u16 num_queues = 0;
+ u16 queue_base_idx = 0;
+ u16 num_irqs = 0;
+ u16 irq_base_idx = 0;
+
+ vsi = sxe2_vsi_node_alloc(adapter, vsi_id, vsi_type);
+ if (vsi == NULL)
+ goto l_end;
+
+ if (vsi_type == SXE2_VSI_T_DPDK_PF ||
+ vsi_type == SXE2_VSI_T_DPDK_VF) {
+ num_queues = adapter->q_ctxt.qp_cnt_assign;
+ queue_base_idx = adapter->q_ctxt.base_idx_in_pf;
+
+ num_irqs = adapter->irq_ctxt.max_cnt_hw;
+ irq_base_idx = adapter->irq_ctxt.base_idx_in_func;
+ } else if (vsi_type == SXE2_VSI_T_DPDK_ESW) {
+ num_queues = 1;
+ num_irqs = 1;
+ }
+
+ sxe2_vsi_queues_num_set(vsi, num_queues, queue_base_idx);
+
+ sxe2_vsi_queues_cfg(vsi);
+
+ sxe2_vsi_irqs_cfg(vsi, num_irqs, irq_base_idx);
+
+l_end:
+ return vsi;
+}
+
+static void sxe2_vsi_node_free(struct sxe2_vsi *vsi)
+{
+ if (!vsi)
+ return;
+
+ rte_free(vsi);
+ vsi = NULL;
+}
+
+static s32 sxe2_vsi_destroy(struct sxe2_adapter *adapter, struct sxe2_vsi *vsi)
+{
+ s32 ret = SXE2_SUCCESS;
+
+ if (vsi == NULL) {
+ PMD_LOG_INFO(DRV, "vsi is not created, no need to destroy.");
+ goto l_end;
+ }
+
+ if (vsi->vsi_type != SXE2_VSI_T_DPDK_ESW) {
+ ret = sxe2_drv_vsi_del(adapter, vsi);
+ if (ret) {
+ PMD_LOG_ERR(DRV, "Failed to del vsi from fw, ret=%d", ret);
+ if (ret == -EPERM)
+ goto l_free;
+ goto l_end;
+ }
+ }
+
+l_free:
+ rte_free(vsi);
+ vsi = NULL;
+
+ PMD_LOG_DEBUG(DRV, "vsi destroyed.");
+l_end:
+ return ret;
+}
+
+static s32 sxe2_main_vsi_create(struct sxe2_adapter *adapter)
+{
+ s32 ret = SXE2_SUCCESS;
+ u16 vsi_id = adapter->vsi_ctxt.dpdk_vsi_id;
+ u16 vsi_type = adapter->vsi_ctxt.vsi_type;
+ bool is_reused = (vsi_id != SXE2_INVALID_VSI_ID);
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (!is_reused)
+ vsi_type = SXE2_VSI_T_DPDK_PF;
+ else
+ PMD_LOG_INFO(DRV, "Reusing existing HW vsi_id:%u", vsi_id);
+
+ adapter->vsi_ctxt.main_vsi = sxe2_vsi_node_create(adapter, vsi_id, vsi_type);
+ if (adapter->vsi_ctxt.main_vsi == NULL) {
+ PMD_LOG_ERR(DRV, "Failed to create vsi struct, ret=%d", ret);
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ if (!is_reused) {
+ ret = sxe2_drv_vsi_add(adapter, adapter->vsi_ctxt.main_vsi);
+ if (ret) {
+ PMD_LOG_ERR(DRV, "Failed to config vsi to fw, ret=%d", ret);
+ goto l_free_vsi;
+ }
+
+ adapter->vsi_ctxt.dpdk_vsi_id = adapter->vsi_ctxt.main_vsi->vsi_id;
+ PMD_LOG_DEBUG(DRV, "Successfully created and synced new VSI");
+ }
+
+ goto l_end;
+
+l_free_vsi:
+ sxe2_vsi_node_free(adapter->vsi_ctxt.main_vsi);
+ adapter->vsi_ctxt.main_vsi = NULL;
+l_end:
+ return ret;
+}
+
+s32 sxe2_vsi_init(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ s32 ret = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ ret = sxe2_main_vsi_create(adapter);
+ if (ret) {
+ PMD_LOG_ERR(DRV, "Failed to create main VSI, ret=%d", ret);
+ goto l_end;
+ }
+
+l_end:
+ return ret;
+}
+
+void sxe2_vsi_uninit(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ s32 ret;
+
+ if (adapter->vsi_ctxt.main_vsi == NULL) {
+ PMD_LOG_INFO(DRV, "vsi is not created, no need to destroy.");
+ goto l_end;
+ }
+
+ ret = sxe2_vsi_destroy(adapter, adapter->vsi_ctxt.main_vsi);
+ if (ret) {
+ PMD_LOG_ERR(DRV, "Failed to del vsi from fw, ret=%d", ret);
+ goto l_end;
+ }
+
+ PMD_LOG_DEBUG(DRV, "vsi destroyed.");
+
+l_end:
+ return;
+}
diff --git a/drivers/net/sxe2/sxe2_vsi.h b/drivers/net/sxe2/sxe2_vsi.h
new file mode 100644
index 0000000000..8870cbe22d
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_vsi.h
@@ -0,0 +1,205 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __sxe2_VSI_H__
+#define __sxe2_VSI_H__
+#include <rte_os.h>
+#include "sxe2_type.h"
+#include "sxe2_drv_cmd.h"
+
+#define SXE2_MAX_BOND_MEMBER_CNT 4
+
+enum sxe2_drv_type {
+ SXE2_MAX_DRV_TYPE_DPDK = 0,
+ SXE2_MAX_DRV_TYPE_KERNEL,
+ SXE2_MAX_DRV_TYPE_CNT,
+};
+
+#define SXE2_MAX_USER_PRIORITY (8)
+
+#define SXE2_DFLT_NUM_RX_DESC 512
+#define SXE2_DFLT_NUM_TX_DESC 512
+
+#define SXE2_DFLT_Q_NUM_OTHER_VSI 1
+#define SXE2_INVALID_VSI_ID 0xFFFF
+
+struct sxe2_adapter;
+struct sxe2_drv_vsi_caps;
+struct rte_eth_dev;
+
+enum sxe2_vsi_type {
+ SXE2_VSI_T_PF = 0,
+ SXE2_VSI_T_VF,
+ SXE2_VSI_T_CTRL,
+ SXE2_VSI_T_LB,
+ SXE2_VSI_T_MACVLAN,
+ SXE2_VSI_T_ESW,
+ SXE2_VSI_T_RDMA,
+ SXE2_VSI_T_DPDK_PF,
+ SXE2_VSI_T_DPDK_VF,
+ SXE2_VSI_T_DPDK_ESW,
+ SXE2_VSI_T_NR,
+};
+
+struct sxe2_queue_info {
+ u16 base_idx_in_nic;
+ u16 base_idx_in_func;
+ u16 q_cnt;
+ u16 depth;
+ u16 rx_buf_len;
+ u16 max_frame_len;
+ struct sxe2_queue **queues;
+};
+
+struct sxe2_vsi_irqs {
+ u16 avail_cnt;
+ u16 used_cnt;
+ u16 base_idx_in_pf;
+};
+
+enum {
+ sxe2_VSI_DOWN = 0,
+ sxe2_VSI_CLOSE,
+ sxe2_VSI_DISABLE,
+ sxe2_VSI_MAX,
+};
+
+struct sxe2_stats {
+ u64 ipackets;
+
+ u64 opackets;
+
+ u64 ibytes;
+
+ u64 obytes;
+
+ u64 ierrors;
+
+ u64 imissed;
+
+ u64 rx_out_of_buffer;
+ u64 rx_qblock_drop;
+
+ u64 tx_frame_good;
+ u64 rx_frame_good;
+ u64 rx_crc_errors;
+ u64 tx_bytes_good;
+ u64 rx_bytes_good;
+ u64 tx_multicast_good;
+ u64 tx_broadcast_good;
+ u64 rx_multicast_good;
+ u64 rx_broadcast_good;
+ u64 rx_len_errors;
+ u64 rx_out_of_range_errors;
+ u64 rx_oversize_pkts_phy;
+ u64 rx_symbol_err;
+ u64 rx_pause_frame;
+ u64 tx_pause_frame;
+
+ u64 rx_discards_phy;
+ u64 rx_discards_ips_phy;
+
+ u64 tx_dropped_link_down;
+ u64 rx_undersize_good;
+ u64 rx_runt_error;
+ u64 tx_bytes_good_bad;
+ u64 tx_frame_good_bad;
+ u64 rx_jabbers;
+ u64 rx_size_64;
+ u64 rx_size_65_127;
+ u64 rx_size_128_255;
+ u64 rx_size_256_511;
+ u64 rx_size_512_1023;
+ u64 rx_size_1024_1522;
+ u64 rx_size_1523_max;
+ u64 rx_pcs_symbol_err_phy;
+ u64 rx_corrected_bits_phy;
+ u64 rx_err_lane_0_phy;
+ u64 rx_err_lane_1_phy;
+ u64 rx_err_lane_2_phy;
+ u64 rx_err_lane_3_phy;
+
+ u64 rx_prio_buf_discard[SXE2_MAX_USER_PRIORITY];
+ u64 rx_illegal_bytes;
+ u64 rx_oversize_good;
+ u64 tx_unicast;
+ u64 tx_broadcast;
+ u64 tx_multicast;
+ u64 tx_vlan_packet_good;
+ u64 tx_size_64;
+ u64 tx_size_65_127;
+ u64 tx_size_128_255;
+ u64 tx_size_256_511;
+ u64 tx_size_512_1023;
+ u64 tx_size_1024_1522;
+ u64 tx_size_1523_max;
+ u64 tx_underflow_error;
+ u64 rx_byte_good_bad;
+ u64 rx_frame_good_bad;
+ u64 rx_unicast_good;
+ u64 rx_vlan_packets;
+
+ u64 prio_xoff_rx[SXE2_MAX_USER_PRIORITY];
+ u64 prio_xon_rx[SXE2_MAX_USER_PRIORITY];
+ u64 prio_xon_tx[SXE2_MAX_USER_PRIORITY];
+ u64 prio_xoff_tx[SXE2_MAX_USER_PRIORITY];
+ u64 prio_xon_2_xoff[SXE2_MAX_USER_PRIORITY];
+
+ u64 rx_vsi_unicast_packets;
+ u64 rx_vsi_bytes;
+ u64 tx_vsi_unicast_packets;
+ u64 tx_vsi_bytes;
+ u64 rx_vsi_multicast_packets;
+ u64 tx_vsi_multicast_packets;
+ u64 rx_vsi_broadcast_packets;
+ u64 tx_vsi_broadcast_packets;
+
+ u64 rx_sw_unicast_packets;
+ u64 rx_sw_broadcast_packets;
+ u64 rx_sw_multicast_packets;
+ u64 rx_sw_drop_packets;
+ u64 rx_sw_drop_bytes;
+};
+
+struct sxe2_vsi_stats {
+ struct sxe2_stats vsi_sw_stats;
+ struct sxe2_stats vsi_sw_stats_prev;
+ struct sxe2_stats vsi_hw_stats;
+ struct sxe2_stats stats;
+};
+
+struct sxe2_vsi {
+ TAILQ_ENTRY(sxe2_vsi) next;
+ struct sxe2_adapter *adapter;
+ u16 vsi_id;
+ u16 vsi_type;
+ struct sxe2_vsi_irqs irqs;
+ struct sxe2_queue_info txqs;
+ struct sxe2_queue_info rxqs;
+ u16 budget;
+ struct sxe2_vsi_stats vsi_stats;
+};
+
+TAILQ_HEAD(sxe2_vsi_list_head, sxe2_vsi);
+
+struct sxe2_vsi_context {
+ u16 func_id;
+ u16 dpdk_vsi_id;
+ u16 kernel_vsi_id;
+ u16 vsi_type;
+
+ u16 bond_member_kernel_vsi_id[SXE2_MAX_BOND_MEMBER_CNT];
+ u16 bond_member_dpdk_vsi_id[SXE2_MAX_BOND_MEMBER_CNT];
+
+ struct sxe2_vsi *main_vsi;
+};
+
+void sxe2_sw_vsi_ctx_hw_cap_set(struct sxe2_adapter *adapter,
+ struct sxe2_drv_vsi_caps *vsi_caps);
+
+s32 sxe2_vsi_init(struct rte_eth_dev *dev);
+
+void sxe2_vsi_uninit(struct rte_eth_dev *dev);
+
+#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v13 02/10] doc: add sxe2 guide and release notes
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Add a new guide for SXE2 PMD in the nics directory.
The guide contains driver capabilities, prerequisites,
and compilation/usage instructions.
Update the release notes to announce the addition of the
sxe2 network driver.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
doc/guides/nics/features/sxe2.ini | 30 +++++++++++++++++++++++
doc/guides/nics/index.rst | 1 +
doc/guides/nics/sxe2.rst | 34 ++++++++++++++++++++++++++
doc/guides/rel_notes/release_26_07.rst | 4 +++
4 files changed, 69 insertions(+)
create mode 100644 doc/guides/nics/features/sxe2.ini
create mode 100644 doc/guides/nics/sxe2.rst
diff --git a/doc/guides/nics/features/sxe2.ini b/doc/guides/nics/features/sxe2.ini
new file mode 100644
index 0000000000..2718a702d4
--- /dev/null
+++ b/doc/guides/nics/features/sxe2.ini
@@ -0,0 +1,30 @@
+;
+; Supported features of the 'sxe2' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Fast mbuf free = P
+Free Tx mbuf on demand = Y
+Burst mode info = Y
+Queue start/stop = Y
+MTU update = Y
+Buffer split on Rx = P
+Scattered Rx = Y
+CRC offload = Y
+VLAN offload = Y
+QinQ offload = P
+L3 checksum offload = Y
+L4 checksum offload = Y
+Timestamp offload = P
+Inner L3 checksum = P
+Inner L4 checksum = P
+Rx descriptor status = Y
+Tx descriptor status = Y
+FreeBSD = Y
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index cb818284fe..e20be478f8 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -68,6 +68,7 @@ Network Interface Controller Drivers
rnp
sfc_efx
softnic
+ sxe2
tap
thunderx
txgbe
diff --git a/doc/guides/nics/sxe2.rst b/doc/guides/nics/sxe2.rst
new file mode 100644
index 0000000000..7fcf9c085b
--- /dev/null
+++ b/doc/guides/nics/sxe2.rst
@@ -0,0 +1,34 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+
+SXE2 Poll Mode Driver
+======================
+
+The sxe2 PMD (**librte_net_sxe2**) provides poll mode driver support for
+10/25/50/100 Gbps Network Adapters.
+The embedded switch, Physical Functions (PF),
+and SR-IOV Virtual Functions (VF) are supported.
+
+Implementation details
+----------------------
+
+The sxe2 PMD is designed to operate alongside the sxe2 kernel network driver.
+For management and control operations, the PMD communicates with the kernel
+driver via ioctl interfaces. These commands are processed by the kernel
+driver and subsequently dispatched to the hardware firmware for execution.
+
+For security and robustness, the driver's data path is optimized to operate
+using virtual addresses (IOVA as VA mode). However, to ensure full
+compatibility in system environments where an IOMMU is absent or disabled,
+the driver also provides an explicit path to support physical addressing
+(IOVA as PA mode).
+
+The hardware is capable of handling the corresponding IOVA addresses (either
+VA or PA) directly, as provided by the DPDK memory subsystem. This ensures
+that DPDK applications can only access memory segments explicitly allocated
+to the current process, preventing unauthorized access to random physical
+memory.
+
+This capability allows the PMD to coexist with kernel network interfaces
+which remain functional, although they stop receiving unicast packets as
+long as they share the same MAC address.
diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
index f012d47a4b..fa0f0f5cca 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -64,6 +64,10 @@ New Features
* ``--auto-probing`` enables the initial bus probing, which is the current default behavior.
+* **Added Linkdata sxe2 ethernet driver.**
+
+ Added network driver for the Linkdata Network Adapters.
+
Removed Items
-------------
--
2.47.3
^ permalink raw reply related
* [PATCH v13 10/10] net/sxe2: add vectorized Rx and Tx
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch implements the vectorized data path for the sxe2 PMD.
It utilizes SIMD instructions (e.g., SSE) to process multiple
packets simultaneously, significantly improving throughput for
small packet processing.
The implementation includes:
* Vectorized Rx burst function for bulk descriptor processing.
* Vectorized Tx burst function with optimized resource cleanup.
* Capability flags update to reflect vectorized path support.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/meson.build | 7 +
drivers/net/sxe2/sxe2_ethdev.c | 35 +-
drivers/net/sxe2/sxe2_ethdev.h | 1 -
drivers/net/sxe2/sxe2_queue.c | 28 ++
drivers/net/sxe2/sxe2_queue.h | 3 +
drivers/net/sxe2/sxe2_txrx.c | 223 +++++++---
drivers/net/sxe2/sxe2_txrx.h | 11 +-
drivers/net/sxe2/sxe2_txrx_poll.h | 3 +-
drivers/net/sxe2/sxe2_txrx_vec.c | 197 +++++++++
drivers/net/sxe2/sxe2_txrx_vec.h | 72 ++++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 235 ++++++++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 545 ++++++++++++++++++++++++
12 files changed, 1277 insertions(+), 83 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_sse.c
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index b348dd71a1..3df57aee8c 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -11,6 +11,12 @@ cflags += ['-g']
deps += ['common_sxe2', 'hash','cryptodev','security']
+includes += include_directories('../../common/sxe2')
+
+if arch_subdir == 'x86'
+ sources += files('sxe2_txrx_vec_sse.c')
+endif
+
sources += files(
'sxe2_ethdev.c',
'sxe2_cmd_chnl.c',
@@ -20,6 +26,7 @@ sources += files(
'sxe2_rx.c',
'sxe2_txrx_poll.c',
'sxe2_txrx.c',
+ 'sxe2_txrx_vec.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 7e9a842eb9..b6b444a600 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -58,17 +58,11 @@ static const struct rte_pci_id pci_id_sxe2_tbl[] = {
};
static struct sxe2_pci_map_addr_info sxe2_net_map_addr_info_pf[SXE2_PCI_MAP_RES_MAX_COUNT] = {
- /* SXE2_PCI_MAP_RES_INVALID */
{0, 0, 0},
- /* SXE2_PCI_MAP_RES_DOORBELL_TX */
{ SXE2_TXQ_LEGACY_DBLL(0), 0, 4},
- /* SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL */
{ SXE2_RXQ_TAIL(0), 0, 4},
- /* SXE2_PCI_MAP_RES_IRQ_DYN */
{ SXE2_VF_DYN_CTL(0), 0, 4},
- /* SXE2_PCI_MAP_RES_IRQ_ITR(默认使用ITR0) */
{ SXE2_VF_INT_ITR(0, 0), 0, 4},
- /* SXE2_PCI_MAP_RES_IRQ_MSIX */
{ SXE2_BAR4_MSIX_CTL(0), 4, 0x10},
};
@@ -101,25 +95,6 @@ static s32 sxe2_dev_stop(struct rte_eth_dev *dev)
return ret;
}
-static s32 sxe2_queues_start(struct rte_eth_dev *dev)
-{
- s32 ret = SXE2_SUCCESS;
- ret = sxe2_txqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start tx queue.");
- goto l_end;
- }
-
- ret = sxe2_rxqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start rx queue.");
- sxe2_txqs_all_stop(dev);
- }
-
-l_end:
- return ret;
-}
-
static s32 sxe2_dev_start(struct rte_eth_dev *dev)
{
s32 ret = SXE2_SUCCESS;
@@ -152,7 +127,7 @@ static s32 sxe2_dev_start(struct rte_eth_dev *dev)
static s32 sxe2_dev_close(struct rte_eth_dev *dev)
{
(void)sxe2_dev_stop(dev);
-
+ (void)sxe2_queues_release(dev);
sxe2_vsi_uninit(dev);
sxe2_dev_pci_map_uinit(dev);
@@ -290,13 +265,19 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_close = sxe2_dev_close,
.dev_infos_get = sxe2_dev_infos_get,
+ .rx_queue_start = sxe2_rx_queue_start,
+ .rx_queue_stop = sxe2_rx_queue_stop,
+ .tx_queue_start = sxe2_tx_queue_start,
+ .tx_queue_stop = sxe2_tx_queue_stop,
.rx_queue_setup = sxe2_rx_queue_setup,
- .tx_queue_setup = sxe2_tx_queue_setup,
.rx_queue_release = sxe2_rx_queue_release,
+ .tx_queue_setup = sxe2_tx_queue_setup,
.tx_queue_release = sxe2_tx_queue_release,
.rxq_info_get = sxe2_rx_queue_info_get,
.txq_info_get = sxe2_tx_queue_info_get,
+ .rx_burst_mode_get = sxe2_rx_burst_mode_get,
+ .tx_burst_mode_get = sxe2_tx_burst_mode_get,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
diff --git a/drivers/net/sxe2/sxe2_ethdev.h b/drivers/net/sxe2/sxe2_ethdev.h
index 4ef7854479..43148f9b03 100644
--- a/drivers/net/sxe2/sxe2_ethdev.h
+++ b/drivers/net/sxe2/sxe2_ethdev.h
@@ -11,7 +11,6 @@
#include <rte_tm_driver.h>
#include <rte_io.h>
-#include "sxe2_common.h"
#include "sxe2_errno.h"
#include "sxe2_type.h"
#include "sxe2_vsi.h"
diff --git a/drivers/net/sxe2/sxe2_queue.c b/drivers/net/sxe2/sxe2_queue.c
index 98343679f6..b1860490aa 100644
--- a/drivers/net/sxe2/sxe2_queue.c
+++ b/drivers/net/sxe2/sxe2_queue.c
@@ -6,6 +6,8 @@
#include "sxe2_queue.h"
#include "sxe2_common_log.h"
#include "sxe2_errno.h"
+#include "sxe2_tx.h"
+#include "sxe2_rx.h"
void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
struct sxe2_drv_queue_caps *q_caps)
@@ -37,3 +39,29 @@ s32 sxe2_queues_init(struct rte_eth_dev *dev)
return ret;
}
+
+s32 sxe2_queues_start(struct rte_eth_dev *dev)
+{
+ s32 ret = SXE2_SUCCESS;
+
+ ret = sxe2_txqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start tx queue.");
+ goto l_end;
+ }
+
+ ret = sxe2_rxqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start rx queue.");
+ sxe2_txqs_all_stop(dev);
+ }
+l_end:
+ return ret;
+}
+
+void sxe2_queues_release(struct rte_eth_dev *dev)
+{
+ sxe2_all_rxqs_release(dev);
+
+ sxe2_all_txqs_release(dev);
+}
diff --git a/drivers/net/sxe2/sxe2_queue.h b/drivers/net/sxe2/sxe2_queue.h
index 7fa22e2820..93402186c7 100644
--- a/drivers/net/sxe2/sxe2_queue.h
+++ b/drivers/net/sxe2/sxe2_queue.h
@@ -188,4 +188,7 @@ void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
s32 sxe2_queues_init(struct rte_eth_dev *dev);
+s32 sxe2_queues_start(struct rte_eth_dev *dev);
+
+void sxe2_queues_release(struct rte_eth_dev *dev);
#endif
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
index a7b94e8967..8bb0880eb6 100644
--- a/drivers/net/sxe2/sxe2_txrx.c
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -9,12 +9,11 @@
#include <rte_memzone.h>
#include <ethdev_driver.h>
#include <unistd.h>
-
#include "sxe2_txrx.h"
#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_vec.h"
#include "sxe2_txrx_poll.h"
#include "sxe2_ethdev.h"
-
#include "sxe2_common_log.h"
#include "sxe2_errno.h"
#include "sxe2_osal.h"
@@ -22,18 +21,38 @@
#if defined(RTE_ARCH_ARM64)
#include <rte_cpuflags.h>
#endif
-
+s32 __rte_cold
+sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ u32 *batch_flags)
+{
+ struct sxe2_tx_queue *txq;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ if (txq->offloads != (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) ||
+ txq->rs_thresh < SXE2_TX_PKTS_BURST_BATCH_NUM) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ }
+ *batch_flags = SXE2_TX_MODE_SIMPLE_BATCH;
+l_end:
+ return ret;
+}
static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
{
struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
s32 ret;
u16 desc_idx;
-
if (unlikely(offset >= txq->ring_depth)) {
ret = SXE2_ERR_INVAL;
goto l_end;
}
-
desc_idx = txq->next_use + offset;
desc_idx = DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
if (desc_idx >= txq->ring_depth) {
@@ -41,19 +60,16 @@ static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
if (desc_idx >= txq->ring_depth)
desc_idx -= txq->ring_depth;
}
-
if (desc_idx == 0)
desc_idx = txq->rs_thresh - 1;
else
desc_idx -= 1;
-
if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
(txq->desc_ring[desc_idx].wb.dd &
rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
ret = RTE_ETH_TX_DESC_DONE;
else
ret = RTE_ETH_TX_DESC_FULL;
-
l_end:
return ret;
}
@@ -61,13 +77,11 @@ static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
static inline s32 sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
{
struct rte_mbuf *m_seg = mbuf;
-
while (m_seg != NULL) {
if (m_seg->data_len == 0)
return SXE2_ERR_INVAL;
m_seg = m_seg->next;
}
-
return SXE2_SUCCESS;
}
@@ -79,7 +93,6 @@ u16 sxe2_tx_pkts_prepare(void *tx_queue,
u64 ol_flags = 0;
s32 ret = SXE2_SUCCESS;
s32 i = 0;
-
for (i = 0; i < nb_pkts; i++) {
mbuf = tx_pkts[i];
if (!mbuf)
@@ -98,12 +111,10 @@ u16 sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -SXE2_ERR_INVAL;
goto l_end;
}
-
if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
rte_errno = -SXE2_ERR_INVAL;
goto l_end;
}
-
#ifdef RTE_ETHDEV_DEBUG_TX
ret = rte_validate_tx_offload(mbuf);
if (ret != SXE2_SUCCESS) {
@@ -116,14 +127,12 @@ u16 sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -ret;
goto l_end;
}
-
ret = sxe2_tx_mbuf_empty_check(mbuf);
if (ret != SXE2_SUCCESS) {
rte_errno = -ret;
goto l_end;
}
}
-
l_end:
return i;
}
@@ -132,42 +141,117 @@ void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
u32 tx_mode_flags = 0;
-
+ s32 ret;
+ u32 vec_flags;
+ u32 batch_flags;
+ RTE_SET_USED(vec_flags);
PMD_INIT_FUNC_TRACE();
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = sxe2_tx_vec_support_check(dev, &vec_flags);
+ if (ret == SXE2_SUCCESS &&
+ (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128)) {
+ tx_mode_flags = vec_flags;
+#ifdef RTE_ARCH_X86
+ if ((rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)) {
+ PMD_LOG_INFO(TX, "AVX512 is not supported in build env.");
+ }
+ if (((tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) == 0) &&
+ ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1) ||
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1)) &&
+ (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256)) {
+ PMD_LOG_INFO(TX, "AVX2 is not supported in build env.");
+ }
- dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
- dev->tx_pkt_burst = sxe2_tx_pkts;
+ if (((tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) == 0))
+ tx_mode_flags |= SXE2_TX_MODE_VEC_SSE;
+#endif
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ ret = sxe2_tx_queues_vec_prepare(dev);
+ if (ret != SXE2_SUCCESS)
+ tx_mode_flags &= (~SXE2_TX_MODE_VEC_SET_MASK);
+ }
+ }
+ ret = sxe2_tx_simple_batch_support_check(dev, &batch_flags);
+ if (ret == SXE2_SUCCESS && batch_flags == SXE2_TX_MODE_SIMPLE_BATCH)
+ tx_mode_flags |= SXE2_TX_MODE_SIMPLE_BATCH;
+ }
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ dev->tx_pkt_prepare = NULL;
+#ifdef RTE_ARCH_X86
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_OFFLOAD) {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse;
+ } else {
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse_simple;
+ }
+#endif
+ } else {
+ if (tx_mode_flags & SXE2_TX_MODE_SIMPLE_BATCH) {
+ dev->tx_pkt_prepare = NULL;
+ dev->tx_pkt_burst = sxe2_tx_pkts_simple;
+ } else {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ }
+ }
adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
tx_mode_flags, dev->data->port_id);
}
+static const struct {
+ eth_tx_burst_t tx_burst;
+ const char *info;
+} sxe2_tx_burst_infos[] = {
+ { sxe2_tx_pkts, "Scalar" },
+#ifdef RTE_ARCH_X86
+ { sxe2_tx_pkts_vec_sse, "Vector SSE" },
+ { sxe2_tx_pkts_vec_sse_simple, "Vector SSE Simple" },
+#endif
+};
+
+s32 sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_tx_burst_t pkt_burst = dev->tx_pkt_burst;
+ s32 ret = SXE2_ERR_INVAL;
+ u32 i;
+ u32 size;
+ size = RTE_DIM(sxe2_tx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_tx_burst_infos[i].tx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_tx_burst_infos[i].info);
+ ret = SXE2_SUCCESS;
+ break;
+ }
+ }
+ return ret;
+}
+
static s32 sxe2_rx_desciptor_status(void *rx_queue, u16 offset)
{
struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
volatile union sxe2_rx_desc *desc;
s32 ret;
-
if (unlikely(offset >= rxq->ring_depth)) {
ret = SXE2_ERR_INVAL;
goto l_end;
}
-
if (offset >= rxq->ring_depth - rxq->hold_num) {
ret = RTE_ETH_RX_DESC_UNAVAIL;
goto l_end;
}
-
if (rxq->processing_idx + offset >= rxq->ring_depth)
desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
else
desc = &rxq->desc_ring[rxq->processing_idx + offset];
-
if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
ret = RTE_ETH_RX_DESC_DONE;
else
ret = RTE_ETH_RX_DESC_AVAIL;
-
l_end:
PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
offset, ret, rxq->queue_id, rxq->port_id);
@@ -179,7 +263,6 @@ static s32 sxe2_rx_queue_count(void *rx_queue)
struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
volatile union sxe2_rx_desc *desc;
u16 done_num = 0;
-
desc = &rxq->desc_ring[rxq->processing_idx];
while ((done_num < rxq->ring_depth) &&
(rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
@@ -190,55 +273,97 @@ static s32 sxe2_rx_queue_count(void *rx_queue)
else
desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
}
-
PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
done_num, rxq->queue_id, rxq->port_id);
-
return done_num;
}
-static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload)
-{
- struct sxe2_rx_queue *rxq;
- bool en = false;
- u16 i;
-
- for (i = 0; i < dev->data->nb_rx_queues; ++i) {
- rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
- if (rxq == NULL)
- continue;
-
- if (0 != (rxq->offloads & offload)) {
- en = true;
- goto l_end;
- }
- }
-
-l_end:
- return en;
-}
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
u32 rx_mode_flags = 0;
+ s32 ret;
+ u32 vec_flags;
PMD_INIT_FUNC_TRACE();
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = sxe2_rx_vec_support_check(dev, &vec_flags);
+ if (ret == SXE2_SUCCESS &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ rx_mode_flags = vec_flags;
+#ifdef RTE_ARCH_X86
+ if ((rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1))
+ PMD_LOG_INFO(RX, "AVX512 is not supported in build env");
+
+ if (((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) == 0) &&
+ ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1) ||
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1)) &&
+ (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256))
+ PMD_LOG_INFO(RX, "AVX2 is not supported in build env");
+
+ if (((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) == 0) &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128)
+ rx_mode_flags |= SXE2_RX_MODE_VEC_SSE;
+#endif
+ if ((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) != 0) {
+ ret = sxe2_rx_queues_vec_prepare(dev);
+ if (ret != SXE2_SUCCESS)
+ rx_mode_flags &= (~SXE2_RX_MODE_VEC_SET_MASK);
+ }
+ }
+ }
+#ifdef RTE_ARCH_X86
+ if (rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) {
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_vec_sse_offload;
+ goto l_end;
+ }
+#endif
if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
else
dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
-
+ goto l_end;
+l_end:
PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
rx_mode_flags, dev->data->port_id);
adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
}
+static const struct {
+ eth_rx_burst_t rx_burst;
+ const char *info;
+} sxe2_rx_burst_infos[] = {
+ { sxe2_rx_pkts_scattered, "Scalar Scattered" },
+ { sxe2_rx_pkts_scattered_split, "Scalar Scattered split" },
+#ifdef RTE_ARCH_X86
+ { sxe2_rx_pkts_scattered_vec_sse_offload, "Vector SSE Scattered" },
+#endif
+};
+
+s32 sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused u16 queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_rx_burst_t pkt_burst = dev->rx_pkt_burst;
+ s32 ret = SXE2_ERR_INVAL;
+ u32 i, size;
+ size = RTE_DIM(sxe2_rx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_rx_burst_infos[i].rx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_rx_burst_infos[i].info);
+ ret = SXE2_SUCCESS;
+ break;
+ }
+ }
+ return ret;
+}
+
void sxe2_set_common_function(struct rte_eth_dev *dev)
{
PMD_INIT_FUNC_TRACE();
-
dev->rx_queue_count = sxe2_rx_queue_count;
dev->rx_descriptor_status = sxe2_rx_desciptor_status;
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
index e6f671e3dc..8f929c4f19 100644
--- a/drivers/net/sxe2/sxe2_txrx.h
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -6,16 +6,17 @@
#define SXE2_TXRX_H
#include <ethdev_driver.h>
#include "sxe2_queue.h"
-
void sxe2_set_common_function(struct rte_eth_dev *dev);
+s32 __rte_cold sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ u32 *batch_flags);
u16 sxe2_tx_pkts_prepare(void *tx_queue,
struct rte_mbuf **tx_pkts, u16 nb_pkts);
-
void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
-
void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
-
+s32 sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
+s32 sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused u16 queue_id, struct rte_eth_burst_mode *mode);
#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.h b/drivers/net/sxe2/sxe2_txrx_poll.h
index 4924b0f41f..67da08e58e 100644
--- a/drivers/net/sxe2/sxe2_txrx_poll.h
+++ b/drivers/net/sxe2/sxe2_txrx_poll.h
@@ -8,7 +8,8 @@
#include "sxe2_queue.h"
u16 sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
-
+u16 sxe2_tx_pkts_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts);
u16 sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts);
u16 sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts);
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.c b/drivers/net/sxe2/sxe2_txrx_vec.c
new file mode 100644
index 0000000000..30e1468020
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.c
@@ -0,0 +1,197 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+
+s32 __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags)
+{
+ struct sxe2_rx_queue *rxq;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ *vec_flags = SXE2_RX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ if (!rte_is_power_of_2(rxq->ring_depth)) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if (rxq->rx_free_thresh < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC &&
+ (rxq->ring_depth % rxq->rx_free_thresh) != 0) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_RX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ u16 i;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+ if ((rxq->offloads & offload) != 0) {
+ en = true;
+ goto l_end;
+ }
+ }
+l_end:
+ return en;
+}
+
+static inline void sxe2_rx_queue_mbufs_release_vec(struct sxe2_rx_queue *rxq)
+{
+ const u16 mask = rxq->ring_depth - 1;
+ u16 i;
+ if (unlikely(!rxq->buffer_ring)) {
+ PMD_LOG_DEBUG(RX, "Rx queue release mbufs vec, buffer_ring if NULL."
+ "port_id:%u queue_id:%u", rxq->port_id, rxq->queue_id);
+ return;
+ }
+ if (rxq->realloc_num >= rxq->ring_depth)
+ return;
+ if (rxq->realloc_num == 0) {
+ for (i = 0; i < rxq->ring_depth; ++i) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ } else {
+ for (i = rxq->processing_idx;
+ i != rxq->realloc_start;
+ i = (i + 1) & mask) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ }
+ rxq->realloc_num = rxq->ring_depth;
+ memset(rxq->buffer_ring, 0, rxq->ring_depth * sizeof(rxq->buffer_ring[0]));
+}
+
+static inline void sxe2_rx_queue_vec_init(struct sxe2_rx_queue *rxq)
+{
+ uintptr_t data;
+ struct rte_mbuf mbuf_def;
+
+ memset(&mbuf_def, 0, sizeof(mbuf_def));
+ mbuf_def.buf_addr = 0;
+ mbuf_def.nb_segs = 1;
+ mbuf_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mbuf_def, 1);
+ rte_compiler_barrier();
+ data = (uintptr_t)&mbuf_def.rearm_data;
+ rxq->mbuf_init_value = *(u64 *)data;
+}
+
+s32 __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_rx_queue *rxq = NULL;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ PMD_LOG_INFO(RX, "Failed to prepare rx queue, rxq[%d] is NULL", i);
+ continue;
+ }
+ rxq->ops.mbufs_release = sxe2_rx_queue_mbufs_release_vec;
+ sxe2_rx_queue_vec_init(rxq);
+ }
+ return ret;
+}
+
+s32 __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags)
+{
+ struct sxe2_tx_queue *txq;
+ s32 ret = SXE2_SUCCESS;
+ u32 i;
+ *vec_flags = SXE2_TX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ if (txq->rs_thresh < SXE2_TX_RS_THRESH_MIN_VEC ||
+ txq->rs_thresh > SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_TX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+static void sxe2_tx_queue_mbufs_release_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ u16 i;
+
+ if (unlikely(txq == NULL || txq->buffer_ring == NULL)) {
+ PMD_LOG_ERR(TX, "Tx release mbufs vec, invalid params.");
+ return;
+ }
+ i = txq->next_dd - (txq->rs_thresh - 1);
+ buffer = txq->buffer_ring;
+ if (txq->next_use < i) {
+ for ( ; i < txq->ring_depth; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+ i = 0;
+ }
+ for (; i < txq->next_use; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+}
+
+s32 __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_tx_queue *txq = NULL;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL) {
+ PMD_LOG_INFO(TX, "Failed to prepare tx queue, txq[%d] is NULL", i);
+ continue;
+ }
+ txq->ops.mbufs_release = sxe2_tx_queue_mbufs_release_vec;
+ }
+ return ret;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.h b/drivers/net/sxe2/sxe2_txrx_vec.h
new file mode 100644
index 0000000000..cb6a3dd3b8
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef _SXE2_TXRX_VEC_H_
+#define _SXE2_TXRX_VEC_H_
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+#include "sxe2_type.h"
+#define SXE2_RX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_RX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_RX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_RX_MODE_VEC_AVX2 RTE_BIT32(3)
+#define SXE2_RX_MODE_VEC_AVX512 RTE_BIT32(4)
+#define SXE2_RX_MODE_VEC_NEON RTE_BIT32(5)
+#define SXE2_RX_MODE_BATCH_ALLOC RTE_BIT32(10)
+#define SXE2_RX_MODE_VEC_SET_MASK (SXE2_RX_MODE_VEC_SIMPLE | \
+ SXE2_RX_MODE_VEC_OFFLOAD | SXE2_RX_MODE_VEC_SSE | \
+ SXE2_RX_MODE_VEC_AVX2 | SXE2_RX_MODE_VEC_AVX512 | \
+ SXE2_RX_MODE_VEC_NEON)
+#define SXE2_TX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_TX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_TX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_TX_MODE_VEC_AVX2 RTE_BIT32(3)
+#define SXE2_TX_MODE_VEC_AVX512 RTE_BIT32(4)
+#define SXE2_TX_MODE_VEC_NEON RTE_BIT32(5)
+#define SXE2_TX_MODE_SIMPLE_BATCH RTE_BIT32(10)
+#define SXE2_TX_MODE_VEC_SET_MASK (SXE2_TX_MODE_VEC_SIMPLE | \
+ SXE2_TX_MODE_VEC_OFFLOAD | SXE2_TX_MODE_VEC_SSE | \
+ SXE2_TX_MODE_VEC_AVX2 | SXE2_TX_MODE_VEC_AVX512 | \
+ SXE2_TX_MODE_VEC_NEON)
+#define SXE2_TX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS | \
+ RTE_ETH_TX_OFFLOAD_QINQ_INSERT | \
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_SECURITY | \
+ RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM)
+#define SXE2_TX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_VLAN_INSERT | \
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM)
+#define SXE2_RX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP | \
+ RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT | \
+ RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_SECURITY | \
+ RTE_ETH_RX_OFFLOAD_QINQ_STRIP)
+#define SXE2_RX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_CHECKSUM | \
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_VLAN_STRIP | \
+ RTE_ETH_RX_OFFLOAD_VLAN_FILTER | \
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)
+#ifdef RTE_ARCH_X86
+u16 sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
+u16 sxe2_tx_pkts_vec_sse_simple(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
+u16 sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts);
+#endif
+s32 __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags);
+s32 __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev);
+s32 __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags);
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload);
+s32 __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev);
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_common.h b/drivers/net/sxe2/sxe2_txrx_vec_common.h
new file mode 100644
index 0000000000..c0405c9a59
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_common.h
@@ -0,0 +1,235 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TXRX_VEC_COMMON_H__
+#define __SXE2_TXRX_VEC_COMMON_H__
+#include <rte_atomic.h>
+#ifdef PCLINT
+#include "avx_stub.h"
+#endif
+#include "sxe2_rx.h"
+#include "sxe2_queue.h"
+#include "sxe2_tx.h"
+#include "sxe2_vsi.h"
+#include "sxe2_ethdev.h"
+#define SXE2_RX_NUM_PER_LOOP_SSE 4
+#define SXE2_RX_NUM_PER_LOOP_AVX 8
+#define SXE2_RX_NUM_PER_LOOP_NEON 4
+#define SXE2_RX_REARM_THRESH_VEC 64
+#define SXE2_RX_PKTS_BURST_BATCH_NUM_VEC 32
+#define SXE2_TX_RS_THRESH_MIN_VEC 32
+#define SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC 64
+
+static __rte_always_inline void
+sxe2_tx_pkts_mbuf_fill(struct sxe2_tx_buffer *buffer,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ u16 i;
+ for (i = 0; i < nb_pkts; ++i)
+ buffer[i].mbuf = tx_pkts[i];
+}
+
+static __rte_always_inline s32
+sxe2_tx_bufs_free_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC];
+ s32 ret;
+ u32 i;
+ u16 rs_thresh;
+ u16 free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - (rs_thresh - 1)];
+ mbuf = rte_pktmbuf_prefree_seg(buffer[0].mbuf);
+ if (likely(mbuf)) {
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (likely(mbuf)) {
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline void
+sxe2_tx_desc_fill_offloads(struct rte_mbuf *mbuf, u64 *desc_qw1)
+{
+ u64 offloads = mbuf->ol_flags;
+ u32 desc_cmd = 0;
+ u32 desc_offset = 0;
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ }
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ default:
+ break;
+ }
+ *desc_qw1 |= ((u64)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ *desc_qw1 |= ((u64)mbuf->vlan_tci) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT;
+ }
+ *desc_qw1 |= ((u64)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT;
+}
+#define SXE2_RX_UMBCAST_FLAGS_VAL_GET(_flags) \
+ (((_flags) & 0x30) >> 4)
+
+static inline void sxe2_vf_rx_vec_sw_stats_cnt(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf *mbuf, u8 umbcast_flag)
+{
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN, rte_memory_order_relaxed);
+ switch (SXE2_RX_UMBCAST_FLAGS_VAL_GET(umbcast_flag)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static inline u16
+sxe2_rx_pkts_refactor(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **mbuf_bufs, u16 mbuf_num,
+ u8 *split_rxe_flags, u8 *umbcast_flags)
+{
+ struct rte_mbuf *done_pkts[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ struct rte_mbuf *first_seg = rxq->pkt_first_seg;
+ struct rte_mbuf *last_seg = rxq->pkt_last_seg;
+ struct rte_mbuf *tmp_seg;
+ u16 done_num, buf_idx;
+ done_num = 0;
+ for (buf_idx = 0; buf_idx < mbuf_num; buf_idx++) {
+ if (last_seg) {
+ last_seg->next = mbuf_bufs[buf_idx];
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ first_seg->nb_segs++;
+ first_seg->pkt_len += mbuf_bufs[buf_idx]->data_len;
+ last_seg = last_seg->next;
+ if (split_rxe_flags[buf_idx] == 0) {
+ first_seg->hash = last_seg->hash;
+ first_seg->vlan_tci = last_seg->vlan_tci;
+ first_seg->ol_flags = last_seg->ol_flags;
+ first_seg->pkt_len -= rxq->crc_len;
+ if (last_seg->data_len > rxq->crc_len) {
+ last_seg->data_len -= rxq->crc_len;
+ } else {
+ tmp_seg = first_seg;
+ first_seg->nb_segs--;
+ while (tmp_seg->next != last_seg)
+ tmp_seg = tmp_seg->next;
+ tmp_seg->data_len -= (rxq->crc_len - last_seg->data_len);
+ tmp_seg->next = NULL;
+ rte_pktmbuf_free_seg(last_seg);
+ last_seg = NULL;
+ }
+ done_pkts[done_num++] = first_seg;
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, first_seg, umbcast_flags[buf_idx]);
+ first_seg = NULL;
+ last_seg = NULL;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ continue;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ last_seg = NULL;
+ continue;
+ }
+ } else {
+ if (split_rxe_flags[buf_idx] == 0) {
+ done_pkts[done_num++] = mbuf_bufs[buf_idx];
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, mbuf_bufs[buf_idx],
+ umbcast_flags[buf_idx]);
+ continue;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ first_seg = mbuf_bufs[buf_idx];
+ last_seg = first_seg;
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ mbuf_bufs[buf_idx]->pkt_len += rxq->crc_len;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ mbuf_bufs[buf_idx]->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free_seg(mbuf_bufs[buf_idx]);
+ continue;
+ }
+ }
+ }
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+ rte_memcpy(mbuf_bufs, done_pkts, done_num * (sizeof(struct rte_mbuf *)));
+ return done_num;
+}
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_sse.c b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
new file mode 100644
index 0000000000..8cf11849d6
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
@@ -0,0 +1,545 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <ethdev_driver.h>
+#include <rte_bitops.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_vect.h>
+#include "rte_common.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_queue.h"
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_vsi.h"
+
+static __rte_always_inline void
+sxe2_tx_desc_fill_one_sse(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf *pkt,
+ u64 desc_cmd, bool with_offloads)
+{
+ __m128i data_desc;
+ u64 desc_qw1;
+ u32 desc_offset;
+ desc_qw1 = (SXE2_TX_DESC_DTYPE_DATA |
+ ((u64)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT |
+ ((u64)pkt->data_len) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL(pkt->l2_len);
+ desc_qw1 |= ((u64)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (with_offloads)
+ sxe2_tx_desc_fill_offloads(pkt, &desc_qw1);
+ data_desc = _mm_set_epi64x(desc_qw1, rte_pktmbuf_iova(pkt));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, desc), data_desc);
+}
+
+static __rte_always_inline u16
+sxe2_tx_pkts_vec_sse_batch(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ u16 nb_pkts, bool with_offloads)
+{
+ volatile union sxe2_tx_data_desc *desc;
+ struct sxe2_tx_buffer *buffer;
+ u16 next_use;
+ u16 res_num;
+ u16 tx_num;
+ u16 i;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free_vec(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx pkts sse batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ tx_num = nb_pkts;
+ next_use = txq->next_use;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ txq->desc_free_num -= nb_pkts;
+ res_num = txq->ring_depth - txq->next_use;
+ if (tx_num >= res_num) {
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, res_num);
+ for (i = 0; i < res_num - 1; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts++,
+ (SXE2_TX_DATA_DESC_CMD_EOP | SXE2_TX_DATA_DESC_CMD_RS),
+ with_offloads);
+ tx_num -= res_num;
+ next_use = 0;
+ txq->next_rs = txq->rs_thresh - 1;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ }
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, tx_num);
+ for (i = 0; i < tx_num; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ next_use += tx_num;
+ if (next_use > txq->next_rs) {
+ txq->desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ }
+ txq->next_use = next_use;
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, nb_pkts);
+l_end:
+ return nb_pkts;
+}
+
+static __rte_always_inline u16
+sxe2_tx_pkts_vec_sse_common(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ u16 nb_pkts, bool with_offloads)
+{
+ u16 tx_done_num = 0;
+ u16 tx_once_num;
+ u16 tx_need_num;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, txq->rs_thresh);
+ tx_once_num = sxe2_tx_pkts_vec_sse_batch(txq,
+ tx_pkts + tx_done_num,
+ tx_need_num, with_offloads);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+ return tx_done_num;
+}
+
+u16 sxe2_tx_pkts_vec_sse_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, false);
+}
+u16 sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, true);
+}
+
+static inline void sxe2_rx_queue_rearm_sse(struct sxe2_rx_queue *rxq)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ struct rte_mbuf *mbuf0, *mbuf1;
+ __m128i dma_addr0, dma_addr1;
+ __m128i virt_addr0, virt_addr1;
+ __m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
+ RTE_PKTMBUF_HEADROOM);
+ s32 ret;
+ u16 i;
+ u16 new_tail;
+ buffer = &rxq->buffer_ring[rxq->realloc_start];
+ desc = &rxq->desc_ring[rxq->realloc_start];
+ ret = rte_mempool_get_bulk(rxq->mb_pool, (void *)buffer,
+ SXE2_RX_REARM_THRESH_VEC);
+ if (ret != 0) {
+ PMD_LOG_INFO(RX, "Rx mbuf vec alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ if ((rxq->realloc_num + SXE2_RX_REARM_THRESH_VEC) >= rxq->ring_depth) {
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < SXE2_RX_NUM_PER_LOOP_SSE; ++i) {
+ buffer[i] = &rxq->fake_mbuf;
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc[i].read),
+ dma_addr0);
+ }
+ }
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed +=
+ SXE2_RX_REARM_THRESH_VEC;
+ goto l_end;
+ }
+ for (i = 0; i < SXE2_RX_REARM_THRESH_VEC; i += 2, buffer += 2) {
+ mbuf0 = buffer[0];
+ mbuf1 = buffer[1];
+#if RTE_IOVA_IN_MBUF
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+#endif
+ virt_addr0 = _mm_loadu_si128((__m128i *)&mbuf0->buf_addr);
+ virt_addr1 = _mm_loadu_si128((__m128i *)&mbuf1->buf_addr);
+#if RTE_IOVA_IN_MBUF
+ dma_addr0 = _mm_unpackhi_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpackhi_epi64(virt_addr1, virt_addr1);
+#else
+ dma_addr0 = _mm_unpacklo_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpacklo_epi64(virt_addr1, virt_addr1);
+#endif
+ dma_addr0 = _mm_add_epi64(dma_addr0, hdr_room);
+ dma_addr1 = _mm_add_epi64(dma_addr1, hdr_room);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr0);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr1);
+ }
+ rxq->realloc_start += SXE2_RX_REARM_THRESH_VEC;
+ if (rxq->realloc_start >= rxq->ring_depth)
+ rxq->realloc_start = 0;
+ rxq->realloc_num -= SXE2_RX_REARM_THRESH_VEC;
+ new_tail = (rxq->realloc_start == 0) ?
+ (rxq->ring_depth - 1) : (rxq->realloc_start - 1);
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, new_tail);
+l_end:
+ return;
+}
+
+static __rte_always_inline __m128i
+sxe2_rx_desc_fnav_flags_sse(__m128i descs_arr[4])
+{
+ __m128i descs_tmp1, descs_tmp2;
+ __m128i descs_fnav_vld;
+ __m128i v_zeros, v_ffff, v_u32_one;
+ __m128i m_flags;
+ const __m128i fdir_flags = _mm_set1_epi32(RTE_MBUF_F_RX_FDIR | RTE_MBUF_F_RX_FDIR_ID);
+ descs_tmp1 = _mm_unpacklo_epi32(descs_arr[0], descs_arr[1]);
+ descs_tmp2 = _mm_unpacklo_epi32(descs_arr[2], descs_arr[3]);
+ descs_fnav_vld = _mm_unpacklo_epi64(descs_tmp1, descs_tmp2);
+ descs_fnav_vld = _mm_slli_epi32(descs_fnav_vld, 26);
+ descs_fnav_vld = _mm_srli_epi32(descs_fnav_vld, 31);
+ v_zeros = _mm_setzero_si128();
+ v_ffff = _mm_cmpeq_epi32(v_zeros, v_zeros);
+ v_u32_one = _mm_srli_epi32(v_ffff, 31);
+ m_flags = _mm_cmpeq_epi32(descs_fnav_vld, v_u32_one);
+ m_flags = _mm_and_si128(m_flags, fdir_flags);
+ return m_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_desc_offloads_para_fill_sse(struct sxe2_rx_queue *rxq,
+ volatile union sxe2_rx_desc *desc __rte_unused,
+ __m128i descs_arr[4],
+ struct rte_mbuf **rx_pkts)
+{
+ const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_init_value);
+ __m128i rearm_arr[4];
+ __m128i tmp_desc_lo, tmp_desc_hi, flags, tmp_flags;
+ const __m128i desc_flags_mask = _mm_set_epi32(0x00001C04, 0x00001C04,
+ 0x00001C04, 0x00001C04);
+ const __m128i desc_flags_rss_mask = _mm_set_epi32(0x20000000, 0x20000000,
+ 0x20000000, 0x20000000);
+ const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ 0, 0, 0, 0);
+ const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, RTE_MBUF_F_RX_RSS_HASH,
+ 0, 0, 0, 0);
+ const __m128i cksum_flags =
+ _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1));
+ const __m128i cksum_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD);
+ const __m128i vlan_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED);
+ flags = _mm_unpackhi_epi32(descs_arr[0], descs_arr[1]);
+ tmp_flags = _mm_unpackhi_epi32(descs_arr[2], descs_arr[3]);
+ tmp_desc_lo = _mm_unpacklo_epi64(flags, tmp_flags);
+ tmp_desc_hi = _mm_unpackhi_epi64(flags, tmp_flags);
+ tmp_desc_lo = _mm_and_si128(tmp_desc_lo, desc_flags_mask);
+ tmp_desc_hi = _mm_and_si128(tmp_desc_hi, desc_flags_rss_mask);
+ tmp_flags = _mm_shuffle_epi8(vlan_flags, tmp_desc_lo);
+ flags = _mm_and_si128(tmp_flags, vlan_mask);
+ tmp_desc_lo = _mm_srli_epi32(tmp_desc_lo, 10);
+ tmp_flags = _mm_shuffle_epi8(cksum_flags, tmp_desc_lo);
+ tmp_flags = _mm_slli_epi32(tmp_flags, 1);
+ tmp_flags = _mm_and_si128(tmp_flags, cksum_mask);
+ flags = _mm_or_si128(flags, tmp_flags);
+ tmp_desc_hi = _mm_srli_epi32(tmp_desc_hi, 27);
+ tmp_flags = _mm_shuffle_epi8(rss_flags, tmp_desc_hi);
+ flags = _mm_or_si128(flags, tmp_flags);
+#ifndef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+ if (rxq->fnav_enable) {
+ __m128i tmp_fnav_flags = sxe2_rx_desc_fnav_flags_sse(descs_arr);
+ flags = _mm_or_si128(flags, tmp_fnav_flags);
+ rx_pkts[0]->hash.fdir.hi = desc[0].wb.fd_filter_id;
+ rx_pkts[1]->hash.fdir.hi = desc[1].wb.fd_filter_id;
+ rx_pkts[2]->hash.fdir.hi = desc[2].wb.fd_filter_id;
+ rx_pkts[3]->hash.fdir.hi = desc[3].wb.fd_filter_id;
+ }
+#endif
+ rearm_arr[0] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x30);
+ rearm_arr[1] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x30);
+ rearm_arr[2] = _mm_blend_epi16(mbuf_init, flags, 0x30);
+ rearm_arr[3] = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x30);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[0]->rearm_data), rearm_arr[0]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[1]->rearm_data), rearm_arr[1]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[2]->rearm_data), rearm_arr[2]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[3]->rearm_data), rearm_arr[3]);
+}
+
+static inline u16
+sxe2_rx_pkts_common_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts, u8 *split_rxe_flags,
+ u8 *umbcast_flags)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ __m128i descs_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i mbuf_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
+ __m128i pmbuf0;
+ __m128i ptype_all;
+#ifdef RTE_ARCH_X86_64
+ __m128i pmbuf1;
+#endif
+ u32 i;
+ u32 bit_num;
+ u16 done_num = 0;
+ const u32 *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ const __m128i crc_adjust =
+ _mm_set_epi16(0, 0, 0,
+ -rxq->crc_len,
+ 0, -rxq->crc_len,
+ 0, 0);
+ const __m128i rvp_shuf_mask =
+ _mm_set_epi8(7, 6, 5, 4,
+ 3, 2,
+ 13, 12,
+ 0XFF, 0xFF, 13, 12,
+ 0xFF, 0xFF, 0xFF, 0xFF);
+ const __m128i dd_mask = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ const __m128i eop_mask = _mm_slli_epi32(dd_mask,
+ SXE2_RX_DESC_STATUS_EOP_SHIFT);
+ const __m128i rxe_mask = _mm_set_epi64x(0x0000208000002080LL,
+ 0x0000208000002080LL);
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+ const __m128i ptype_mask = _mm_set_epi16(SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ rte_prefetch0(desc);
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, SXE2_RX_NUM_PER_LOOP_SSE);
+ if (rxq->realloc_num > SXE2_RX_REARM_THRESH_VEC)
+ sxe2_rx_queue_rearm_sse(rxq);
+ if ((rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK) == 0)
+ goto l_end;
+ buffer = &rxq->buffer_ring[rxq->processing_idx];
+ for (i = 0; i < nb_pkts; i += SXE2_RX_NUM_PER_LOOP_SSE,
+ desc += SXE2_RX_NUM_PER_LOOP_SSE) {
+ pmbuf0 = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, &buffer[i]));
+ descs_arr[3] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 3));
+ rte_compiler_barrier();
+ _mm_storeu_si128((__m128i *)&rx_pkts[i], pmbuf0);
+#ifdef RTE_ARCH_X86_64
+ pmbuf1 = _mm_loadu_si128((__m128i *)&buffer[i + 2]);
+#endif
+ descs_arr[2] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 2));
+ rte_compiler_barrier();
+ descs_arr[1] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 1));
+ rte_compiler_barrier();
+ descs_arr[0] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc));
+#ifdef RTE_ARCH_X86_64
+ _mm_storeu_si128((__m128i *)&rx_pkts[i + 2], pmbuf1);
+#endif
+ if (split_rxe_flags) {
+ rte_mbuf_prefetch_part2(rx_pkts[i]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 1]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 2]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 3]);
+ }
+ rte_compiler_barrier();
+ mbuf_arr[3] = _mm_shuffle_epi8(descs_arr[3], rvp_shuf_mask);
+ mbuf_arr[2] = _mm_shuffle_epi8(descs_arr[2], rvp_shuf_mask);
+ mbuf_arr[1] = _mm_shuffle_epi8(descs_arr[1], rvp_shuf_mask);
+ mbuf_arr[0] = _mm_shuffle_epi8(descs_arr[0], rvp_shuf_mask);
+ sterr_tmp2 = _mm_unpackhi_epi32(descs_arr[3], descs_arr[2]);
+ sterr_tmp1 = _mm_unpackhi_epi32(descs_arr[1], descs_arr[0]);
+ sxe2_rx_desc_offloads_para_fill_sse(rxq, desc, descs_arr, rx_pkts);
+ mbuf_arr[3] = _mm_add_epi16(mbuf_arr[3], crc_adjust);
+ mbuf_arr[2] = _mm_add_epi16(mbuf_arr[2], crc_adjust);
+ mbuf_arr[1] = _mm_add_epi16(mbuf_arr[1], crc_adjust);
+ mbuf_arr[0] = _mm_add_epi16(mbuf_arr[0], crc_adjust);
+ staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
+ ptype_all = _mm_and_si128(staterr, ptype_mask);
+ _mm_storeu_si128((void *)&rx_pkts[i + 3]->rx_descriptor_fields1,
+ mbuf_arr[3]);
+ _mm_storeu_si128((void *)&rx_pkts[i + 2]->rx_descriptor_fields1,
+ mbuf_arr[2]);
+ if (umbcast_flags != NULL) {
+ const __m128i umbcast_mask =
+ _mm_set_epi32(SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK);
+ const __m128i umbcast_shuf_mask =
+ _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x07, 0x0F,
+ 0x03, 0x0B);
+ __m128i umbcast_bits = _mm_and_si128(staterr, umbcast_mask);
+ umbcast_bits = _mm_shuffle_epi8(umbcast_bits, umbcast_shuf_mask);
+ *(s32 *)umbcast_flags = _mm_cvtsi128_si32(umbcast_bits);
+ umbcast_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ if (split_rxe_flags != NULL) {
+ __m128i eop_bits = _mm_andnot_si128(staterr, eop_mask);
+ __m128i rxe_bits = _mm_and_si128(staterr, rxe_mask);
+ rxe_bits = _mm_srli_epi32(rxe_bits, 7);
+ eop_bits = _mm_or_si128(eop_bits, rxe_bits);
+ eop_bits = _mm_shuffle_epi8(eop_bits, eop_shuf_mask);
+ *(s32 *)split_rxe_flags = _mm_cvtsi128_si32(eop_bits);
+ split_rxe_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ staterr = _mm_and_si128(staterr, dd_mask);
+ staterr = _mm_packs_epi32(staterr, _mm_setzero_si128());
+ _mm_storeu_si128((void *)&rx_pkts[i + 1]->rx_descriptor_fields1,
+ mbuf_arr[1]);
+ _mm_storeu_si128((void *)&rx_pkts[i]->rx_descriptor_fields1,
+ mbuf_arr[0]);
+ rx_pkts[i + 3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[i + 2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
+ rx_pkts[i + 1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[i]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ bit_num = rte_popcount64(_mm_cvtsi128_si64(staterr));
+ done_num += bit_num;
+ if (likely(bit_num != SXE2_RX_NUM_PER_LOOP_SSE))
+ break;
+ }
+ rxq->processing_idx += done_num;
+ rxq->processing_idx &= (rxq->ring_depth - 1);
+ rxq->realloc_num += done_num;
+ PMD_LOG_DEBUG(RX, "port_id=%u queue_id=%u last_id=%u recv_pkts=%d",
+ rxq->port_id, rxq->queue_id, rxq->processing_idx, done_num);
+l_end:
+ return done_num;
+}
+static __rte_always_inline u16
+sxe2_rx_pkts_scattered_batch_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ const u64 *split_rxe_flags64;
+ u8 split_rxe_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ u8 umbcast_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ u16 rx_done_num;
+ u16 rx_pkt_done_num;
+ rx_pkt_done_num = 0;
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, umbcast_flags);
+ } else {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, NULL);
+ }
+ if (rx_done_num == 0)
+ goto l_end;
+ if (!rxq->vsi->adapter->devargs.sw_stats_en) {
+ split_rxe_flags64 = (u64 *)split_rxe_flags;
+ if (rxq->pkt_first_seg == NULL &&
+ split_rxe_flags64[0] == 0 &&
+ split_rxe_flags64[1] == 0 &&
+ split_rxe_flags64[2] == 0 &&
+ split_rxe_flags64[3] == 0) {
+ rx_pkt_done_num = rx_done_num;
+ goto l_end;
+ }
+ if (rxq->pkt_first_seg == NULL) {
+ while (rx_pkt_done_num < rx_done_num &&
+ split_rxe_flags[rx_pkt_done_num] == 0)
+ rx_pkt_done_num++;
+ if (rx_pkt_done_num == rx_done_num)
+ goto l_end;
+ rxq->pkt_first_seg = rx_pkts[rx_pkt_done_num];
+ }
+ }
+ rx_pkt_done_num += sxe2_rx_pkts_refactor(rxq, &rx_pkts[rx_pkt_done_num],
+ rx_done_num - rx_pkt_done_num, &split_rxe_flags[rx_pkt_done_num],
+ &umbcast_flags[rx_pkt_done_num]);
+l_end:
+ return rx_pkt_done_num;
+}
+
+u16 sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ u16 done_num = 0;
+ u16 once_num;
+ while (nb_pkts > SXE2_RX_PKTS_BURST_BATCH_NUM_VEC) {
+ once_num =
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num,
+ SXE2_RX_PKTS_BURST_BATCH_NUM_VEC);
+ done_num += once_num;
+ nb_pkts -= once_num;
+ if (once_num < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC)
+ goto l_end;
+ }
+ done_num +=
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num, nb_pkts);
+l_end:
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
* [PATCH v13 08/10] net/sxe2: support queue setup and control
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Add support for Rx and Tx queue setup, release, and management.
Implement eth_dev_ops callbacks for rx_queue_setup, tx_queue_setup,
rx_queue_release, and tx_queue_release.
This includes:
- Allocating memory for hardware ring descriptors.
- Initializing software ring structures and hardware head/tail pointers.
- Implementing proper resource cleanup logic to prevent memory leaks
during queue reconfiguration or device close.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/meson.build | 2 +
drivers/net/sxe2/sxe2_drv_cmd.h | 9 -
drivers/net/sxe2/sxe2_ethdev.c | 66 +++-
drivers/net/sxe2/sxe2_ethdev.h | 3 +
drivers/net/sxe2/sxe2_rx.c | 579 ++++++++++++++++++++++++++++++++
drivers/net/sxe2/sxe2_rx.h | 34 ++
drivers/net/sxe2/sxe2_tx.c | 447 ++++++++++++++++++++++++
drivers/net/sxe2/sxe2_tx.h | 32 ++
8 files changed, 1145 insertions(+), 27 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_rx.c
create mode 100644 drivers/net/sxe2/sxe2_rx.h
create mode 100644 drivers/net/sxe2/sxe2_tx.c
create mode 100644 drivers/net/sxe2/sxe2_tx.h
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 6c9a86423a..8638244d80 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -16,6 +16,8 @@ sources += files(
'sxe2_cmd_chnl.c',
'sxe2_vsi.c',
'sxe2_queue.c',
+ 'sxe2_tx.c',
+ 'sxe2_rx.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_drv_cmd.h b/drivers/net/sxe2/sxe2_drv_cmd.h
index 4094442077..f236e30c40 100644
--- a/drivers/net/sxe2/sxe2_drv_cmd.h
+++ b/drivers/net/sxe2/sxe2_drv_cmd.h
@@ -5,17 +5,8 @@
#ifndef __SXE2_DRV_CMD_H__
#define __SXE2_DRV_CMD_H__
-#ifdef SXE2_DPDK_DRIVER
#include "sxe2_type.h"
#define SXE2_DPDK_RESOURCE_INSUFFICIENT
-#endif
-
-#ifdef SXE2_LINUX_DRIVER
-#ifdef __KERNEL__
-#include <linux/types.h>
-#include <linux/if_ether.h>
-#endif
-#endif
#define SXE2_DRV_CMD_MODULE_S (16)
#define SXE2_MK_DRV_CMD(module, cmd) (((module) << SXE2_DRV_CMD_MODULE_S) | ((cmd) & 0xFFFF))
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 4836c338bc..2a07c211bf 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -24,6 +24,8 @@
#include "sxe2_ethdev.h"
#include "sxe2_drv_cmd.h"
#include "sxe2_cmd_chnl.h"
+#include "sxe2_tx.h"
+#include "sxe2_rx.h"
#include "sxe2_common.h"
#include "sxe2_common_log.h"
#include "sxe2_host_regs.h"
@@ -80,14 +82,6 @@ static s32 sxe2_dev_configure(struct rte_eth_dev *dev)
return ret;
}
-static void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev __rte_unused)
-{
-}
-
-static void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev __rte_unused)
-{
-}
-
static s32 sxe2_dev_stop(struct rte_eth_dev *dev)
{
s32 ret = SXE2_SUCCESS;
@@ -106,16 +100,6 @@ static s32 sxe2_dev_stop(struct rte_eth_dev *dev)
return ret;
}
-static s32 __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev __rte_unused)
-{
- return 0;
-}
-
-static s32 __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev __rte_unused)
-{
- return 0;
-}
-
static s32 sxe2_queues_start(struct rte_eth_dev *dev)
{
s32 ret = SXE2_SUCCESS;
@@ -301,6 +285,14 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_stop = sxe2_dev_stop,
.dev_close = sxe2_dev_close,
.dev_infos_get = sxe2_dev_infos_get,
+
+ .rx_queue_setup = sxe2_rx_queue_setup,
+ .tx_queue_setup = sxe2_tx_queue_setup,
+ .rx_queue_release = sxe2_rx_queue_release,
+ .tx_queue_release = sxe2_tx_queue_release,
+
+ .rxq_info_get = sxe2_rx_queue_info_get,
+ .txq_info_get = sxe2_tx_queue_info_get,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
@@ -328,6 +320,44 @@ struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter
return bar_info;
}
+void __iomem *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type, u16 idx_in_func)
+{
+ struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
+ struct sxe2_pci_map_segment_info *seg_info = NULL;
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ void __iomem *addr = NULL;
+ u8 reg_width = 0;
+ u8 i = 0;
+
+ bar_info = sxe2_dev_get_bar_info(adapter, res_type);
+ if (bar_info == NULL) {
+ PMD_DEV_LOG_WARN(adapter, INIT, "Failed to get bar info, res_type=[%d]",
+ res_type);
+ goto l_end;
+ }
+ seg_info = bar_info->seg_info;
+
+ reg_width = map_ctxt->addr_info[res_type].reg_width;
+ if (reg_width == 0) {
+ PMD_DEV_LOG_WARN(adapter, INIT, "Invalid reg width with resource type %d",
+ res_type);
+ goto l_end;
+ }
+
+ for (i = 0; i < bar_info->map_cnt; i++) {
+ seg_info = &bar_info->seg_info[i];
+ if (res_type == seg_info->type) {
+ addr = (void __iomem *)((uintptr_t)seg_info->addr +
+ seg_info->page_inner_offset + reg_width * idx_in_func);
+ goto l_end;
+ }
+ }
+
+l_end:
+ return addr;
+}
+
static void sxe2_drv_dev_caps_set(struct sxe2_adapter *adapter,
struct sxe2_drv_dev_caps_resp *dev_caps)
{
diff --git a/drivers/net/sxe2/sxe2_ethdev.h b/drivers/net/sxe2/sxe2_ethdev.h
index 698e2ee4a2..4ef7854479 100644
--- a/drivers/net/sxe2/sxe2_ethdev.h
+++ b/drivers/net/sxe2/sxe2_ethdev.h
@@ -295,6 +295,9 @@ struct sxe2_adapter {
#define SXE2_DEV_TO_PCI(eth_dev) \
RTE_DEV_TO_PCI((eth_dev)->device)
+void __iomem *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type, u16 idx_in_func);
+
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
enum sxe2_pci_map_resource res_type);
diff --git a/drivers/net/sxe2/sxe2_rx.c b/drivers/net/sxe2/sxe2_rx.c
new file mode 100644
index 0000000000..6b42297382
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_rx.c
@@ -0,0 +1,579 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+
+#include "sxe2_ethdev.h"
+#include "sxe2_queue.h"
+#include "sxe2_rx.h"
+#include "sxe2_cmd_chnl.h"
+
+#include "sxe2_osal.h"
+#include "sxe2_common_log.h"
+
+static void __iomem *sxe2_rx_doorbell_tail_addr_get(struct sxe2_adapter *adapter, u16 queue_id)
+{
+ return sxe2_pci_map_addr_get(adapter, SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL, queue_id);
+}
+
+static void sxe2_rx_head_tail_init(struct sxe2_adapter *adapter, struct sxe2_rx_queue *rxq)
+{
+ rxq->rdt_reg_addr = sxe2_rx_doorbell_tail_addr_get(adapter, rxq->queue_id);
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, 0);
+}
+
+static void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq)
+{
+ u16 i = 0;
+ u16 len = 0;
+ static const union sxe2_rx_desc zeroed_desc = {{0}};
+
+ len = rxq->ring_depth + SXE2_RX_PKTS_BURST_BATCH_NUM;
+ for (i = 0; i < len; ++i)
+ rxq->desc_ring[i] = zeroed_desc;
+
+ memset(&rxq->fake_mbuf, 0, sizeof(rxq->fake_mbuf));
+ for (i = rxq->ring_depth; i < len; i++)
+ rxq->buffer_ring[i] = &rxq->fake_mbuf;
+
+ rxq->hold_num = 0;
+ rxq->next_ret_pkt = 0;
+ rxq->processing_idx = 0;
+ rxq->completed_pkts_num = 0;
+ rxq->batch_alloc_trigger = rxq->rx_free_thresh - 1;
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+
+ rxq->realloc_num = 0;
+ rxq->realloc_start = 0;
+}
+
+void __rte_cold sxe2_rx_queue_mbufs_release(struct sxe2_rx_queue *rxq)
+{
+ u16 i;
+
+ if (rxq->buffer_ring != NULL) {
+ for (i = 0; i < rxq->ring_depth; i++) {
+ if (rxq->buffer_ring[i] != NULL) {
+ rte_pktmbuf_free(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ }
+
+ if (rxq->completed_pkts_num) {
+ for (i = 0; i < rxq->completed_pkts_num; ++i) {
+ if (rxq->completed_buf[rxq->next_ret_pkt + i] != NULL) {
+ rte_pktmbuf_free(rxq->completed_buf[rxq->next_ret_pkt + i]);
+ rxq->completed_buf[rxq->next_ret_pkt + i] = NULL;
+ }
+ }
+ rxq->completed_pkts_num = 0;
+ }
+}
+
+const struct sxe2_rxq_ops sxe2_default_rxq_ops = {
+ .queue_reset = sxe2_rx_queue_reset,
+ .mbufs_release = sxe2_rx_queue_mbufs_release,
+};
+
+static struct sxe2_rxq_ops sxe2_rx_default_ops_get(void)
+{
+ return sxe2_default_rxq_ops;
+}
+
+void __rte_cold sxe2_rx_queue_info_get(struct rte_eth_dev *dev,
+ u16 queue_id, struct rte_eth_rxq_info *qinfo)
+{
+ struct sxe2_rx_queue *rxq = NULL;
+
+ if (queue_id >= dev->data->nb_rx_queues) {
+ PMD_LOG_ERR(RX, "rx queue:%u is out of range:%u",
+ queue_id, dev->data->nb_rx_queues);
+ goto end;
+ }
+
+ rxq = dev->data->rx_queues[queue_id];
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue:%u is NULL", queue_id);
+ goto end;
+ }
+
+ qinfo->mp = rxq->mb_pool;
+ qinfo->nb_desc = rxq->ring_depth;
+ qinfo->scattered_rx = dev->data->scattered_rx;
+ qinfo->conf.rx_free_thresh = rxq->rx_free_thresh;
+ qinfo->conf.rx_drop_en = rxq->drop_en;
+ qinfo->conf.rx_deferred_start = rxq->rx_deferred_start;
+
+end:
+ return;
+}
+
+s32 __rte_cold sxe2_rx_queue_stop(struct rte_eth_dev *dev, u16 rx_queue_id)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_rx_queue *rxq;
+ s32 ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues) {
+ PMD_LOG_ERR(RX, "Rx queue %u is out of range %u",
+ rx_queue_id, dev->data->nb_rx_queues);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (dev->data->rx_queue_state[rx_queue_id] ==
+ RTE_ETH_QUEUE_STATE_STOPPED) {
+ ret = SXE2_SUCCESS;
+ goto l_end;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (rxq == NULL) {
+ ret = SXE2_SUCCESS;
+ goto l_end;
+ }
+ ret = sxe2_drv_rxq_switch(adapter, rxq, false);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Failed to switch rx queue %u off, ret = %d",
+ rx_queue_id, ret);
+ if (ret == -EPERM)
+ goto l_free;
+ goto l_end;
+ }
+
+l_free:
+ rxq->ops.mbufs_release(rxq);
+ rxq->ops.queue_reset(rxq);
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STOPPED;
+l_end:
+ return ret;
+}
+
+static void __rte_cold sxe2_rx_queue_free(struct sxe2_rx_queue *rxq)
+{
+ if (rxq != NULL) {
+ rxq->ops.mbufs_release(rxq);
+ if (rxq->buffer_ring != NULL) {
+ rte_free(rxq->buffer_ring);
+ rxq->buffer_ring = NULL;
+ }
+ rte_memzone_free(rxq->mz);
+ rte_free(rxq);
+ }
+}
+
+void __rte_cold sxe2_rx_queue_release(struct rte_eth_dev *dev,
+ u16 queue_idx)
+{
+ (void)sxe2_rx_queue_stop(dev, queue_idx);
+ sxe2_rx_queue_free(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+}
+
+void __rte_cold sxe2_all_rxqs_release(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ u16 nb_rxq;
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ if (data->rx_queues[nb_rxq] == NULL)
+ continue;
+ sxe2_rx_queue_release(dev, nb_rxq);
+ data->rx_queues[nb_rxq] = NULL;
+ }
+ data->nb_rx_queues = 0;
+}
+
+static struct sxe2_rx_queue *sxe2_rx_queue_alloc(struct rte_eth_dev *dev, u16 queue_idx,
+ u16 ring_depth, u32 socket_id)
+{
+ struct sxe2_rx_queue *rxq;
+ const struct rte_memzone *tz;
+ u16 len;
+
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ sxe2_rx_queue_release(dev, queue_idx);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ rxq = rte_zmalloc_socket("rx_queue", sizeof(*rxq),
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue[%d] alloc failed", queue_idx);
+ goto l_end;
+ }
+
+ rxq->ring_depth = ring_depth;
+ len = rxq->ring_depth + SXE2_RX_PKTS_BURST_BATCH_NUM;
+
+ rxq->buffer_ring = rte_zmalloc_socket("rx_buffer_ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (!rxq->buffer_ring) {
+ PMD_LOG_ERR(RX, "Rxq malloc mbuf mem failed");
+ rte_free(rxq);
+ rxq = NULL;
+ goto l_end;
+ }
+
+ tz = rte_eth_dma_zone_reserve(dev, "rx_dma", queue_idx,
+ SXE2_RX_RING_SIZE, SXE2_DESC_ADDR_ALIGN, socket_id);
+ if (tz == NULL) {
+ PMD_LOG_ERR(RX, "Rxq malloc desc mem failed");
+ rte_free(rxq->buffer_ring);
+ rxq->buffer_ring = NULL;
+ rte_free(rxq);
+ rxq = NULL;
+ goto l_end;
+ }
+
+ rxq->mz = tz;
+ memset(tz->addr, 0, SXE2_RX_RING_SIZE);
+ rxq->base_addr = tz->iova;
+ rxq->desc_ring = (union sxe2_rx_desc *)tz->addr;
+
+l_end:
+ return rxq;
+}
+
+s32 __rte_cold sxe2_rx_queue_setup(struct rte_eth_dev *dev,
+ u16 queue_idx, u16 nb_desc, u32 socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ struct sxe2_rx_queue *rxq;
+ u64 offloads;
+ s32 ret;
+ u16 rx_nseg;
+ u16 i;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (queue_idx >= dev->data->nb_rx_queues) {
+ PMD_LOG_ERR(RX, "Rx queue %u is out of range %u",
+ queue_idx, dev->data->nb_rx_queues);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (nb_desc % SXE2_RX_DESC_RING_ALIGN != 0 ||
+ nb_desc > SXE2_MAX_RING_DESC ||
+ nb_desc < SXE2_MIN_RING_DESC) {
+ PMD_LOG_ERR(RX, "param desc num:%u is invalid", nb_desc);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (mp != NULL)
+ rx_nseg = 1;
+ else
+ rx_nseg = rx_conf->rx_nseg;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (rx_nseg > 1 && !(offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ PMD_LOG_ERR(RX, "Port %u queue %u Buffer split offload not configured, but rx_nseg is %u",
+ dev->data->port_id, queue_idx, rx_nseg);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) && !(rx_nseg > 1)) {
+ PMD_LOG_ERR(RX, "Port %u queue %u Buffer split offload configured, but rx_nseg is %u",
+ dev->data->port_id, queue_idx, rx_nseg);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO) &&
+ (offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)) {
+ PMD_LOG_ERR(RX, "port_id %u queue %u, LRO can't be configure with Keep crc.",
+ dev->data->port_id, queue_idx);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ rxq = sxe2_rx_queue_alloc(dev, queue_idx, nb_desc, socket_id);
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue[%d] resource alloc failed", queue_idx);
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ if (offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
+ dev->data->lro = 1;
+
+ if (rx_nseg > 1) {
+ for (i = 0; i < rx_nseg; i++) {
+ rte_memcpy(&rxq->rx_seg[i], &rx_conf->rx_seg[i].split,
+ sizeof(struct rte_eth_rxseg_split));
+ }
+ rxq->mb_pool = rxq->rx_seg[0].mp;
+ } else {
+ rxq->mb_pool = mp;
+ }
+
+ rxq->rx_free_thresh = rx_conf->rx_free_thresh;
+ rxq->port_id = dev->data->port_id;
+ rxq->offloads = offloads;
+ if (offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ rxq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ rxq->crc_len = 0;
+
+ rxq->queue_id = queue_idx;
+ rxq->idx_in_func = vsi->rxqs.base_idx_in_func + queue_idx;
+ rxq->drop_en = rx_conf->rx_drop_en;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->vsi = vsi;
+ rxq->ops = sxe2_rx_default_ops_get();
+ rxq->ops.queue_reset(rxq);
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ ret = SXE2_SUCCESS;
+l_end:
+ return ret;
+}
+
+struct rte_mbuf *sxe2_mbuf_raw_alloc(struct rte_mempool *mp)
+{
+ return rte_mbuf_raw_alloc(mp);
+}
+
+static s32 __rte_cold sxe2_rx_queue_mbufs_alloc(struct sxe2_rx_queue *rxq)
+{
+ struct rte_mbuf **buf_ring = rxq->buffer_ring;
+ struct rte_mbuf *mbuf = NULL;
+ struct rte_mbuf *mbuf_pay;
+ volatile union sxe2_rx_desc *desc;
+ u64 dma_addr;
+ s32 ret;
+ u16 i, j;
+
+ for (i = 0; i < rxq->ring_depth; i++) {
+ mbuf = sxe2_mbuf_raw_alloc(rxq->mb_pool);
+ if (mbuf == NULL) {
+ PMD_LOG_ERR(RX, "Rx queue is not available or setup");
+ ret = -ENOMEM;
+ goto l_err_free_mbuf;
+ }
+
+ buf_ring[i] = mbuf;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+ desc = &rxq->desc_ring[i];
+ if (!(rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ mbuf->next = NULL;
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr = dma_addr;
+ } else {
+ mbuf_pay = rte_mbuf_raw_alloc(rxq->rx_seg[1].mp);
+ if (unlikely(!mbuf_pay)) {
+ PMD_LOG_ERR(RX, "Failed to allocate payload mbuf for RX");
+ ret = -ENOMEM;
+ goto l_err_free_mbuf;
+ }
+
+ mbuf_pay->next = NULL;
+ mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf_pay->nb_segs = 1;
+ mbuf_pay->port = rxq->port_id;
+ mbuf->next = mbuf_pay;
+
+ desc->read.hdr_addr = dma_addr;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf_pay));
+ }
+
+#ifndef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+ desc->read.rsvd1 = 0;
+ desc->read.rsvd2 = 0;
+#endif
+ }
+
+ ret = SXE2_SUCCESS;
+ goto l_end;
+
+l_err_free_mbuf:
+ for (j = 0; j <= i; j++) {
+ if (buf_ring[j] != NULL && buf_ring[j]->next != NULL) {
+ rte_pktmbuf_free(buf_ring[j]->next);
+ buf_ring[j]->next = NULL;
+ }
+
+ if (buf_ring[j] != NULL) {
+ rte_pktmbuf_free(buf_ring[j]);
+ buf_ring[j] = NULL;
+ }
+ }
+
+l_end:
+ return ret;
+}
+
+s32 __rte_cold sxe2_rx_queue_start(struct rte_eth_dev *dev, u16 rx_queue_id)
+{
+ struct sxe2_rx_queue *rxq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ s32 ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (rx_queue_id >= dev->data->nb_rx_queues) {
+ PMD_LOG_ERR(RX, "Rx queue %u is out of range %u",
+ rx_queue_id, dev->data->nb_rx_queues);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "Rx queue %u is not available or setup",
+ rx_queue_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (dev->data->rx_queue_state[rx_queue_id] ==
+ RTE_ETH_QUEUE_STATE_STARTED) {
+ ret = SXE2_SUCCESS;
+ goto l_end;
+ }
+
+ ret = sxe2_rx_queue_mbufs_alloc(rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Rx queue %u apply desc ring fail",
+ rx_queue_id);
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ sxe2_rx_head_tail_init(adapter, rxq);
+
+ ret = sxe2_drv_rxq_ctxt_cfg(adapter, rxq, 1);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Rx queue %u config ctxt fail, ret=%d",
+ rx_queue_id, ret);
+
+ (void)sxe2_drv_rxq_switch(adapter, rxq, false);
+ rxq->ops.mbufs_release(rxq);
+ rxq->ops.queue_reset(rxq);
+ goto l_end;
+ }
+
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, rxq->ring_depth - 1);
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+
+l_end:
+ return ret;
+}
+
+s32 __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_rx_queue *rxq;
+ u16 nb_rxq;
+ u16 nb_started_rxq;
+ s32 ret;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ rxq = dev->data->rx_queues[nb_rxq];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+
+ ret = sxe2_rx_queue_start(dev, nb_rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Fail to start rx queue %u", nb_rxq);
+ goto l_free_started_queue;
+ }
+
+ rte_atomic_store_explicit(&rxq->sw_stats.pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.bytes, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.drop_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.drop_bytes, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.unicast_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.broadcast_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.multicast_pkts, 0,
+ rte_memory_order_relaxed);
+ }
+ ret = SXE2_SUCCESS;
+ goto l_end;
+
+l_free_started_queue:
+ for (nb_started_rxq = 0; nb_started_rxq <= nb_rxq; nb_started_rxq++)
+ (void)sxe2_rx_queue_stop(dev, nb_started_rxq);
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ struct sxe2_stats *sw_stats_prev = &vsi->vsi_stats.vsi_sw_stats_prev;
+ struct sxe2_rx_queue *rxq = NULL;
+ s32 ret;
+ u16 nb_rxq;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ ret = sxe2_rx_queue_stop(dev, nb_rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Fail to start rx queue %u", nb_rxq);
+ continue;
+ }
+
+ rxq = dev->data->rx_queues[nb_rxq];
+ if (rxq) {
+ sw_stats_prev->ipackets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->ierrors +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->ibytes +=
+ rte_atomic_load_explicit(&rxq->sw_stats.bytes,
+ rte_memory_order_relaxed);
+
+ sw_stats_prev->rx_sw_unicast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.unicast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_broadcast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.broadcast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_multicast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.multicast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_drop_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_drop_bytes +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_bytes,
+ rte_memory_order_relaxed);
+ }
+ }
+}
diff --git a/drivers/net/sxe2/sxe2_rx.h b/drivers/net/sxe2/sxe2_rx.h
new file mode 100644
index 0000000000..7c6239b387
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_rx.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_RX_H__
+#define __SXE2_RX_H__
+
+#include "sxe2_queue.h"
+
+s32 __rte_cold sxe2_rx_queue_setup(struct rte_eth_dev *dev,
+ u16 queue_idx, u16 nb_desc, u32 socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+
+s32 __rte_cold sxe2_rx_queue_stop(struct rte_eth_dev *dev, u16 rx_queue_id);
+
+void __rte_cold sxe2_rx_queue_mbufs_release(struct sxe2_rx_queue *rxq);
+
+void __rte_cold sxe2_rx_queue_release(struct rte_eth_dev *dev, u16 queue_idx);
+
+void __rte_cold sxe2_all_rxqs_release(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rx_queue_info_get(struct rte_eth_dev *dev, u16 queue_id,
+ struct rte_eth_rxq_info *qinfo);
+
+s32 __rte_cold sxe2_rx_queue_start(struct rte_eth_dev *dev, u16 rx_queue_id);
+
+s32 __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev);
+
+struct rte_mbuf *sxe2_mbuf_raw_alloc(struct rte_mempool *mp);
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_tx.c b/drivers/net/sxe2/sxe2_tx.c
new file mode 100644
index 0000000000..b043611c8d
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_tx.c
@@ -0,0 +1,447 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include "sxe2_tx.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+#include "sxe2_cmd_chnl.h"
+
+static void __iomem *sxe2_tx_doorbell_addr_get(struct sxe2_adapter *adapter, u16 queue_id)
+{
+ return sxe2_pci_map_addr_get(adapter, SXE2_PCI_MAP_RES_DOORBELL_TX, queue_id);
+}
+
+static void sxe2_tx_tail_init(struct sxe2_adapter *adapter, struct sxe2_tx_queue *txq)
+{
+ txq->tdt_reg_addr = sxe2_tx_doorbell_addr_get(adapter, txq->queue_id);
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, 0);
+}
+
+void __rte_cold sxe2_tx_queue_reset(struct sxe2_tx_queue *txq)
+{
+ u16 prev, i;
+ volatile union sxe2_tx_data_desc *txd;
+ static const union sxe2_tx_data_desc zeroed_desc = {{0}};
+ struct sxe2_tx_buffer *tx_buffer = txq->buffer_ring;
+
+ for (i = 0; i < txq->ring_depth; i++)
+ txq->desc_ring[i] = zeroed_desc;
+
+ prev = txq->ring_depth - 1;
+ for (i = 0; i < txq->ring_depth; i++) {
+ txd = &txq->desc_ring[i];
+ if (txd == NULL)
+ continue;
+
+ txd->wb.dd = rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE);
+ tx_buffer[i].mbuf = NULL;
+ tx_buffer[i].last_id = i;
+ tx_buffer[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->desc_used_num = 0;
+ txq->desc_free_num = txq->ring_depth - 1;
+ txq->next_use = 0;
+ txq->next_clean = txq->ring_depth - 1;
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+void __rte_cold sxe2_tx_queue_mbufs_release(struct sxe2_tx_queue *txq)
+{
+ u32 i;
+
+ if (txq != NULL && txq->buffer_ring != NULL) {
+ for (i = 0; i < txq->ring_depth; i++) {
+ if (txq->buffer_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->buffer_ring[i].mbuf);
+ txq->buffer_ring[i].mbuf = NULL;
+ }
+ }
+ }
+}
+
+static void sxe2_tx_buffer_ring_free(struct sxe2_tx_queue *txq)
+{
+ if (txq != NULL && txq->buffer_ring != NULL)
+ rte_free(txq->buffer_ring);
+}
+
+const struct sxe2_txq_ops sxe2_default_txq_ops = {
+ .queue_reset = sxe2_tx_queue_reset,
+ .mbufs_release = sxe2_tx_queue_mbufs_release,
+ .buffer_ring_free = sxe2_tx_buffer_ring_free,
+};
+
+static struct sxe2_txq_ops sxe2_tx_default_ops_get(void)
+{
+ return sxe2_default_txq_ops;
+}
+
+static s32 sxe2_txq_arg_validate(struct rte_eth_dev *dev, u16 ring_depth,
+ u16 *rs_thresh, u16 *free_thresh, const struct rte_eth_txconf *tx_conf)
+{
+ s32 ret = SXE2_SUCCESS;
+
+ if ((ring_depth % SXE2_TX_DESC_RING_ALIGN) != 0 ||
+ ring_depth > SXE2_MAX_RING_DESC ||
+ ring_depth < SXE2_MIN_RING_DESC) {
+ PMD_LOG_ERR(TX, "number:%u of receive descriptors is invalid", ring_depth);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ *free_thresh = (u16)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : DEFAULT_TX_FREE_THRESH);
+ *rs_thresh = (u16)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : DEFAULT_TX_RS_THRESH);
+
+ if (*rs_thresh >= (ring_depth - 2)) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be less than the number "
+ "of tx descriptors minus 2. (tx_rs_thresh:%u port:%u)",
+ *rs_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (*free_thresh >= (ring_depth - 3)) {
+ PMD_LOG_ERR(TX, "tx_free_thresh must be less than the number "
+ "of tx descriptors minus 3. (tx_free_thresh:%u port:%u)",
+ *free_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (*rs_thresh > *free_thresh) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be less than or equal to "
+ "tx_free_thresh. (tx_free_thresh:%u tx_rs_thresh:%u port:%u)",
+ *free_thresh, *rs_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((ring_depth % *rs_thresh) != 0) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be a divisor of the "
+ "number of tx descriptors. (tx_rs_thresh:%u port:%d ring_depth:%u)",
+ *rs_thresh, dev->data->port_id, ring_depth);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ ret = SXE2_SUCCESS;
+
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_tx_queue_info_get(struct rte_eth_dev *dev, u16 queue_id,
+ struct rte_eth_txq_info *qinfo)
+{
+ struct sxe2_tx_queue *txq = NULL;
+
+ if (queue_id >= dev->data->nb_tx_queues) {
+ PMD_LOG_ERR(TX, "tx queue:%u is out of range:%u",
+ queue_id, dev->data->nb_tx_queues);
+ goto end;
+ }
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ PMD_LOG_WARN(TX, "tx queue:%u is NULL", queue_id);
+ goto end;
+ }
+
+ qinfo->nb_desc = txq->ring_depth;
+
+ qinfo->conf.tx_thresh.pthresh = txq->pthresh;
+ qinfo->conf.tx_thresh.hthresh = txq->hthresh;
+ qinfo->conf.tx_thresh.wthresh = txq->wthresh;
+ qinfo->conf.tx_free_thresh = txq->free_thresh;
+ qinfo->conf.tx_rs_thresh = txq->rs_thresh;
+ qinfo->conf.offloads = txq->offloads;
+ qinfo->conf.tx_deferred_start = txq->tx_deferred_start;
+
+end:
+ return;
+}
+
+s32 __rte_cold sxe2_tx_queue_stop(struct rte_eth_dev *dev, u16 queue_id)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_tx_queue *txq;
+ s32 ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (queue_id >= dev->data->nb_tx_queues) {
+ PMD_LOG_ERR(TX, "tx queue:%u is out of range:%u",
+ queue_id, dev->data->nb_tx_queues);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (dev->data->tx_queue_state[queue_id] ==
+ RTE_ETH_QUEUE_STATE_STOPPED) {
+ ret = SXE2_SUCCESS;
+ goto l_end;
+ }
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ ret = SXE2_SUCCESS;
+ goto l_end;
+ }
+
+ ret = sxe2_drv_txq_switch(adapter, txq, false);
+ if (ret) {
+ PMD_LOG_ERR(TX, "Failed to switch tx queue %u off",
+ queue_id);
+ goto l_end;
+ }
+
+ txq->ops.mbufs_release(txq);
+ txq->ops.queue_reset(txq);
+ dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+ ret = SXE2_SUCCESS;
+
+l_end:
+ return ret;
+}
+
+static void __rte_cold sxe2_tx_queue_free(struct sxe2_tx_queue *txq)
+{
+ if (txq != NULL) {
+ txq->ops.mbufs_release(txq);
+ txq->ops.buffer_ring_free(txq);
+
+ rte_memzone_free(txq->mz);
+ rte_free(txq);
+ }
+}
+
+void __rte_cold sxe2_tx_queue_release(struct rte_eth_dev *dev, u16 queue_idx)
+{
+ (void)sxe2_tx_queue_stop(dev, queue_idx);
+ sxe2_tx_queue_free(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+}
+
+void __rte_cold sxe2_all_txqs_release(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ u16 nb_txq;
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ if (data->tx_queues[nb_txq] == NULL)
+ continue;
+
+ sxe2_tx_queue_release(dev, nb_txq);
+ data->tx_queues[nb_txq] = NULL;
+ }
+ data->nb_tx_queues = 0;
+}
+
+static struct sxe2_tx_queue
+*sxe2_tx_queue_alloc(struct rte_eth_dev *dev, u16 queue_idx,
+ u16 ring_depth, u32 socket_id)
+{
+ struct sxe2_tx_queue *txq;
+ const struct rte_memzone *tz;
+
+ if (dev->data->tx_queues[queue_idx]) {
+ sxe2_tx_queue_release(dev, queue_idx);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ txq = rte_zmalloc_socket("tx_queue", sizeof(struct sxe2_tx_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "tx queue:%d alloc failed", queue_idx);
+ goto l_end;
+ }
+
+ tz = rte_eth_dma_zone_reserve(dev, "tx_dma", queue_idx,
+ sizeof(union sxe2_tx_data_desc) * SXE2_MAX_RING_DESC,
+ SXE2_DESC_ADDR_ALIGN, socket_id);
+ if (tz == NULL) {
+ PMD_LOG_ERR(TX, "tx desc ring alloc failed, queue_id:%d", queue_idx);
+ rte_free(txq);
+ txq = NULL;
+ goto l_end;
+ }
+
+ txq->buffer_ring = rte_zmalloc_socket("tx_buffer_ring",
+ sizeof(struct sxe2_tx_buffer) * ring_depth,
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq->buffer_ring == NULL) {
+ PMD_LOG_ERR(TX, "tx buffer alloc failed, queue_id:%d", queue_idx);
+ rte_memzone_free(tz);
+ rte_free(txq);
+ txq = NULL;
+ goto l_end;
+ }
+
+ txq->mz = tz;
+ txq->base_addr = tz->iova;
+ txq->desc_ring = (volatile union sxe2_tx_data_desc *)tz->addr;
+
+l_end:
+ return txq;
+}
+
+s32 __rte_cold sxe2_tx_queue_setup(struct rte_eth_dev *dev,
+ u16 queue_idx, u16 nb_desc, u32 socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ s32 ret = SXE2_SUCCESS;
+ u16 tx_rs_thresh;
+ u16 tx_free_thresh;
+ struct sxe2_tx_queue *txq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ u64 offloads;
+ PMD_INIT_FUNC_TRACE();
+
+ if (queue_idx >= dev->data->nb_tx_queues) {
+ PMD_LOG_ERR(TX, "tx queue:%u is out of range:%u",
+ queue_idx, dev->data->nb_tx_queues);
+ ret = -EINVAL;
+ goto end;
+ }
+
+ ret = sxe2_txq_arg_validate(dev, nb_desc, &tx_rs_thresh, &tx_free_thresh, tx_conf);
+ if (ret) {
+ PMD_LOG_ERR(TX, "tx queue:%u arg validate failed", queue_idx);
+ goto end;
+ }
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ txq = sxe2_tx_queue_alloc(dev, queue_idx, nb_desc, socket_id);
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "failed to alloc sxe2vf tx queue:%u resource", queue_idx);
+ ret = -ENOMEM;
+ goto end;
+ }
+
+ txq->vlan_flag = SXE2_TX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ txq->ring_depth = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->pthresh = tx_conf->tx_thresh.pthresh;
+ txq->hthresh = tx_conf->tx_thresh.hthresh;
+ txq->wthresh = tx_conf->tx_thresh.wthresh;
+ txq->queue_id = queue_idx;
+ txq->idx_in_func = vsi->txqs.base_idx_in_func + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+ txq->vsi = vsi;
+ txq->ops = sxe2_tx_default_ops_get();
+ txq->ops.queue_reset(txq);
+
+ dev->data->tx_queues[queue_idx] = txq;
+ ret = SXE2_SUCCESS;
+
+end:
+ return ret;
+}
+
+s32 __rte_cold sxe2_tx_queue_start(struct rte_eth_dev *dev, u16 queue_id)
+{
+ s32 ret = SXE2_SUCCESS;
+ struct sxe2_tx_queue *txq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ PMD_INIT_FUNC_TRACE();
+
+ if (queue_id >= dev->data->nb_tx_queues) {
+ PMD_LOG_ERR(TX, "tx queue:%u is out of range:%u",
+ queue_id, dev->data->nb_tx_queues);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (dev->data->tx_queue_state[queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
+ ret = SXE2_SUCCESS;
+ goto l_end;
+ }
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "tx queue:%u is not available or setup", queue_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ ret = sxe2_drv_txq_ctxt_cfg(adapter, txq, 1);
+ if (ret) {
+ PMD_LOG_ERR(TX, "tx queue:%u config ctxt fail", queue_id);
+
+ (void)sxe2_drv_txq_switch(adapter, txq, false);
+ txq->ops.mbufs_release(txq);
+ txq->ops.queue_reset(txq);
+ goto l_end;
+ }
+
+ sxe2_tx_tail_init(adapter, txq);
+
+ dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+ ret = SXE2_SUCCESS;
+
+l_end:
+ return ret;
+}
+
+s32 __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_tx_queue *txq;
+ u16 nb_txq;
+ u16 nb_started_txq;
+ s32 ret;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ txq = dev->data->tx_queues[nb_txq];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+
+ ret = sxe2_tx_queue_start(dev, nb_txq);
+ if (ret) {
+ PMD_LOG_ERR(TX, "Fail to start tx queue %u", nb_txq);
+ goto l_free_started_queue;
+ }
+ }
+ ret = SXE2_SUCCESS;
+ goto l_end;
+
+l_free_started_queue:
+ for (nb_started_txq = 0; nb_started_txq <= nb_txq; nb_started_txq++)
+ (void)sxe2_tx_queue_stop(dev, nb_started_txq);
+
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ u16 nb_txq;
+ s32 ret;
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ ret = sxe2_tx_queue_stop(dev, nb_txq);
+ if (ret) {
+ PMD_LOG_WARN(TX, "Fail to stop tx queue %u", nb_txq);
+ continue;
+ }
+ }
+}
diff --git a/drivers/net/sxe2/sxe2_tx.h b/drivers/net/sxe2/sxe2_tx.h
new file mode 100644
index 0000000000..58b668e337
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_tx.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TX_H__
+#define __SXE2_TX_H__
+#include "sxe2_queue.h"
+
+void __rte_cold sxe2_tx_queue_reset(struct sxe2_tx_queue *txq);
+
+s32 __rte_cold sxe2_tx_queue_start(struct rte_eth_dev *dev, u16 queue_id);
+
+void sxe2_tx_queue_mbufs_release(struct sxe2_tx_queue *txq);
+
+s32 __rte_cold sxe2_tx_queue_stop(struct rte_eth_dev *dev, u16 queue_id);
+
+s32 __rte_cold sxe2_tx_queue_setup(struct rte_eth_dev *dev,
+ u16 queue_idx, u16 nb_desc, u32 socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+void __rte_cold sxe2_tx_queue_release(struct rte_eth_dev *dev, u16 queue_idx);
+
+void __rte_cold sxe2_all_txqs_release(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_tx_queue_info_get(struct rte_eth_dev *dev, u16 queue_id,
+ struct rte_eth_txq_info *qinfo);
+
+s32 __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev);
+
+#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v13 03/10] common/sxe2: add sxe2 basic structures
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch adds the base infrastructure for the sxe2 common
library. It includes the mandatory OS abstraction layer (OSAL),
common structure definitions, error codes, and the logging
system implementation.
Specifically, this commit:
- Implements the logging stream management using RTE_LOG_LINE.
- Defines device-specific error codes and status registers.
- Adds the initial meson build configuration for the common library.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_common_log.h | 84 +++
drivers/common/sxe2/sxe2_errno.h | 113 ++++
drivers/common/sxe2/sxe2_host_regs.h | 707 ++++++++++++++++++++++++
drivers/common/sxe2/sxe2_internal_ver.h | 33 ++
drivers/common/sxe2/sxe2_osal.h | 586 ++++++++++++++++++++
drivers/common/sxe2/sxe2_type.h | 60 ++
6 files changed, 1583 insertions(+)
create mode 100644 drivers/common/sxe2/sxe2_common_log.h
create mode 100644 drivers/common/sxe2/sxe2_errno.h
create mode 100644 drivers/common/sxe2/sxe2_host_regs.h
create mode 100644 drivers/common/sxe2/sxe2_internal_ver.h
create mode 100644 drivers/common/sxe2/sxe2_osal.h
create mode 100644 drivers/common/sxe2/sxe2_type.h
diff --git a/drivers/common/sxe2/sxe2_common_log.h b/drivers/common/sxe2/sxe2_common_log.h
new file mode 100644
index 0000000000..a7d2157610
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_common_log.h
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_COMMON_LOG_H__
+#define __SXE2_COMMON_LOG_H__
+
+#include "sxe2_type.h"
+
+extern s32 sxe2_common_log;
+extern s32 sxe2_log_init;
+extern s32 sxe2_log_driver;
+extern s32 sxe2_log_rx;
+extern s32 sxe2_log_tx;
+extern s32 sxe2_log_hw;
+
+#define RTE_LOGTYPE_SXE2_COM sxe2_common_log
+#define RTE_LOGTYPE_SXE2_INIT sxe2_log_init
+#define RTE_LOGTYPE_SXE2_DRV sxe2_log_driver
+#define RTE_LOGTYPE_SXE2_RX sxe2_log_rx
+#define RTE_LOGTYPE_SXE2_TX sxe2_log_tx
+#define RTE_LOGTYPE_SXE2_HW sxe2_log_hw
+
+#define SXE2_PMD_LOG(level, log_type, ...) \
+ RTE_LOG_LINE_PREFIX(level, log_type, "%s(): ", \
+ __func__, __VA_ARGS__)
+
+#define SXE2_PMD_DRV_LOG(level, log_type, adapter, ...) \
+ RTE_LOG_LINE_PREFIX(level, log_type, "%s(): port:%u ", \
+ __func__ RTE_LOG_COMMA \
+ adapter->dev_port_id, __VA_ARGS__)
+
+#define PMD_LOG_DEBUG(logtype, fmt, ...) \
+ SXE2_PMD_LOG(DEBUG, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_LOG_INFO(logtype, fmt, ...) \
+ SXE2_PMD_LOG(INFO, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_LOG_NOTICE(logtype, fmt, ...) \
+ SXE2_PMD_LOG(NOTICE, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_LOG_WARN(logtype, fmt, ...) \
+ SXE2_PMD_LOG(WARNING, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_LOG_ERR(logtype, fmt, ...) \
+ SXE2_PMD_LOG(ERR, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_LOG_CRIT(logtype, fmt, ...) \
+ SXE2_PMD_LOG(CRIT, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_LOG_ALERT(logtype, fmt, ...) \
+ SXE2_PMD_LOG(ALERT, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_LOG_EMERG(logtype, fmt, ...) \
+ SXE2_PMD_LOG(EMERG, SXE2_##logtype, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_DEBUG(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(DEBUG, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_INFO(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(INFO, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_NOTICE(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(NOTICE, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_WARN(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(WARNING, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_ERR(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(ERR, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_CRIT(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(CRIT, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_ALERT(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(ALERT, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_DEV_LOG_EMERG(adapter, logtype, fmt, ...) \
+ SXE2_PMD_DRV_LOG(EMERG, SXE2_##logtype, adapter, fmt, ##__VA_ARGS__)
+
+#define PMD_INIT_FUNC_TRACE() PMD_LOG_DEBUG(INIT, " >>")
+
+#endif /* __SXE2_COMMON_LOG_H__ */
+
diff --git a/drivers/common/sxe2/sxe2_errno.h b/drivers/common/sxe2/sxe2_errno.h
new file mode 100644
index 0000000000..89a715eaef
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_errno.h
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_ERRNO_H__
+#define __SXE2_ERRNO_H__
+#include <errno.h>
+
+enum sxe2_status {
+
+ SXE2_SUCCESS = 0,
+
+ SXE2_ERR_PERM = -EPERM,
+ SXE2_ERR_NOFILE = -ENOENT,
+ SXE2_ERR_NOENT = -ENOENT,
+ SXE2_ERR_SRCH = -ESRCH,
+ SXE2_ERR_INTR = -EINTR,
+ SXE2_ERR_IO = -EIO,
+ SXE2_ERR_NXIO = -ENXIO,
+ SXE2_ERR_2BIG = -E2BIG,
+ SXE2_ERR_NOEXEC = -ENOEXEC,
+ SXE2_ERR_BADF = -EBADF,
+ SXE2_ERR_CHILD = -ECHILD,
+ SXE2_ERR_AGAIN = -EAGAIN,
+ SXE2_ERR_NOMEM = -ENOMEM,
+ SXE2_ERR_ACCES = -EACCES,
+ SXE2_ERR_FAULT = -EFAULT,
+ SXE2_ERR_BUSY = -EBUSY,
+ SXE2_ERR_EXIST = -EEXIST,
+ SXE2_ERR_XDEV = -EXDEV,
+ SXE2_ERR_NODEV = -ENODEV,
+ SXE2_ERR_NOTSUP = -ENOTSUP,
+ SXE2_ERR_NOTDIR = -ENOTDIR,
+ SXE2_ERR_ISDIR = -EISDIR,
+ SXE2_ERR_INVAL = -EINVAL,
+ SXE2_ERR_NFILE = -ENFILE,
+ SXE2_ERR_MFILE = -EMFILE,
+ SXE2_ERR_NOTTY = -ENOTTY,
+ SXE2_ERR_FBIG = -EFBIG,
+ SXE2_ERR_NOSPC = -ENOSPC,
+ SXE2_ERR_SPIPE = -ESPIPE,
+ SXE2_ERR_ROFS = -EROFS,
+ SXE2_ERR_MLINK = -EMLINK,
+ SXE2_ERR_PIPE = -EPIPE,
+ SXE2_ERR_DOM = -EDOM,
+ SXE2_ERR_RANGE = -ERANGE,
+ SXE2_ERR_DEADLOCK = -EDEADLK,
+ SXE2_ERR_DEADLK = -EDEADLK,
+ SXE2_ERR_NAMETOOLONG = -ENAMETOOLONG,
+ SXE2_ERR_NOLCK = -ENOLCK,
+ SXE2_ERR_NOSYS = -ENOSYS,
+ SXE2_ERR_NOTEMPTY = -ENOTEMPTY,
+ SXE2_ERR_ILSEQ = -EILSEQ,
+ SXE2_ERR_NODATA = -ENODATA,
+ SXE2_ERR_CANCELED = -ECANCELED,
+ SXE2_ERR_TIMEDOUT = -ETIMEDOUT,
+
+ SXE2_ERROR = -150,
+ SXE2_ERR_NO_MEMORY = -151,
+ SXE2_ERR_HW_VERSION = -152,
+ SXE2_ERR_FW_VERSION = -153,
+ SXE2_ERR_FW_MODE = -154,
+
+ SXE2_ERR_CMD_ERROR = -156,
+ SXE2_ERR_CMD_NO_MEMORY = -157,
+ SXE2_ERR_CMD_NOT_READY = -158,
+ SXE2_ERR_CMD_TIMEOUT = -159,
+ SXE2_ERR_CMD_CANCELED = -160,
+ SXE2_ERR_CMD_RETRY = -161,
+ SXE2_ERR_CMD_HW_CRITICAL = -162,
+ SXE2_ERR_CMD_NO_DATA = -163,
+ SXE2_ERR_CMD_INVAL_SIZE = -164,
+ SXE2_ERR_CMD_INVAL_TYPE = -165,
+ SXE2_ERR_CMD_INVAL_LEN = -165,
+ SXE2_ERR_CMD_INVAL_MAGIC = -166,
+ SXE2_ERR_CMD_INVAL_HEAD = -167,
+ SXE2_ERR_CMD_INVAL_ID = -168,
+
+ SXE2_ERR_DESC_NO_DONE = -171,
+
+ SXE2_ERR_INIT_ARGS_NAME_INVAL = -181,
+ SXE2_ERR_INIT_ARGS_VAL_INVAL = -182,
+ SXE2_ERR_INIT_VSI_CRITICAL = -183,
+
+ SXE2_ERR_CFG_FILE_PATH = -191,
+ SXE2_ERR_CFG_FILE = -192,
+ SXE2_ERR_CFG_INVALID_SIZE = -193,
+ SXE2_ERR_CFG_NO_PIPELINE_CFG = -194,
+
+ SXE2_ERR_RESET_TIMIEOUT = -200,
+ SXE2_ERR_VF_NOT_ACTIVE = -201,
+ SXE2_ERR_BUF_CSUM_ERR = -202,
+ SXE2_ERR_VF_DROP = -203,
+
+ SXE2_ERR_FLOW_PARAM = -301,
+ SXE2_ERR_FLOW_CFG = -302,
+ SXE2_ERR_FLOW_CFG_NOT_SUPPORT = -303,
+ SXE2_ERR_FLOW_PROF_EXISTS = -304,
+ SXE2_ERR_FLOW_PROF_NOT_EXISTS = -305,
+ SXE2_ERR_FLOW_VSIG_FULL = -306,
+ SXE2_ERR_FLOW_VSIG_INFO = -307,
+ SXE2_ERR_FLOW_VSIG_NOT_FIND = -308,
+ SXE2_ERR_FLOW_VSIG_NOT_USED = -309,
+ SXE2_ERR_FLOW_VSI_NOT_IN_VSIG = -310,
+ SXE2_ERR_FLOW_MAX_LIMIT = -311,
+
+ SXE2_ERR_SCHED_NEED_RECURSION = -400,
+
+ SXE2_ERR_BFD_SESS_FLOW_HT_COLLISION = -500,
+ SXE2_ERR_BFD_SESS_FLOW_NOSPC = -501,
+};
+
+#endif
diff --git a/drivers/common/sxe2/sxe2_host_regs.h b/drivers/common/sxe2/sxe2_host_regs.h
new file mode 100644
index 0000000000..984ea6214c
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_host_regs.h
@@ -0,0 +1,707 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_HOST_REGS_H__
+#define __SXE2_HOST_REGS_H__
+
+#define SXE2_BITS_MASK(m, s) ((m ## UL) << (s))
+
+#define SXE2_RXQ_CTXT(_i, _QRX) (0x0050000 + ((_i) * 4 + (_QRX) * 0x20))
+#define SXE2_RXQ_HEAD(_QRX) (0x0060000 + ((_QRX) * 4))
+#define SXE2_RXQ_TAIL(_QRX) (0x0070000 + ((_QRX) * 4))
+#define SXE2_RXQ_CTRL(_QRX) (0x006d000 + ((_QRX) * 4))
+#define SXE2_RXQ_WB(_QRX) (0x006B000 + ((_QRX) * 4))
+
+#define SXE2_RXQ_CTRL_STATUS_ACTIVE 0x00000004
+#define SXE2_RXQ_CTRL_ENABLED 0x00000001
+#define SXE2_RXQ_CTRL_CDE_ENABLE BIT(3)
+
+#define SXE2_PCIEPROC_BASE 0x002d6000
+
+#define SXE2_PF_INT_BASE 0x00260000
+#define SXE2_PF_INT_ALLOC (SXE2_PF_INT_BASE + 0x0000)
+#define SXE2_PF_INT_ALLOC_FIRST 0x7FF
+#define SXE2_PF_INT_ALLOC_LAST_S 12
+#define SXE2_PF_INT_ALLOC_LAST \
+ (0x7FF << SXE2_PF_INT_ALLOC_LAST_S)
+#define SXE2_PF_INT_ALLOC_VALID BIT(31)
+
+#define SXE2_PF_INT_OICR (SXE2_PF_INT_BASE + 0x0040)
+#define SXE2_PF_INT_OICR_PCIE_TIMEOUT BIT(0)
+#define SXE2_PF_INT_OICR_UR BIT(1)
+#define SXE2_PF_INT_OICR_CA BIT(2)
+#define SXE2_PF_INT_OICR_VFLR BIT(3)
+#define SXE2_PF_INT_OICR_VFR_DONE BIT(4)
+#define SXE2_PF_INT_OICR_LAN_TX_ERR BIT(5)
+#define SXE2_PF_INT_OICR_BFDE BIT(6)
+#define SXE2_PF_INT_OICR_LAN_RX_ERR BIT(7)
+#define SXE2_PF_INT_OICR_ECC_ERR BIT(8)
+#define SXE2_PF_INT_OICR_GPIO BIT(9)
+#define SXE2_PF_INT_OICR_TSYN_TX BIT(11)
+#define SXE2_PF_INT_OICR_TSYN_EVENT BIT(12)
+#define SXE2_PF_INT_OICR_TSYN_TGT BIT(13)
+#define SXE2_PF_INT_OICR_EXHAUST BIT(14)
+#define SXE2_PF_INT_OICR_FW BIT(15)
+#define SXE2_PF_INT_OICR_SWINT BIT(16)
+#define SXE2_PF_INT_OICR_LINKSEC_CHG BIT(17)
+#define SXE2_PF_INT_OICR_INT_CFG_ADDR_ERR BIT(18)
+#define SXE2_PF_INT_OICR_INT_CFG_DATA_ERR BIT(19)
+#define SXE2_PF_INT_OICR_INT_CFG_ADR_UNRANGE BIT(20)
+#define SXE2_PF_INT_OICR_INT_RAM_CONFLICT BIT(21)
+#define SXE2_PF_INT_OICR_GRST BIT(22)
+#define SXE2_PF_INT_OICR_FWQ_INT BIT(29)
+#define SXE2_PF_INT_OICR_FWQ_TOOL_INT BIT(30)
+#define SXE2_PF_INT_OICR_MBXQ_INT BIT(31)
+
+#define SXE2_PF_INT_OICR_ENABLE (SXE2_PF_INT_BASE + 0x0020)
+
+#define SXE2_PF_INT_FW_EVENT (SXE2_PF_INT_BASE + 0x0100)
+#define SXE2_PF_INT_FW_ABNORMAL BIT(0)
+#define SXE2_PF_INT_RDMA_AEQ_OVERFLOW BIT(1)
+#define SXE2_PF_INT_CGMAC_LINK_CHG BIT(18)
+#define SXE2_PF_INT_VFLR_DONE BIT(2)
+
+#define SXE2_PF_INT_OICR_CTL (SXE2_PF_INT_BASE + 0x0060)
+#define SXE2_PF_INT_OICR_CTL_MSIX_IDX 0x7FF
+#define SXE2_PF_INT_OICR_CTL_ITR_IDX_S 11
+#define SXE2_PF_INT_OICR_CTL_ITR_IDX \
+ (0x3 << SXE2_PF_INT_OICR_CTL_ITR_IDX_S)
+#define SXE2_PF_INT_OICR_CTL_CAUSE_ENABLE BIT(30)
+
+#define SXE2_PF_INT_FWQ_CTL (SXE2_PF_INT_BASE + 0x00C0)
+#define SXE2_PF_INT_FWQ_CTL_MSIX_IDX 0x7FFF
+#define SXE2_PF_INT_FWQ_CTL_ITR_IDX_S 11
+#define SXE2_PF_INT_FWQ_CTL_ITR_IDX \
+ (0x3 << SXE2_PF_INT_FWQ_CTL_ITR_IDX_S)
+#define SXE2_PF_INT_FWQ_CTL_CAUSE_ENABLE BIT(30)
+
+#define SXE2_PF_INT_MBX_CTL (SXE2_PF_INT_BASE + 0x00A0)
+#define SXE2_PF_INT_MBX_CTL_MSIX_IDX 0x7FF
+#define SXE2_PF_INT_MBX_CTL_ITR_IDX_S 11
+#define SXE2_PF_INT_MBX_CTL_ITR_IDX (0x3 << SXE2_PF_INT_MBX_CTL_ITR_IDX_S)
+#define SXE2_PF_INT_MBX_CTL_CAUSE_ENABLE BIT(30)
+
+#define SXE2_PF_INT_GPIO_ENA (SXE2_PF_INT_BASE + 0x0100)
+#define SXE2_PF_INT_GPIO_X_ENA(x) BIT(x)
+
+#define SXE2_PFG_INT_CTL (SXE2_PF_INT_BASE + 0x0120)
+#define SXE2_PFG_INT_CTL_ITR_GRAN 0x7
+#define SXE2_PFG_INT_CTL_ITR_GRAN_0 (2)
+#define SXE2_PFG_INT_CTL_CREDIT_GRAN BIT(4)
+#define SXE2_PFG_INT_CTL_CREDIT_GRAN_0 (4)
+#define SXE2_PFG_INT_CTL_CREDIT_GRAN_1 (8)
+
+#define SXE2_VFG_RAM_INIT_DONE \
+ (SXE2_PF_INT_BASE + 0x0128)
+#define SXE2_VFG_RAM_INIT_DONE_0 BIT(0)
+#define SXE2_VFG_RAM_INIT_DONE_1 BIT(1)
+#define SXE2_VFG_RAM_INIT_DONE_2 BIT(2)
+
+#define SXE2_LINK_REG_GET_10G_VALUE 4
+#define SXE2_LINK_REG_GET_25G_VALUE 1
+#define SXE2_LINK_REG_GET_50G_VALUE 2
+#define SXE2_LINK_REG_GET_100G_VALUE 3
+
+#define SXE2_PORT0_CNT 0
+#define SXE2_PORT1_CNT 1
+#define SXE2_PORT2_CNT 2
+#define SXE2_PORT3_CNT 3
+
+#define SXE2_LINK_STATUS_BASE (0x002ac200)
+#define SXE2_LINK_STATUS_PORT0_POS 3
+#define SXE2_LINK_STATUS_PORT1_POS 11
+#define SXE2_LINK_STATUS_PORT2_POS 19
+#define SXE2_LINK_STATUS_PORT3_POS 27
+#define SXE2_LINK_STATUS_MASK 1
+
+#define SXE2_LINK_SPEED_BASE (0x002ac200)
+#define SXE2_LINK_SPEED_PORT0_POS 0
+#define SXE2_LINK_SPEED_PORT1_POS 8
+#define SXE2_LINK_SPEED_PORT2_POS 16
+#define SXE2_LINK_SPEED_PORT3_POS 24
+#define SXE2_LINK_SPEED_MASK 7
+
+#define SXE2_PFVP_INT_ALLOC(vf_idx) (SXE2_PF_INT_BASE + 0x012C + ((vf_idx) * 4))
+#define SXE2_PFVP_INT_ALLOC_FIRST_S 0
+
+#define SXE2_PFVP_INT_ALLOC_FIRST_M (0x7FF << SXE2_PFVP_INT_ALLOC_FIRST_S)
+#define SXE2_PFVP_INT_ALLOC_LAST_S 12
+#define SXE2_PFVP_INT_ALLOC_LAST_M \
+ (0x7FF << SXE2_PFVP_INT_ALLOC_LAST_S)
+#define SXE2_PFVP_INT_ALLOC_VALID BIT(31)
+
+#define SXE2_PCI_PFVP_INT_ALLOC(vf_idx) (SXE2_PCIEPROC_BASE + 0x5800 + ((vf_idx) * 4))
+#define SXE2_PCI_PFVP_INT_ALLOC_FIRST_S 0
+
+#define SXE2_PCI_PFVP_INT_ALLOC_FIRST_M (0x7FF << SXE2_PCI_PFVP_INT_ALLOC_FIRST_S)
+#define SXE2_PCI_PFVP_INT_ALLOC_LAST_S 12
+
+#define SXE2_PCI_PFVP_INT_ALLOC_LAST_M \
+ (0x7FF << SXE2_PCI_PFVP_INT_ALLOC_LAST_S)
+#define SXE2_PCI_PFVP_INT_ALLOC_VALID BIT(31)
+
+#define SXE2_PCIEPROC_INT2FUNC(_INT) (SXE2_PCIEPROC_BASE + 0xe000 + ((_INT) * 4))
+#define SXE2_PCIEPROC_INT2FUNC_VF_NUM_S 0
+#define SXE2_PCIEPROC_INT2FUNC_VF_NUM_M (0xFF << SXE2_PCIEPROC_INT2FUNC_VF_NUM_S)
+#define SXE2_PCIEPROC_INT2FUNC_PF_NUM_S 12
+#define SXE2_PCIEPROC_INT2FUNC_PF_NUM_M (0x7 << SXE2_PCIEPROC_INT2FUNC_PF_NUM_S)
+#define SXE2_PCIEPROC_INT2FUNC_IS_PF_S 16
+#define SXE2_PCIEPROC_INT2FUNC_IS_PF_M BIT(16)
+
+#define SXE2_VSI_PF(vf_idx) (SXE2_PF_INT_BASE + 0x14000 + ((vf_idx) * 4))
+#define SXE2_VSI_PF_ID_S 0
+#define SXE2_VSI_PF_ID_M (0x7 << SXE2_VSI_PF_ID_S)
+#define SXE2_VSI_PF_EN_M BIT(3)
+
+#define SXE2_MBX_CTL(_VSI) (0x0026692C + ((_VSI) * 4))
+#define SXE2_MBX_CTL_MSIX_INDX_S 0
+#define SXE2_MBX_CTL_MSIX_INDX_M (0x7FF << SXE2_MBX_CTL_MSIX_INDX_S)
+#define SXE2_MBX_CTL_CAUSE_ENA_M BIT(30)
+
+#define SXE2_PF_INT_TQCTL(q_idx) (SXE2_PF_INT_BASE + 0x092C + 4 * (q_idx))
+#define SXE2_PF_INT_TQCTL_MSIX_IDX 0x7FF
+#define SXE2_PF_INT_TQCTL_ITR_IDX_S 11
+#define SXE2_PF_INT_TQCTL_ITR_IDX \
+ (0x3 << SXE2_PF_INT_TQCTL_ITR_IDX_S)
+#define SXE2_PF_INT_TQCTL_CAUSE_ENABLE BIT(30)
+
+#define SXE2_PF_INT_RQCTL(q_idx) (SXE2_PF_INT_BASE + 0x292C + 4 * (q_idx))
+#define SXE2_PF_INT_RQCTL_MSIX_IDX 0x7FF
+#define SXE2_PF_INT_RQCTL_ITR_IDX_S 11
+#define SXE2_PF_INT_RQCTL_ITR_IDX \
+ (0x3 << SXE2_PF_INT_RQCTL_ITR_IDX_S)
+#define SXE2_PF_INT_RQCTL_CAUSE_ENABLE BIT(30)
+
+#define SXE2_PF_INT_RATE(irq_idx) (SXE2_PF_INT_BASE + 0x7530 + 4 * (irq_idx))
+#define SXE2_PF_INT_RATE_CREDIT_INTERVAL (0x3F)
+#define SXE2_PF_INT_RATE_CREDIT_INTERVAL_MAX \
+ (0x3F)
+#define SXE2_PF_INT_RATE_INTRL_ENABLE (BIT(6))
+#define SXE2_PF_INT_RATE_CREDIT_MAX_VALUE_SHIFT (7)
+#define SXE2_PF_INT_RATE_CREDIT_MAX_VALUE \
+ (0x3F << SXE2_PF_INT_RATE_CREDIT_MAX_VALUE_SHIFT)
+
+#define SXE2_VF_INT_ITR(itr_idx, irq_idx) \
+ (SXE2_PF_INT_BASE + 0xB530 + 0x2000 * (itr_idx) + 4 * (irq_idx))
+#define SXE2_VF_INT_ITR_INTERVAL 0xFFF
+
+#define SXE2_VF_DYN_CTL(irq_idx) (SXE2_PF_INT_BASE + 0x9530 + 4 * (irq_idx))
+#define SXE2_VF_DYN_CTL_INTENABLE BIT(0)
+#define SXE2_VF_DYN_CTL_CLEARPBA BIT(1)
+#define SXE2_VF_DYN_CTL_SWINT_TRIG BIT(2)
+#define SXE2_VF_DYN_CTL_ITR_IDX_S \
+ 3
+#define SXE2_VF_DYN_CTL_ITR_IDX_M 0x3
+#define SXE2_VF_DYN_CTL_INTERVAL_S 5
+#define SXE2_VF_DYN_CTL_INTERVAL_M 0xFFF
+#define SXE2_VF_DYN_CTL_SW_ITR_IDX_ENABLE BIT(24)
+#define SXE2_VF_DYN_CTL_SW_ITR_IDX_S 25
+#define SXE2_VF_DYN_CTL_SW_ITR_IDX_M 0x3
+
+#define SXE2_VF_DYN_CTL_INTENABLE_MSK \
+ BIT(31)
+
+#define SXE2_BAR4_MSIX_BASE 0
+#define SXE2_BAR4_MSIX_CTL(_idx) (SXE2_BAR4_MSIX_BASE + 0xC + ((_idx) * 0x10))
+#define SXE2_BAR4_MSIX_ENABLE 0
+#define SXE2_BAR4_MSIX_DISABLE 1
+
+#define SXE2_TXQ_LEGACY_DBLL(_DBQM) (0x1000 + ((_DBQM) * 4))
+
+#define SXE2_TXQ_CONTEXT0(_pfIdx) (0x10040 + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_CONTEXT1(_pfIdx) (0x10044 + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_CONTEXT2(_pfIdx) (0x10048 + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_CONTEXT3(_pfIdx) (0x1004C + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_CONTEXT4(_pfIdx) (0x10050 + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_CONTEXT7(_pfIdx) (0x1005C + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_CONTEXT7_HEAD_S 0
+#define SXE2_TXQ_CONTEXT7_HEAD_M SXE2_BITS_MASK(0xFFF, SXE2_TXQ_CONTEXT7_HEAD_S)
+#define SXE2_TXQ_CONTEXT7_READ_HEAD_S 16
+#define SXE2_TXQ_CONTEXT7_READ_HEAD_M SXE2_BITS_MASK(0xFFF, SXE2_TXQ_CONTEXT7_READ_HEAD_S)
+
+#define SXE2_TXQ_CTRL(_pfIdx) (0x10064 + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_CTXT_CTRL(_pfIdx) (0x100C8 + ((_pfIdx) * 0x100))
+#define SXE2_TXQ_DIS_CNT(_pfIdx) (0x100D0 + ((_pfIdx) * 0x100))
+
+#define SXE2_TXQ_CTXT_CTRL_USED_MASK 0x00000800
+#define SXE2_TXQ_CTRL_SW_EN_M BIT(0)
+#define SXE2_TXQ_CTRL_HW_EN_M BIT(1)
+
+#define SXE2_TXQ_CTXT2_PROT_IDX_S 0
+#define SXE2_TXQ_CTXT2_PROT_IDX_M SXE2_BITS_MASK(0x7, 0)
+#define SXE2_TXQ_CTXT2_CGD_IDX_S 4
+#define SXE2_TXQ_CTXT2_CGD_IDX_M SXE2_BITS_MASK(0x1F, 4)
+#define SXE2_TXQ_CTXT2_PF_IDX_S 9
+#define SXE2_TXQ_CTXT2_PF_IDX_M SXE2_BITS_MASK(0x7, 9)
+#define SXE2_TXQ_CTXT2_VMVF_IDX_S 12
+#define SXE2_TXQ_CTXT2_VMVF_IDX_M SXE2_BITS_MASK(0x3FF, 12)
+#define SXE2_TXQ_CTXT2_VMVF_TYPE_S 23
+#define SXE2_TXQ_CTXT2_VMVF_TYPE_M SXE2_BITS_MASK(0x3, 23)
+#define SXE2_TXQ_CTXT2_TSYN_ENA_S 25
+#define SXE2_TXQ_CTXT2_TSYN_ENA_M BIT(25)
+#define SXE2_TXQ_CTXT2_ALT_VLAN_S 26
+#define SXE2_TXQ_CTXT2_ALT_VLAN_M BIT(26)
+#define SXE2_TXQ_CTXT2_WB_MODE_S 27
+#define SXE2_TXQ_CTXT2_WB_MODE_M BIT(27)
+#define SXE2_TXQ_CTXT2_ITR_WB_S 28
+#define SXE2_TXQ_CTXT2_ITR_WB_M BIT(28)
+#define SXE2_TXQ_CTXT2_LEGACY_EN_S 29
+#define SXE2_TXQ_CTXT2_LEGACY_EN_M BIT(29)
+#define SXE2_TXQ_CTXT2_SSO_EN_S 30
+#define SXE2_TXQ_CTXT2_SSO_EN_M BIT(30)
+
+#define SXE2_TXQ_CTXT3_SRC_VSI_S 0
+#define SXE2_TXQ_CTXT3_SRC_VSI_M SXE2_BITS_MASK(0x3FF, 0)
+#define SXE2_TXQ_CTXT3_CPU_ID_S 12
+#define SXE2_TXQ_CTXT3_CPU_ID_M SXE2_BITS_MASK(0xFF, 12)
+#define SXE2_TXQ_CTXT3_TPH_RDDESC_S 20
+#define SXE2_TXQ_CTXT3_TPH_RDDESC_M BIT(20)
+#define SXE2_TXQ_CTXT3_TPH_RDDATA_S 21
+#define SXE2_TXQ_CTXT3_TPH_RDDATA_M BIT(21)
+#define SXE2_TXQ_CTXT3_TPH_WRDESC_S 22
+#define SXE2_TXQ_CTXT3_TPH_WRDESC_M BIT(22)
+
+#define SXE2_TXQ_CTXT3_QID_IN_FUNC_S 0
+#define SXE2_TXQ_CTXT3_QID_IN_FUNC_M SXE2_BITS_MASK(0x7FF, 0)
+#define SXE2_TXQ_CTXT3_RDDESC_RO_S 13
+#define SXE2_TXQ_CTXT3_RDDESC_RO_M BIT(13)
+#define SXE2_TXQ_CTXT3_WRDESC_RO_S 14
+#define SXE2_TXQ_CTXT3_WRDESC_RO_M BIT(14)
+#define SXE2_TXQ_CTXT3_RDDATA_RO_S 15
+#define SXE2_TXQ_CTXT3_RDDATA_RO_M BIT(15)
+#define SXE2_TXQ_CTXT3_QLEN_S 16
+#define SXE2_TXQ_CTXT3_QLEN_M SXE2_BITS_MASK(0x1FFF, 16)
+
+#define SXE2_RX_BUF_CHAINED_MAX 10
+#define SXE2_RX_DESC_BASE_ADDR_UNIT 7
+#define SXE2_RX_HBUF_LEN_UNIT 6
+#define SXE2_RX_DBUF_LEN_UNIT 7
+#define SXE2_RX_DBUF_LEN_MASK (~0x7F)
+#define SXE2_RX_HWTAIL_VALUE_MASK (~0x7)
+
+enum {
+ SXE2_RX_CTXT0 = 0,
+ SXE2_RX_CTXT1,
+ SXE2_RX_CTXT2,
+ SXE2_RX_CTXT3,
+ SXE2_RX_CTXT4,
+ SXE2_RX_CTXT_CNT,
+};
+
+#define SXE2_RX_CTXT_BASE_L_S 0
+#define SXE2_RX_CTXT_BASE_L_W 32
+
+#define SXE2_RX_CTXT_BASE_H_S 0
+#define SXE2_RX_CTXT_BASE_H_W 25
+#define SXE2_RX_CTXT_DEPTH_L_S 25
+#define SXE2_RX_CTXT_DEPTH_L_W 7
+
+#define SXE2_RX_CTXT_DEPTH_H_S 0
+#define SXE2_RX_CTXT_DEPTH_H_W 6
+
+#define SXE2_RX_CTXT_DBUFF_S 6
+#define SXE2_RX_CTXT_DBUFF_W 7
+
+#define SXE2_RX_CTXT_HBUFF_S 13
+#define SXE2_RX_CTXT_HBUFF_W 5
+
+#define SXE2_RX_CTXT_HSPLT_TYPE_S 18
+#define SXE2_RX_CTXT_HSPLT_TYPE_W 2
+
+#define SXE2_RX_CTXT_DESC_TYPE_S 20
+#define SXE2_RX_CTXT_DESC_TYPE_W 1
+
+#define SXE2_RX_CTXT_CRC_S 21
+#define SXE2_RX_CTXT_CRC_W 1
+
+#define SXE2_RX_CTXT_L2TAG_FLAG_S 23
+#define SXE2_RX_CTXT_L2TAG_FLAG_W 1
+
+#define SXE2_RX_CTXT_HSPLT_0_S 24
+#define SXE2_RX_CTXT_HSPLT_0_W 4
+
+#define SXE2_RX_CTXT_HSPLT_1_S 28
+#define SXE2_RX_CTXT_HSPLT_1_W 2
+
+#define SXE2_RX_CTXT_INVALN_STP_S 31
+#define SXE2_RX_CTXT_INVALN_STP_W 1
+
+#define SXE2_RX_CTXT_LRO_ENABLE_S 0
+#define SXE2_RX_CTXT_LRO_ENABLE_W 1
+
+#define SXE2_RX_CTXT_CPUID_S 3
+#define SXE2_RX_CTXT_CPUID_W 8
+
+#define SXE2_RX_CTXT_MAX_FRAME_SIZE_S 11
+#define SXE2_RX_CTXT_MAX_FRAME_SIZE_W 14
+
+#define SXE2_RX_CTXT_LRO_DESC_MAX_S 25
+#define SXE2_RX_CTXT_LRO_DESC_MAX_W 4
+
+#define SXE2_RX_CTXT_RELAX_DATA_S 29
+#define SXE2_RX_CTXT_RELAX_DATA_W 1
+
+#define SXE2_RX_CTXT_RELAX_WB_S 30
+#define SXE2_RX_CTXT_RELAX_WB_W 1
+
+#define SXE2_RX_CTXT_RELAX_RD_S 31
+#define SXE2_RX_CTXT_RELAX_RD_W 1
+
+#define SXE2_RX_CTXT_THPRDESC_ENABLE_S 1
+#define SXE2_RX_CTXT_THPRDESC_ENABLE_W 1
+
+#define SXE2_RX_CTXT_THPWDESC_ENABLE_S 2
+#define SXE2_RX_CTXT_THPWDESC_ENABLE_W 1
+
+#define SXE2_RX_CTXT_THPRDATA_ENABLE_S 3
+#define SXE2_RX_CTXT_THPRDATA_ENABLE_W 1
+
+#define SXE2_RX_CTXT_THPHEAD_ENABLE_S 4
+#define SXE2_RX_CTXT_THPHEAD_ENABLE_W 1
+
+#define SXE2_RX_CTXT_LOW_DESC_LINE_S 6
+#define SXE2_RX_CTXT_LOW_DESC_LINE_W 3
+
+#define SXE2_RX_CTXT_VF_ID_S 9
+#define SXE2_RX_CTXT_VF_ID_W 8
+
+#define SXE2_RX_CTXT_PF_ID_S 17
+#define SXE2_RX_CTXT_PF_ID_W 3
+
+#define SXE2_RX_CTXT_VF_ENABLE_S 20
+#define SXE2_RX_CTXT_VF_ENABLE_W 1
+
+#define SXE2_RX_CTXT_VSI_ID_S 21
+#define SXE2_RX_CTXT_VSI_ID_W 10
+
+#define SXE2_PF_CTRLQ_FW_BASE 0x00312000
+#define SXE2_PF_CTRLQ_FW_ATQBAL (SXE2_PF_CTRLQ_FW_BASE + 0x0000)
+#define SXE2_PF_CTRLQ_FW_ARQBAL (SXE2_PF_CTRLQ_FW_BASE + 0x0080)
+#define SXE2_PF_CTRLQ_FW_ATQBAH (SXE2_PF_CTRLQ_FW_BASE + 0x0100)
+#define SXE2_PF_CTRLQ_FW_ARQBAH (SXE2_PF_CTRLQ_FW_BASE + 0x0180)
+#define SXE2_PF_CTRLQ_FW_ATQLEN (SXE2_PF_CTRLQ_FW_BASE + 0x0200)
+#define SXE2_PF_CTRLQ_FW_ARQLEN (SXE2_PF_CTRLQ_FW_BASE + 0x0280)
+#define SXE2_PF_CTRLQ_FW_ATQH (SXE2_PF_CTRLQ_FW_BASE + 0x0300)
+#define SXE2_PF_CTRLQ_FW_ARQH (SXE2_PF_CTRLQ_FW_BASE + 0x0380)
+#define SXE2_PF_CTRLQ_FW_ATQT (SXE2_PF_CTRLQ_FW_BASE + 0x0400)
+#define SXE2_PF_CTRLQ_FW_ARQT (SXE2_PF_CTRLQ_FW_BASE + 0x0480)
+
+#define SXE2_PF_CTRLQ_MBX_BASE 0x00316000
+#define SXE2_PF_CTRLQ_MBX_ATQBAL (SXE2_PF_CTRLQ_MBX_BASE + 0xE100)
+#define SXE2_PF_CTRLQ_MBX_ATQBAH (SXE2_PF_CTRLQ_MBX_BASE + 0xE180)
+#define SXE2_PF_CTRLQ_MBX_ATQLEN (SXE2_PF_CTRLQ_MBX_BASE + 0xE200)
+#define SXE2_PF_CTRLQ_MBX_ATQH (SXE2_PF_CTRLQ_MBX_BASE + 0xE280)
+#define SXE2_PF_CTRLQ_MBX_ATQT (SXE2_PF_CTRLQ_MBX_BASE + 0xE300)
+#define SXE2_PF_CTRLQ_MBX_ARQBAL (SXE2_PF_CTRLQ_MBX_BASE + 0xE380)
+#define SXE2_PF_CTRLQ_MBX_ARQBAH (SXE2_PF_CTRLQ_MBX_BASE + 0xE400)
+#define SXE2_PF_CTRLQ_MBX_ARQLEN (SXE2_PF_CTRLQ_MBX_BASE + 0xE480)
+#define SXE2_PF_CTRLQ_MBX_ARQH (SXE2_PF_CTRLQ_MBX_BASE + 0xE500)
+#define SXE2_PF_CTRLQ_MBX_ARQT (SXE2_PF_CTRLQ_MBX_BASE + 0xE580)
+
+#define SXE2_CMD_REG_LEN_M 0x3FF
+#define SXE2_CMD_REG_LEN_VFE_M BIT(28)
+#define SXE2_CMD_REG_LEN_OVFL_M BIT(29)
+#define SXE2_CMD_REG_LEN_CRIT_M BIT(30)
+#define SXE2_CMD_REG_LEN_ENABLE_M BIT(31)
+
+#define SXE2_CMD_REG_HEAD_M 0x3FF
+
+#define SXE2_PF_CTRLQ_FW_HW_STS (SXE2_PF_CTRLQ_FW_BASE + 0x0500)
+#define SXE2_PF_CTRLQ_FW_ATQ_IDLE_MASK BIT(0)
+#define SXE2_PF_CTRLQ_FW_ARQ_IDLE_MASK BIT(1)
+
+#define SXE2_TOP_CFG_BASE 0x00292000
+#define SXE2_HW_VER (SXE2_TOP_CFG_BASE + 0x48c)
+#define SXE2_HW_FPGA_VER_M SXE2_BITS_MASK(0xFFF, 0)
+
+#define SXE2_FW_VER (SXE2_TOP_CFG_BASE + 0x214)
+#define SXE2_FW_VER_BUILD_M SXE2_BITS_MASK(0xFF, 0)
+#define SXE2_FW_VER_FIX_M SXE2_BITS_MASK(0xFF, 8)
+#define SXE2_FW_VER_SUB_M SXE2_BITS_MASK(0xFF, 16)
+#define SXE2_FW_VER_MAIN_M SXE2_BITS_MASK(0xFF, 24)
+#define SXE2_FW_VER_FIX_SHIFT (8)
+#define SXE2_FW_VER_SUB_SHIFT (16)
+#define SXE2_FW_VER_MAIN_SHIFT (24)
+
+#define SXE2_FW_COMP_VER_ADDR (SXE2_TOP_CFG_BASE + 0x20c)
+
+#define SXE2_STATUS SXE2_FW_VER
+
+#define SXE2_FW_STATE (SXE2_TOP_CFG_BASE + 0x210)
+
+#define SXE2_FW_HEARTBEAT (SXE2_TOP_CFG_BASE + 0x218)
+
+#define SXE2_FW_MISC (SXE2_TOP_CFG_BASE + 0x21c)
+#define SXE2_FW_MISC_MODE_M SXE2_BITS_MASK(0xF, 0)
+#define SXE2_FW_MISC_POP_M SXE2_BITS_MASK(0x80000000, 0)
+
+#define SXE2_TX_OE_BASE 0x00030000
+#define SXE2_RX_OE_BASE 0x00050000
+
+#define SXE2_PFP_L2TAGSEN(_i) (SXE2_TX_OE_BASE + 0x00300 + ((_i) * 4))
+#define SXE2_VSI_L2TAGSTXVALID(_i) \
+ (SXE2_TX_OE_BASE + 0x01000 + ((_i) * 4))
+#define SXE2_VSI_TIR0(_i) (SXE2_TX_OE_BASE + 0x01C00 + ((_i) * 4))
+#define SXE2_VSI_TIR1(_i) (SXE2_TX_OE_BASE + 0x02800 + ((_i) * 4))
+#define SXE2_VSI_TAR(_i) (SXE2_TX_OE_BASE + 0x04C00 + ((_i) * 4))
+#define SXE2_VSI_TSR(_i) (SXE2_RX_OE_BASE + 0x18000 + ((_i) * 4))
+
+#define SXE2_STATS_TX_LAN_CONFIG(_i) (SXE2_TX_OE_BASE + 0x08300 + ((_i) * 4))
+#define SXE2_STATS_TX_LAN_PKT_CNT_GET(_i) (SXE2_TX_OE_BASE + 0x08340 + ((_i) * 4))
+#define SXE2_STATS_TX_LAN_BYTE_CNT_GET(_i) (SXE2_TX_OE_BASE + 0x08380 + ((_i) * 4))
+
+#define SXE2_STATS_RX_CONFIG(_i) (SXE2_RX_OE_BASE + 0x230B0 + ((_i) * 4))
+#define SXE2_STATS_RX_LAN_PKT_CNT_GET(_i) (SXE2_RX_OE_BASE + 0x230C0 + ((_i) * 8))
+#define SXE2_STATS_RX_LAN_BYTE_CNT_GET(_i) (SXE2_RX_OE_BASE + 0x23120 + ((_i) * 8))
+#define SXE2_STATS_RX_FD_PKT_CNT_GET(_i) (SXE2_RX_OE_BASE + 0x230E0 + ((_i) * 8))
+#define SXE2_STATS_RX_MNG_IN_PKT_CNT_GET(_i) (SXE2_RX_OE_BASE + 0x23100 + ((_i) * 8))
+#define SXE2_STATS_RX_MNG_IN_BYTE_CNT_GET(_i) (SXE2_RX_OE_BASE + 0x23140 + ((_i) * 8))
+#define SXE2_STATS_RX_MNG_OUT_PKT_CNT_GET(_i) (SXE2_RX_OE_BASE + 0x23160 + ((_i) * 8))
+
+#define SXE2_L2TAG_ID_STAG 0
+#define SXE2_L2TAG_ID_OUT_VLAN1 1
+#define SXE2_L2TAG_ID_OUT_VLAN2 2
+#define SXE2_L2TAG_ID_VLAN 3
+
+#define SXE2_PFP_L2TAGSEN_ALL_TAG 0xFF
+#define SXE2_PFP_L2TAGSEN_DVM BIT(10)
+
+#define SXE2_VSI_TSR_STRIP_TAG_S 0
+#define SXE2_VSI_TSR_SHOW_TAG_S 4
+
+#define SXE2_VSI_TSR_ID_STAG BIT(0)
+#define SXE2_VSI_TSR_ID_OUT_VLAN1 BIT(1)
+#define SXE2_VSI_TSR_ID_OUT_VLAN2 BIT(2)
+#define SXE2_VSI_TSR_ID_VLAN BIT(3)
+
+#define SXE2_VSI_L2TAGSTXVALID_L2TAG1_ID_S 0
+#define SXE2_VSI_L2TAGSTXVALID_L2TAG1_ID_M 0x7
+#define SXE2_VSI_L2TAGSTXVALID_L2TAG1_VALID BIT(3)
+#define SXE2_VSI_L2TAGSTXVALID_L2TAG2_ID_S 4
+#define SXE2_VSI_L2TAGSTXVALID_L2TAG2_ID_M 0x7
+#define SXE2_VSI_L2TAGSTXVALID_L2TAG2_VALID BIT(7)
+#define SXE2_VSI_L2TAGSTXVALID_TIR0_ID_S 16
+#define SXE2_VSI_L2TAGSTXVALID_TIR0_VALID BIT(19)
+#define SXE2_VSI_L2TAGSTXVALID_TIR1_ID_S 20
+#define SXE2_VSI_L2TAGSTXVALID_TIR1_VALID BIT(23)
+
+#define SXE2_VSI_L2TAGSTXVALID_ID_STAG 0
+#define SXE2_VSI_L2TAGSTXVALID_ID_OUT_VLAN1 2
+#define SXE2_VSI_L2TAGSTXVALID_ID_OUT_VLAN2 3
+#define SXE2_VSI_L2TAGSTXVALID_ID_VLAN 4
+
+#define SXE2_SWITCH_OG_BASE 0x00140000
+#define SXE2_SWITCH_SWE_BASE 0x00150000
+#define SXE2_SWITCH_RG_BASE 0x00160000
+
+#define SXE2_VSI_RX_SWITCH_CTRL(_i) (SXE2_SWITCH_RG_BASE + 0x01074 + ((_i) * 4))
+#define SXE2_VSI_TX_SWITCH_CTRL(_i) (SXE2_SWITCH_RG_BASE + 0x01C74 + ((_i) * 4))
+
+#define SXE2_VSI_RX_SW_CTRL_VLAN_PRUNE BIT(9)
+
+#define SXE2_VSI_TX_SW_CTRL_LOOPBACK_EN BIT(1)
+#define SXE2_VSI_TX_SW_CTRL_LAN_EN BIT(2)
+#define SXE2_VSI_TX_SW_CTRL_MACAS_EN BIT(3)
+#define SXE2_VSI_TX_SW_CTRL_VLAN_PRUNE BIT(9)
+
+#define SXE2_VSI_TAR_UNTAGGED_SHIFT (16)
+
+#define SXE2_PCIE_SYS_READY 0x38c
+#define SXE2_PCIE_SYS_READY_CORER_ASSERT BIT(0)
+#define SXE2_PCIE_SYS_READY_STOP_DROP_DONE BIT(2)
+#define SXE2_PCIE_SYS_READY_R5 BIT(3)
+#define SXE2_PCIE_SYS_READY_STOP_DROP BIT(16)
+
+#define SXE2_PCIE_DEV_CTRL_DEV_STATUS 0x78
+#define SXE2_PCIE_DEV_CTRL_DEV_STATUS_TRANS_PENDING BIT(21)
+
+#define SXE2_TOP_CFG_CORE (SXE2_TOP_CFG_BASE + 0x0630)
+#define SXE2_TOP_CFG_CORE_RST_CODE 0x09FBD586
+
+#define SXE2_PFGEN_CTRL (0x00336000)
+#define SXE2_PFGEN_CTRL_PFSWR BIT(0)
+
+#define SXE2_VFGEN_CTRL(_vf) (0x00337000 + ((_vf) * 4))
+#define SXE2_VFGEN_CTRL_VFSWR BIT(0)
+
+#define SXE2_VF_VRC_VFGEN_RSTAT(_vf) (0x00338000 + (_vf)*4)
+#define SXE2_VF_VRC_VFGEN_VFRSTAT (0x3)
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_VFR (0)
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_COMPLETE (BIT(0))
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_VF_ACTIVE (BIT(1))
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_MASK (BIT(2))
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_FORVF (0x300)
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_FORVF_NO_VFR (0)
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_FORVF_VFR (1)
+#define SXE2_VF_VRC_VFGEN_VFRSTAT_FORVF_MASK (BIT(10))
+
+#define SXE2_GLGEN_VFLRSTAT(_reg) (0x0033A000 + ((_reg)*4))
+
+#define SXE2_ACCEPT_RULE_TAGGED_S 0
+#define SXE2_ACCEPT_RULE_UNTAGGED_S 16
+
+#define SXE2_VF_RXQ_BASE(_VF) (0x000b0800 + ((_VF) * 4))
+#define SXE2_VF_RXQ_BASE_FIRST_Q_S 0
+#define SXE2_VF_RXQ_BASE_FIRST_Q_M (0x7FF << SXE2_VF_RXQ_BASE_FIRST_Q_S)
+#define SXE2_VF_RXQ_BASE_Q_NUM_S 16
+#define SXE2_VF_RXQ_BASE_Q_NUM_M (0x7FF << SXE2_VF_RXQ_BASE_Q_NUM_S)
+
+#define SXE2_VF_RXQ_MAPENA(_VF) (0x000b0400 + ((_VF) * 4))
+#define SXE2_VF_RXQ_MAPENA_M BIT(0)
+
+#define SXE2_VF_TXQ_BASE(_VF) (0x00040400 + ((_VF) * 4))
+#define SXE2_VF_TXQ_BASE_FIRST_Q_S 0
+#define SXE2_VF_TXQ_BASE_FIRST_Q_M (0x3FFF << SXE2_VF_TXQ_BASE_FIRST_Q_S)
+#define SXE2_VF_TXQ_BASE_Q_NUM_S 16
+#define SXE2_VF_TXQ_BASE_Q_NUM_M (0xFF << SXE2_VF_TXQ_BASE_Q_NUM_S)
+
+#define SXE2_VF_TXQ_MAPENA(_VF) (0x00045000 + ((_VF) * 4))
+#define SXE2_VF_TXQ_MAPENA_M BIT(0)
+
+#define PRI_PTP_BASEADDR 0x2a8000
+
+#define GLTSYN (PRI_PTP_BASEADDR + 0x0)
+#define GLTSYN_ENA_M BIT(0)
+
+#define GLTSYN_CMD (PRI_PTP_BASEADDR + 0x4)
+#define GLTSYN_CMD_INIT_TIME 0x01
+#define GLTSYN_CMD_INIT_INCVAL 0x02
+#define GLTSYN_CMD_ADJ_TIME 0x04
+#define GLTSYN_CMD_ADJ_TIME_AT_TIME 0x0C
+#define GLTSYN_CMD_LATCHING_SHTIME 0x80
+
+#define GLTSYN_SYNC (PRI_PTP_BASEADDR + 0x8)
+#define GLTSYN_SYNC_PLUS_1NS 0x1
+#define GLTSYN_SYNC_MINUS_1NS 0x2
+#define GLTSYN_SYNC_EXEC 0x3
+#define GLTSYN_SYNC_GEN_PULSE 0x4
+
+#define GLTSYN_SEM (PRI_PTP_BASEADDR + 0xC)
+#define GLTSYN_SEM_BUSY_M BIT(0)
+
+#define GLTSYN_STAT (PRI_PTP_BASEADDR + 0x10)
+#define GLTSYN_STAT_EVENT0_M BIT(0)
+#define GLTSYN_STAT_EVENT1_M BIT(1)
+#define GLTSYN_STAT_EVENT2_M BIT(2)
+
+#define GLTSYN_TIME_SUBNS (PRI_PTP_BASEADDR + 0x20)
+#define GLTSYN_TIME_NS (PRI_PTP_BASEADDR + 0x24)
+#define GLTSYN_TIME_S_H (PRI_PTP_BASEADDR + 0x28)
+#define GLTSYN_TIME_S_L (PRI_PTP_BASEADDR + 0x2C)
+
+#define GLTSYN_SHTIME_SUBNS (PRI_PTP_BASEADDR + 0x30)
+#define GLTSYN_SHTIME_NS (PRI_PTP_BASEADDR + 0x34)
+#define GLTSYN_SHTIME_S_H (PRI_PTP_BASEADDR + 0x38)
+#define GLTSYN_SHTIME_S_L (PRI_PTP_BASEADDR + 0x3C)
+
+#define GLTSYN_SHADJ_SUBNS (PRI_PTP_BASEADDR + 0x40)
+#define GLTSYN_SHADJ_NS (PRI_PTP_BASEADDR + 0x44)
+
+#define GLTSYN_INCVAL_NS (PRI_PTP_BASEADDR + 0x50)
+#define GLTSYN_INCVAL_SUBNS (PRI_PTP_BASEADDR + 0x54)
+
+#define GLTSYN_TGT_NS(_i) \
+ (PRI_PTP_BASEADDR + 0x60 + ((_i) * 16))
+#define GLTSYN_TGT_S_H(_i) (PRI_PTP_BASEADDR + 0x64 + ((_i) * 16))
+#define GLTSYN_TGT_S_L(_i) (PRI_PTP_BASEADDR + 0x68 + ((_i) * 16))
+
+#define GLTSYN_EVENT_NS(_i) \
+ (PRI_PTP_BASEADDR + 0xA0 + ((_i) * 16))
+
+#define GLTSYN_EVENT_S_H(_i) (PRI_PTP_BASEADDR + 0xA4 + ((_i) * 16))
+#define GLTSYN_EVENT_S_H_MASK (0xFFFF)
+
+#define GLTSYN_EVENT_S_L(_i) (PRI_PTP_BASEADDR + 0xA8 + ((_i) * 16))
+
+#define GLTSYN_AUXOUT(_i) \
+ (PRI_PTP_BASEADDR + 0xD0 + ((_i) * 4))
+#define GLTSYN_AUXOUT_OUT_ENA BIT(0)
+#define GLTSYN_AUXOUT_OUT_MOD (0x03 << 1)
+#define GLTSYN_AUXOUT_OUTLVL BIT(3)
+#define GLTSYN_AUXOUT_INT_ENA BIT(4)
+#define GLTSYN_AUXOUT_PULSEW (0x1fff << 3)
+
+#define GLTSYN_CLKO(_i) \
+ (PRI_PTP_BASEADDR + 0xE0 + ((_i) * 4))
+
+#define GLTSYN_AUXIN(_i) (PRI_PTP_BASEADDR + 0xF4 + ((_i) * 4))
+#define GLTSYN_AUXIN_RISING_EDGE BIT(0)
+#define GLTSYN_AUXIN_FALLING_EDGE BIT(1)
+#define GLTSYN_AUXIN_ENABLE BIT(4)
+
+#define CGMAC_CSR_BASE 0x2B4000
+
+#define CGMAC_PORT_OFFSET 0x00004000
+
+#define PFP_CGM_TX_TSMEM(_port, _i) \
+ (CGMAC_CSR_BASE + 0x100 + \
+ + CGMAC_PORT_OFFSET * _port + ((_i) * 4))
+
+#define PFP_CGM_TX_TXHI(_port, _i) (CGMAC_CSR_BASE + CGMAC_PORT_OFFSET * _port + 0x108 + ((_i) * 8))
+#define PFP_CGM_TX_TXLO(_port, _i) (CGMAC_CSR_BASE + CGMAC_PORT_OFFSET * _port + 0x10C + ((_i) * 8))
+
+#define CGMAC_CSR_MAC0_OFFSET 0x2B4000
+#define CGMAC_CSR_MAC_OFFSET(_i) (CGMAC_CSR_MAC0_OFFSET + ((_i) * 0x4000))
+
+#define PFP_CGM_MAC_TX_TSMEM(_phy, _i) \
+ (CGMAC_CSR_MAC_OFFSET(_phy) + 0x100 + \
+ ((_i) * 4))
+
+#define PFP_CGM_MAC_TX_TXHI(_phy, _i) (CGMAC_CSR_MAC_OFFSET(_phy) + 0x108 + ((_i) * 8))
+#define PFP_CGM_MAC_TX_TXLO(_phy, _i) (CGMAC_CSR_MAC_OFFSET(_phy) + 0x10C + ((_i) * 8))
+
+#define SXE2_VF_GLINT_CEQCTL_MSIX_INDX_M SXE2_BITS_MASK(0x7FF, 0)
+#define SXE2_VF_GLINT_CEQCTL_ITR_INDX_S 11
+#define SXE2_VF_GLINT_CEQCTL_ITR_INDX_M SXE2_BITS_MASK(0x3, 11)
+#define SXE2_VF_GLINT_CEQCTL_CAUSE_ENA_M BIT(30)
+#define SXE2_VF_GLINT_CEQCTL(_INT) (0x0026492C + ((_INT) * 4))
+
+#define SXE2_VF_PFINT_AEQCTL_MSIX_INDX_M SXE2_BITS_MASK(0x7FF, 0)
+#define SXE2_VF_VPINT_AEQCTL_ITR_INDX_S 11
+#define SXE2_VF_VPINT_AEQCTL_ITR_INDX_M SXE2_BITS_MASK(0x3, 11)
+#define SXE2_VF_VPINT_AEQCTL_CAUSE_ENA_M BIT(30)
+#define SXE2_VF_VPINT_AEQCTL(_VF) (0x0026052c + ((_VF) * 4))
+
+#define SXE2_IPSEC_TX_BASE (0x2A0000)
+#define SXE2_IPSEC_RX_BASE (0x2A2000)
+
+#define SXE2_IPSEC_RX_IPSIDX_ADDR (SXE2_IPSEC_RX_BASE + 0x0084)
+#define SXE2_IPSEC_RX_IPSIDX_RST (0x00040000)
+#define SXE2_IPSEC_RX_IPSIDX_VBI_SHIFT (18)
+#define SXE2_IPSEC_RX_IPSIDX_VBI_MASK (0x00040000)
+#define SXE2_IPSEC_RX_IPSIDX_SWRITE_SHIFT (17)
+#define SXE2_IPSEC_RX_IPSIDX_SWRITE_MASK (0x00020000)
+#define SXE2_IPSEC_RX_IPSIDX_SA_IDX_SHIFT (4)
+#define SXE2_IPSEC_RX_IPSIDX_SA_IDX_MASK (0x0000fff0)
+#define SXE2_IPSEC_RX_IPSIDX_TABLE_SHIFT (2)
+#define SXE2_IPSEC_RX_IPSIDX_TABLE_MASK (0x0000000c)
+
+#define SXE2_IPSEC_RX_IPSIPID_ADDR (SXE2_IPSEC_RX_BASE + 0x0088)
+#define SXE2_IPSEC_RX_IPSIPID_IP_ID_X_SHIFT (0)
+#define SXE2_IPSEC_RX_IPSIPID_IP_ID_X_MASK (0x000000ff)
+
+#define SXE2_IPSEC_RX_IPSSPI0_ADDR (SXE2_IPSEC_RX_BASE + 0x008c)
+#define SXE2_IPSEC_RX_IPSSPI0_SPI_X_SHIFT (0)
+#define SXE2_IPSEC_RX_IPSSPI0_SPI_X_MASK (0xffffffff)
+
+#define SXE2_IPSEC_RX_IPSSPI1_ADDR (SXE2_IPSEC_RX_BASE + 0x0090)
+#define SXE2_IPSEC_RX_IPSSPI1_SPI_Y_MASK (0xffffffff)
+
+#define SXE2_PAUSE_STATS_BASE(port) (0x002b2000 + port * 0x4000)
+#define SXE2_TXPAUSEXONFRAMES_LO(port) (SXE2_PAUSE_STATS_BASE(port) + 0x0894)
+#define SXE2_TXPAUSEXOFFFRAMES_LO(port) (SXE2_PAUSE_STATS_BASE(port) + 0x0a18)
+#define SXE2_TXPFCXONFRAMES_LO(port, pri) (SXE2_PAUSE_STATS_BASE(port) + \
+ (0x0a20 + 8 * (pri)))
+#define SXE2_TXPFCXOFFFRAMES_LO(port, pri) (SXE2_PAUSE_STATS_BASE(port) + \
+ (0x0a60 + 8 * (pri)))
+#define SXE2_TXPFCXONTOXOFFFRAMES_LO(port, pri) (SXE2_PAUSE_STATS_BASE(port) + \
+ (0x0aa0 + 8 * (pri)))
+#define SXE2_RXPAUSEXONFRAMES_LO(port) (SXE2_PAUSE_STATS_BASE(port) + 0x0988)
+#define SXE2_RXPAUSEXOFFFRAMES_LO(port) (SXE2_PAUSE_STATS_BASE(port) + 0x0b28)
+#define SXE2_RXPFCXONFRAMES_LO(port, pri) (SXE2_PAUSE_STATS_BASE(port) + \
+ (0x0b30 + 8 * (pri)))
+#define SXE2_RXPFCXOFFFRAMES_LO(port, pri) (SXE2_PAUSE_STATS_BASE(port) + \
+ (0x0b70 + 8 * (pri)))
+
+#endif
diff --git a/drivers/common/sxe2/sxe2_internal_ver.h b/drivers/common/sxe2/sxe2_internal_ver.h
new file mode 100644
index 0000000000..92f49e7a20
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_internal_ver.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_INTERNAL_VER_H__
+#define __SXE2_INTERNAL_VER_H__
+
+#define SXE2_VER_MAJOR_OFFSET (16)
+#define SXE2_MK_VER(major, minor) \
+ (major << SXE2_VER_MAJOR_OFFSET | minor)
+#define SXE2_MK_VER_MAJOR(ver) (((ver) >> SXE2_VER_MAJOR_OFFSET) & 0xff)
+#define SXE2_MK_VER_MINOR(ver) ((ver) & 0xff)
+
+#define SXE2_ITR_VER_MAJOR_V100 1
+#define SXE2_ITR_VER_MAJOR_V200 2
+
+#define SXE2_ITR_VER_MAJOR 1
+#define SXE2_ITR_VER_MINOR 1
+#define SXE2_ITR_VER SXE2_MK_VER(SXE2_ITR_VER_MAJOR, SXE2_ITR_VER_MINOR)
+
+#define SXE2_CTRL_VER_IS_V100(ver) (SXE2_MK_VER_MAJOR(ver) == SXE2_ITR_VER_MAJOR_V100)
+#define SXE2_CTRL_VER_IS_V200(ver) (SXE2_MK_VER_MAJOR(ver) == SXE2_ITR_VER_MAJOR_V200)
+
+#define SXE2LIB_ITR_VER_MAJOR 1
+#define SXE2LIB_ITR_VER_MINOR 1
+#define SXE2LIB_ITR_VER SXE2_MK_VER(SXE2LIB_ITR_VER_MAJOR, SXE2LIB_ITR_VER_MINOR)
+
+#define SXE2_DRV_CLI_VER_MAJOR 1
+#define SXE2_DRV_CLI_VER_MINOR 1
+#define SXE2_DRV_CLI_VER \
+ SXE2_MK_VER(SXE2_DRV_CLI_VER_MAJOR, SXE2_DRV_CLI_VER_MINOR)
+
+#endif /* __SXE2_INTERNAL_VER_H__ */
diff --git a/drivers/common/sxe2/sxe2_osal.h b/drivers/common/sxe2/sxe2_osal.h
new file mode 100644
index 0000000000..d77057e7ee
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_osal.h
@@ -0,0 +1,586 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_OSAL_H__
+#define __SXE2_OSAL_H__
+#include <string.h>
+#include <stdint.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_ether.h>
+#include <rte_version.h>
+
+#include "sxe2_type.h"
+
+#define BIT(nr) (1UL << (nr))
+#ifndef __BITS_PER_LONG
+#define __BITS_PER_LONG (__SIZEOF_LONG__ * 8)
+#endif
+#define BIT_WORD(nr) ((nr) / __BITS_PER_LONG)
+#define BIT_MASK(nr) (1UL << ((nr) % __BITS_PER_LONG))
+
+#ifndef BIT_ULL
+#define BIT_ULL(a) (1ULL << (a))
+#endif
+
+#define MIN(a, b) ((a) < (b) ? (a) : (b))
+
+#define BITS_PER_BYTE 8
+
+#define IS_UNICAST_ETHER_ADDR(addr) \
+ ((bool)((((u8 *)(addr))[0] % ((u8)0x2)) == 0))
+
+#define STRUCT_SIZE(ptr, field, num) \
+ (sizeof(*(ptr)) + sizeof(*(ptr)->field) * (num))
+
+#ifndef TAILQ_FOREACH_SAFE
+#define TAILQ_FOREACH_SAFE(var, head, field, tvar) \
+ for ((var) = TAILQ_FIRST((head)); \
+ (var) && ((tvar) = TAILQ_NEXT((var), field), 1); \
+ (var) = (tvar))
+#endif
+
+#define SXE2_QUEUE_WAIT_RETRY_CNT (50)
+
+#define __iomem
+
+#define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
+#define lower_32_bits(n) ((u32)((n) & 0xffffffff))
+
+#define dma_addr_t rte_iova_t
+
+#define resource_size_t u64
+
+#define FIELD_SIZEOF(t, f) RTE_SIZEOF_FIELD(t, f)
+#define ARRAY_SIZE(arr) RTE_DIM(arr)
+
+#define CPU_TO_LE16(o) rte_cpu_to_le_16(o)
+#define CPU_TO_LE32(s) rte_cpu_to_le_32(s)
+#define CPU_TO_LE64(h) rte_cpu_to_le_64(h)
+#define LE16_TO_CPU(a) rte_le_to_cpu_16(a)
+#define LE32_TO_CPU(c) rte_le_to_cpu_32(c)
+#define LE64_TO_CPU(k) rte_le_to_cpu_64(k)
+
+#define CPU_TO_BE16(o) rte_cpu_to_be_16(o)
+#define CPU_TO_BE32(o) rte_cpu_to_be_32(o)
+#define CPU_TO_BE64(o) rte_cpu_to_be_64(o)
+#define BE16_TO_CPU(o) rte_be_to_cpu_16(o)
+
+#define NTOHS(a) rte_be_to_cpu_16(a)
+#define NTOHL(a) rte_be_to_cpu_32(a)
+#define HTONS(a) rte_cpu_to_be_16(a)
+#define HTONL(a) rte_cpu_to_be_32(a)
+
+#define udelay(x) rte_delay_us(x)
+
+#define mdelay(x) rte_delay_us(1000 * (x))
+
+#define msleep(x) rte_delay_us(1000 * (x))
+
+#ifndef DIV_ROUND_UP
+#define DIV_ROUND_UP(n, d) \
+ (((n) + (typeof(n))(d) - (typeof(n))1) / (typeof(n))(d))
+#endif
+
+#define usleep_range(min) msleep(DIV_ROUND_UP(min, 1000))
+
+#define __bf_shf(x) ((uint32_t)rte_bsf64(x))
+
+#ifndef BITS_PER_LONG
+#define BITS_PER_LONG 32
+#endif
+
+#define FIELD_PREP(mask, val) (((typeof(mask))(val) << __bf_shf(mask)) & (mask))
+#define FIELD_GET(_mask, _reg) ((typeof(_mask))(((_reg) & (_mask)) >> __bf_shf(_mask)))
+
+#define SXE2_NUM_ROUND_UP(n, d) (DIV_ROUND_UP(n, d) * d)
+
+static inline void sxe2_swap_u16(u16 *a, u16 *b)
+{
+ u16 tmp;
+
+ if (unlikely(*a == *b))
+ return;
+ tmp = *a;
+ *a = *b;
+ *b = tmp;
+}
+
+#define SXE2_SWAP_U16(a, b) sxe2_swap_u16(a, b)
+
+enum sxe2_itr_idx {
+ SXE2_ITR_IDX_0 = 0,
+ SXE2_ITR_IDX_1,
+ SXE2_ITR_IDX_2,
+ SXE2_ITR_IDX_NONE,
+};
+
+#define MAX_ERRNO 4095
+#define IS_ERR_VALUE(x) unlikely((uintptr_t)(void *)(x) >= (uintptr_t)-MAX_ERRNO)
+static inline bool IS_ERR(const void *ptr)
+{
+ return IS_ERR_VALUE((uintptr_t)ptr);
+}
+
+#define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
+
+#define SXE2_CTXT_REG_VALUE(value, shift, width) ((value << shift) & \
+ (((1ULL << width) - 1) << shift))
+
+#define ETH_P_8021Q 0x8100
+#define ETH_P_8021AD 0x88a8
+#define ETH_P_QINQ1 0x9100
+
+#define FLEX_ARRAY_SIZE(_ptr, _mem, cnt) ((cnt) * sizeof(_ptr->_mem[0]))
+
+#define sxe2_init_lock(sp) rte_spinlock_init(&(sp)->spinlock)
+#define sxe2_acquire_lock(sp) rte_spinlock_lock(&(sp)->spinlock)
+#define sxe2_release_lock(sp) rte_spinlock_unlock(&(sp)->spinlock)
+#define sxe2_destroy_lock(sp) RTE_SET_USED(sp)
+
+#define COMPILER_BARRIER() \
+ { asm volatile("" ::: "memory"); }
+
+struct sxe2_list_head_type {
+ struct sxe2_list_head_type *next, *prev;
+};
+
+#define LIST_HEAD_TYPE sxe2_list_head_type
+
+#define SXE2_LIST_ENTRY(ptr, type, member) container_of(ptr, type, member)
+#define LIST_FIRST_ENTRY(ptr, type, member) \
+ SXE2_LIST_ENTRY((ptr)->next, type, member)
+#define LIST_NEXT_ENTRY(pos, member) \
+ SXE2_LIST_ENTRY((pos)->member.next, typeof(*(pos)), member)
+
+static inline void INIT_LIST_HEAD(struct LIST_HEAD_TYPE *list)
+{
+ list->next = list;
+ COMPILER_BARRIER();
+ list->prev = list;
+ COMPILER_BARRIER();
+}
+
+static inline void sxe2_list_add(struct LIST_HEAD_TYPE *curr,
+ struct LIST_HEAD_TYPE *prev,
+ struct LIST_HEAD_TYPE *next)
+{
+ next->prev = curr;
+ curr->next = next;
+ curr->prev = prev;
+ COMPILER_BARRIER();
+ prev->next = curr;
+ COMPILER_BARRIER();
+}
+
+#define LIST_ADD(entry, head) sxe2_list_add(entry, (head), (head)->next)
+#define LIST_ADD_TAIL(entry, head) sxe2_list_add(entry, (head)->prev, head)
+
+static inline void __list_del(struct LIST_HEAD_TYPE *prev, struct LIST_HEAD_TYPE *next)
+{
+ next->prev = prev;
+ COMPILER_BARRIER();
+ prev->next = next;
+ COMPILER_BARRIER();
+}
+
+static inline void __list_del_entry(struct LIST_HEAD_TYPE *entry)
+{
+ __list_del(entry->prev, entry->next);
+}
+#define LIST_DEL(entry) __list_del_entry(entry)
+
+static inline bool __list_is_empty(const struct LIST_HEAD_TYPE *head)
+{
+ COMPILER_BARRIER();
+ return head->next == head;
+}
+
+#define LIST_IS_EMPTY(head) __list_is_empty(head)
+
+#define LIST_FOR_EACH_ENTRY(pos, head, member) \
+ for (pos = LIST_FIRST_ENTRY(head, typeof(*pos), member); \
+ &pos->member != (head); \
+ pos = LIST_NEXT_ENTRY(pos, member))
+
+#define LIST_FOR_EACH_ENTRY_SAFE(pos, n, head, member) \
+ for (pos = LIST_FIRST_ENTRY(head, typeof(*pos), member), \
+ n = LIST_NEXT_ENTRY(pos, member); \
+ &pos->member != (head); \
+ pos = n, n = LIST_NEXT_ENTRY(n, member))
+
+struct sxe2_blk_list_head_type {
+ struct sxe2_blk_list_head_type *next_blk;
+ struct sxe2_blk_list_head_type *next;
+ u16 blk_size;
+ u16 blk_id;
+};
+
+#define BLK_LIST_HEAD_TYPE sxe2_blk_list_head_type
+
+static inline void sxe2_blk_list_add(struct BLK_LIST_HEAD_TYPE *node,
+ struct BLK_LIST_HEAD_TYPE *head)
+{
+ struct BLK_LIST_HEAD_TYPE *curr = head->next_blk;
+ struct BLK_LIST_HEAD_TYPE *prev = head;
+
+ while (curr != NULL && curr->blk_id < node->blk_id) {
+ prev = curr;
+ curr = curr->next_blk;
+ }
+
+ if (prev != head && prev->blk_id + prev->blk_size == node->blk_id) {
+ prev->blk_size += node->blk_size;
+ node->blk_size = 0;
+ } else {
+ node->next_blk = curr;
+ prev->next_blk = node;
+ }
+
+ node = (node->blk_size == 0) ? prev : node;
+
+ if (curr) {
+
+ if (node->blk_id + node->blk_size == curr->blk_id) {
+ node->blk_size += curr->blk_size;
+ curr->blk_size = 0;
+ node->next_blk = curr->next_blk;
+ } else {
+ node->next_blk = curr;
+ }
+ }
+}
+
+static inline struct BLK_LIST_HEAD_TYPE *sxe2_blk_list_get(
+ struct BLK_LIST_HEAD_TYPE *head, u16 blk_size)
+{
+ struct BLK_LIST_HEAD_TYPE *curr = head->next_blk;
+ struct BLK_LIST_HEAD_TYPE *prev = head;
+ struct BLK_LIST_HEAD_TYPE *blk_max_node = curr;
+ struct BLK_LIST_HEAD_TYPE *blk_max_node_pre = head;
+ struct BLK_LIST_HEAD_TYPE *ret = NULL;
+ s32 i = blk_size;
+
+ while (curr && curr->blk_size != blk_size) {
+ if (curr->blk_size > blk_max_node->blk_size) {
+ blk_max_node = curr;
+ blk_max_node_pre = prev;
+ }
+ prev = curr;
+ curr = curr->next_blk;
+ }
+
+ if (curr != NULL) {
+ prev->next_blk = curr->next_blk;
+ ret = curr;
+ goto l_end;
+ }
+
+ if (blk_max_node->blk_size < blk_size)
+ goto l_end;
+
+ ret = blk_max_node;
+ prev = blk_max_node_pre;
+
+ curr = blk_max_node;
+ while (i != 0) {
+ curr = curr->next;
+ i--;
+ }
+ curr->blk_size = blk_max_node->blk_size - blk_size;
+ blk_max_node->blk_size = blk_size;
+ prev->next_blk = curr;
+
+l_end:
+ return ret;
+}
+
+#define BLK_LIST_ADD(entry, head) sxe2_blk_list_add(entry, head)
+#define BLK_LIST_GET(head, blk_size) sxe2_blk_list_get(head, blk_size)
+
+#ifndef BIT_ULL
+#define BIT_ULL(nr) (ULL(1) << (nr))
+#endif
+
+static inline bool check_is_pow2(u64 val)
+{
+ return (val && !(val & (val - 1)));
+}
+
+static inline u8 sxe2_setbit_cnt8(u8 num)
+{
+ u8 bits = 0;
+ u32 i;
+
+ for (i = 0; i < 8; i++) {
+ bits += (num & 0x1);
+ num >>= 1;
+ }
+
+ return bits;
+}
+
+static inline bool max_set_bit_check(const u8 *mask, u16 size, u16 max)
+{
+ u16 count = 0;
+ u16 i;
+ bool ret = false;
+
+ for (i = 0; i < size; i++) {
+ if (!mask[i])
+ continue;
+
+ if (count == max)
+ goto l_end;
+
+ count += sxe2_setbit_cnt8(mask[i]);
+ if (count > max)
+ goto l_end;
+ }
+
+ ret = true;
+l_end:
+ return ret;
+}
+
+#define BITS_TO_LONGS(nr) DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(unsigned long))
+#define BITS_TO_U32(nr) DIV_ROUND_UP(nr, 32)
+
+#define GENMASK(h, l) \
+ (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
+
+#define BITMAP_LAST_WORD_MASK(nbits) (~0UL >> (-(nbits) & (__BITS_PER_LONG - 1)))
+
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+#define BITMAP_MEM_ALIGNMENT 8
+#else
+#define BITMAP_MEM_ALIGNMENT (8 * sizeof(unsigned long))
+#endif
+#define BITMAP_MEM_MASK (BITMAP_MEM_ALIGNMENT - 1)
+#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
+
+#define DECLARE_BITMAP(name, bits) \
+ unsigned long name[BITS_TO_LONGS(bits)]
+#define BITMAP_TYPE unsigned long
+#define small_const_nbits(nbits) \
+ (__rte_constant(nbits) && (nbits) <= __BITS_PER_LONG && (nbits) > 0)
+
+static inline void set_bit(u32 nr, unsigned long *addr)
+{
+ addr[nr / __BITS_PER_LONG] |= 1UL << (nr % __BITS_PER_LONG);
+}
+
+static inline void clear_bit(u32 nr, unsigned long *addr)
+{
+ addr[nr / __BITS_PER_LONG] &= ~(1UL << (nr % __BITS_PER_LONG));
+}
+
+static inline u32 test_bit(u32 nr, const volatile unsigned long *addr)
+{
+ return 1UL & (addr[BIT_WORD(nr)] >> (nr & (__BITS_PER_LONG-1)));
+}
+
+static inline u32 bitmap_weight(const unsigned long *src, u32 nbits)
+{
+ u32 cnt = 0;
+ u16 i;
+ for (i = 0; i < nbits; i++) {
+ if (test_bit(i, src))
+ cnt++;
+ }
+ return cnt;
+}
+
+static inline bool bitmap_empty(const unsigned long *src, u32 nbits)
+{
+ u16 i;
+ for (i = 0; i < nbits; i++) {
+ if (test_bit(i, src))
+ return false;
+ }
+ return true;
+}
+
+static inline void bitmap_zero(unsigned long *dst, u32 nbits)
+{
+ u32 len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
+ memset(dst, 0, len);
+}
+
+static bool __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, u32 bits)
+{
+ u32 k;
+ u32 lim = bits/__BITS_PER_LONG;
+ unsigned long result = 0;
+ for (k = 0; k < lim; k++)
+ result |= (dst[k] = bitmap1[k] & bitmap2[k]);
+ if (bits % __BITS_PER_LONG)
+ result |= (dst[k] = bitmap1[k] & bitmap2[k] &
+ BITMAP_LAST_WORD_MASK(bits));
+ return result != 0;
+}
+
+static inline bool bitmap_and(unsigned long *dst, const unsigned long *src1,
+ const unsigned long *src2, u32 nbits)
+{
+ if (small_const_nbits(nbits))
+ return (*dst = *src1 & *src2 & BITMAP_LAST_WORD_MASK(nbits)) != 0;
+ return __bitmap_and(dst, src1, src2, nbits);
+}
+
+static void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits)
+{
+ int k;
+ int nr = BITS_TO_LONGS(bits);
+
+ for (k = 0; k < nr; k++)
+ dst[k] = bitmap1[k] | bitmap2[k];
+}
+
+static inline void bitmap_or(unsigned long *dst, const unsigned long *src1,
+ const unsigned long *src2, u32 nbits)
+{
+ if (small_const_nbits(nbits))
+ *dst = *src1 | *src2;
+ else
+ __bitmap_or(dst, src1, src2, nbits);
+}
+
+static int __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, u32 bits)
+{
+ u32 k;
+ u32 lim = bits/__BITS_PER_LONG;
+ unsigned long result = 0;
+
+ for (k = 0; k < lim; k++)
+ result |= (dst[k] = bitmap1[k] & ~bitmap2[k]);
+ if (bits % __BITS_PER_LONG)
+ result |= (dst[k] = bitmap1[k] & ~bitmap2[k] &
+ BITMAP_LAST_WORD_MASK(bits));
+ return result != 0;
+}
+
+static inline int bitmap_andnot(unsigned long *dst, const unsigned long *src1,
+ const unsigned long *src2, u32 nbits)
+{
+ if (small_const_nbits(nbits))
+ return (*dst = *src1 & ~(*src2) & BITMAP_LAST_WORD_MASK(nbits)) != 0;
+ return __bitmap_andnot(dst, src1, src2, nbits);
+}
+
+static bool __bitmap_equal(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, u32 bits)
+{
+ u32 k, lim = bits/__BITS_PER_LONG;
+ for (k = 0; k < lim; ++k)
+ if (bitmap1[k] != bitmap2[k])
+ return false;
+
+ if (bits % __BITS_PER_LONG)
+ if ((bitmap1[k] ^ bitmap2[k]) & BITMAP_LAST_WORD_MASK(bits))
+ return false;
+
+ return true;
+}
+
+static inline bool bitmap_equal(const unsigned long *src1,
+ const unsigned long *src2, u32 nbits)
+{
+ if (small_const_nbits(nbits))
+ return !((*src1 ^ *src2) & BITMAP_LAST_WORD_MASK(nbits));
+ if (__rte_constant(nbits & BITMAP_MEM_MASK) &&
+ IS_ALIGNED(nbits, BITMAP_MEM_ALIGNMENT))
+ return !memcmp(src1, src2, nbits / 8);
+ return __bitmap_equal(src1, src2, nbits);
+}
+
+static inline unsigned long
+find_next_bit(const unsigned long *addr, unsigned long size,
+ unsigned long offset)
+{
+ u16 i;
+
+ for (i = offset; i < size; i++) {
+ if (test_bit(i, addr))
+ break;
+ }
+ return i;
+}
+
+static inline unsigned long
+find_next_zero_bit(const unsigned long *addr, unsigned long size,
+ unsigned long offset)
+{
+ u16 i;
+ for (i = offset; i < size; i++) {
+ if (!test_bit(i, addr))
+ break;
+ }
+ return i;
+}
+
+static inline void bitmap_copy(unsigned long *dst, const unsigned long *src,
+ u32 nbits)
+{
+ u32 len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
+ memcpy(dst, src, len);
+}
+
+static inline unsigned long find_first_zero_bit(const unsigned long *addr, unsigned long size)
+{
+ return find_next_zero_bit(addr, size, 0);
+}
+
+static inline unsigned long find_first_bit(const unsigned long *addr, unsigned long size)
+{
+ return find_next_bit(addr, size, 0);
+}
+
+#define for_each_clear_bit(bit, addr, size) \
+ for ((bit) = find_first_zero_bit((addr), (size)); \
+ (bit) < (size); \
+ (bit) = find_next_zero_bit((addr), (size), (bit) + 1))
+
+#define for_each_set_bit(bit, addr, size) \
+ for ((bit) = find_first_bit((addr), (size)); \
+ (bit) < (size); \
+ (bit) = find_next_bit((addr), (size), (bit) + 1))
+
+struct sxe2_adapter;
+
+static inline void *sxe2_malloc(__rte_unused struct sxe2_adapter *ad, size_t size)
+{
+ return rte_zmalloc(NULL, size, 0);
+}
+
+static inline void *sxe2_calloc(__rte_unused struct sxe2_adapter *ad, size_t num, size_t size)
+{
+ return rte_calloc(NULL, num, size, 0);
+}
+
+static inline void sxe2_free(__rte_unused struct sxe2_adapter *ad, void *ptr)
+{
+ rte_free(ptr);
+}
+
+static inline void *sxe2_memdup(__rte_unused struct sxe2_adapter *ad,
+ const void *src, size_t size)
+{
+ void *p;
+
+ p = sxe2_malloc(ad, size);
+ if (p)
+ rte_memcpy(p, src, size);
+ return p;
+}
+
+#endif
diff --git a/drivers/common/sxe2/sxe2_type.h b/drivers/common/sxe2/sxe2_type.h
new file mode 100644
index 0000000000..e4ef6ed2ce
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_type.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TYPES_H__
+#define __SXE2_TYPES_H__
+
+#include <sys/time.h>
+
+#include <stdlib.h>
+#include <errno.h>
+#include <stdarg.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdint.h>
+
+#if defined __BYTE_ORDER__
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+#define __BIG_ENDIAN_BITFIELD
+#elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+#define __LITTLE_ENDIAN_BITFIELD
+#endif
+#elif defined __BYTE_ORDER
+#if __BYTE_ORDER == __BIG_ENDIAN
+#define __BIG_ENDIAN_BITFIELD
+#elif __BYTE_ORDER == __LITTLE_ENDIAN
+#define __LITTLE_ENDIAN_BITFIELD
+#endif
+#elif defined __BIG_ENDIAN__
+#define __BIG_ENDIAN_BITFIELD
+#elif defined __LITTLE_ENDIAN__
+#define __LITTLE_ENDIAN_BITFIELD
+#elif defined RTE_TOOLCHAIN_MSVC
+#define __LITTLE_ENDIAN_BITFIELD
+#else
+#error "Unknown endianness."
+#endif
+typedef uint8_t u8;
+typedef uint16_t u16;
+typedef uint32_t u32;
+typedef uint64_t u64;
+
+typedef int8_t s8;
+typedef int16_t s16;
+typedef int32_t s32;
+typedef int64_t s64;
+
+#define __le16 u16
+#define __le32 u32
+#define __le64 u64
+
+#define __be16 u16
+#define __be32 u32
+#define __be64 u64
+
+#define STATIC static
+
+#define ETH_ALEN 6
+
+#endif /* __SXE2_TYPES_H__ */
--
2.47.3
^ permalink raw reply related
* [PATCH v13 01/10] mailmap: add Jie Liu
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
.mailmap | 1 +
1 file changed, 1 insertion(+)
diff --git a/.mailmap b/.mailmap
index 895412e568..d2c4485636 100644
--- a/.mailmap
+++ b/.mailmap
@@ -739,6 +739,7 @@ Jiawen Wu <jiawenwu@trustnetic.com>
Jiayu Hu <hujiayu.hu@foxmail.com> <jiayu.hu@intel.com>
Jie Hai <haijie1@huawei.com>
Jie Liu <jie2.liu@hxt-semitech.com>
+Jie Liu <liujie5@linkdatatechnology.com>
Jie Pan <panjie5@jd.com>
Jie Wang <jie1x.wang@intel.com>
Jie Zhou <jizh@linux.microsoft.com> <jizh@microsoft.com>
--
2.47.3
^ permalink raw reply related
* [PATCH v13 06/10] drivers: support PCI BAR mapping
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Implement PCI BAR (Base Address Register) mapping and unmapping
logic to enable MMIO (Memory Mapped I/O) access to hardware
registers.
The driver retrieves the BAR0 virtual address from the PCI resource
during the probing phase. This mapping is used for subsequent
register-level operations. Proper cleanup is implemented in the
device close path.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_ioctl_chnl.c | 40 +++-
drivers/net/sxe2/sxe2_ethdev.c | 307 ++++++++++++++++++++++++++
drivers/net/sxe2/sxe2_ethdev.h | 18 ++
3 files changed, 362 insertions(+), 3 deletions(-)
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
index 4b041765de..80fccc6a11 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl.c
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -160,6 +160,40 @@ sxe2_drv_dev_handshark(struct sxe2_common_device *cdev)
return ret;
}
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_mmap)
+void
+*sxe2_drv_dev_mmap(struct sxe2_common_device *cdev, u8 bar_idx, u64 len, u64 offset)
+{
+ s32 cmd_fd = 0;
+ void *virt = NULL;
+
+ if (cdev->config.kernel_reset) {
+ PMD_LOG_WARN(COM, "kernel reset, need restart app.");
+ goto l_err;
+ }
+
+ cmd_fd = SXE2_CDEV_TO_CMD_FD(cdev);
+ if (cmd_fd < 0) {
+ PMD_LOG_ERR(COM, "Failed to exec cmd, fd=%d", cmd_fd);
+ goto l_err;
+ }
+
+ PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=0x%zx, src=0x%"PRIx64", offset=0x%"PRIx64"",
+ bar_idx, cmd_fd, len, offset, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
+
+ virt = mmap(NULL, len, PROT_READ | PROT_WRITE,
+ MAP_SHARED, cmd_fd, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
+ if (virt == MAP_FAILED) {
+ PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=0x%zx, offset=0x%"PRIx64", err:%s",
+ cmd_fd, len, offset, strerror(errno));
+ goto l_err;
+ }
+
+ return virt;
+l_err:
+ return NULL;
+}
+
RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_munmap)
s32
sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
@@ -167,8 +201,8 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
s32 ret = SXE2_SUCCESS;
if (cdev->config.kernel_reset) {
- ret = SXE2_ERR_PERM;
- PMD_LOG_WARN(COM, "kernel reset, need restart app.");
+ ret = -EPERM;
+ PMD_LOG_WARN(COM, "kernel reseted, need restart app.");
goto l_end;
}
@@ -179,7 +213,7 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
if (ret < 0) {
PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=0x%zx, err:%s",
virt, len, strerror(errno));
- ret = SXE2_ERR_IO;
+ ret = -EIO;
goto l_end;
}
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index a6cb51789e..4836c338bc 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -54,6 +54,21 @@ static const struct rte_pci_id pci_id_sxe2_tbl[] = {
{ .vendor_id = 0, },
};
+static struct sxe2_pci_map_addr_info sxe2_net_map_addr_info_pf[SXE2_PCI_MAP_RES_MAX_COUNT] = {
+ /* SXE2_PCI_MAP_RES_INVALID */
+ {0, 0, 0},
+ /* SXE2_PCI_MAP_RES_DOORBELL_TX */
+ { SXE2_TXQ_LEGACY_DBLL(0), 0, 4},
+ /* SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL */
+ { SXE2_RXQ_TAIL(0), 0, 4},
+ /* SXE2_PCI_MAP_RES_IRQ_DYN */
+ { SXE2_VF_DYN_CTL(0), 0, 4},
+ /* SXE2_PCI_MAP_RES_IRQ_ITR(默认使用ITR0) */
+ { SXE2_VF_INT_ITR(0, 0), 0, 4},
+ /* SXE2_PCI_MAP_RES_IRQ_MSIX */
+ { SXE2_BAR4_MSIX_CTL(0), 4, 0x10},
+};
+
static s32 sxe2_dev_configure(struct rte_eth_dev *dev)
{
s32 ret = SXE2_SUCCESS;
@@ -151,6 +166,7 @@ static s32 sxe2_dev_close(struct rte_eth_dev *dev)
(void)sxe2_dev_stop(dev);
sxe2_vsi_uninit(dev);
+ sxe2_dev_pci_map_uinit(dev);
return SXE2_SUCCESS;
}
@@ -287,6 +303,31 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_infos_get = sxe2_dev_infos_get,
};
+struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type)
+{
+ struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ u8 bar_idx = SXE2_PCI_MAP_BAR_INVALID;
+ u8 i;
+
+ bar_idx = map_ctxt->addr_info[res_type].bar_idx;
+ if (bar_idx == SXE2_PCI_MAP_BAR_INVALID) {
+ PMD_DEV_LOG_ERR(adapter, INIT, "Invalid bar index with resource type %d", res_type);
+ goto l_end;
+ }
+
+ for (i = 0; i < map_ctxt->bar_cnt; i++) {
+ if (bar_idx == map_ctxt->bar_info[i].bar_idx) {
+ bar_info = &map_ctxt->bar_info[i];
+ break;
+ }
+ }
+
+l_end:
+ return bar_info;
+}
+
static void sxe2_drv_dev_caps_set(struct sxe2_adapter *adapter,
struct sxe2_drv_dev_caps_resp *dev_caps)
{
@@ -354,6 +395,67 @@ static s32 sxe2_dev_caps_get(struct sxe2_adapter *adapter)
return ret;
}
+s32 sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type, u64 org_len, u64 org_offset)
+{
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ struct sxe2_pci_map_segment_info *seg_info = NULL;
+ void *map_addr = NULL;
+ s32 ret = SXE2_SUCCESS;
+ size_t page_size = 0;
+ size_t aligned_len = 0;
+ size_t page_inner_offset = 0;
+ off_t aligned_offset = 0;
+ u8 i = 0;
+
+ if (org_len == 0) {
+ PMD_DEV_LOG_ERR(adapter, INIT, "Invalid length, ori_len = 0");
+ ret = -EFAULT;
+ goto l_end;
+ }
+
+ bar_info = sxe2_dev_get_bar_info(adapter, res_type);
+ if (!bar_info) {
+ PMD_LOG_ERR(INIT, "Failed to get bar info, res_type=[%d]", res_type);
+ ret = -EFAULT;
+ goto l_end;
+ }
+ seg_info = bar_info->seg_info;
+
+ page_size = rte_mem_page_size();
+
+ aligned_offset = RTE_ALIGN_FLOOR(org_offset, page_size);
+ page_inner_offset = org_offset - aligned_offset;
+ aligned_len = RTE_ALIGN(page_inner_offset + org_len, page_size);
+
+ map_addr = sxe2_drv_dev_mmap(adapter->cdev, bar_info->bar_idx, aligned_len, aligned_offset);
+ if (!map_addr) {
+ PMD_LOG_ERR(INIT, "Failed to mmap BAR space, type=%d, len=%zu, page_size=%zu",
+ res_type, org_len, page_size);
+ ret = -EFAULT;
+ goto l_end;
+ }
+
+ for (i = 0; i < bar_info->map_cnt; i++) {
+ if (seg_info[i].type != SXE2_PCI_MAP_RES_INVALID)
+ continue;
+ seg_info[i].type = res_type;
+ seg_info[i].addr = map_addr;
+ seg_info[i].page_inner_offset = page_inner_offset;
+ seg_info[i].len = aligned_len;
+ break;
+ }
+ if (i == bar_info->map_cnt) {
+ PMD_LOG_ERR(INIT, "No memory to save resource, res_type=%d", res_type);
+ ret = -ENOMEM;
+ sxe2_drv_dev_munmap(adapter->cdev, map_addr, aligned_len);
+ goto l_end;
+ }
+
+l_end:
+ return ret;
+}
+
static s32 sxe2_hw_init(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
@@ -368,6 +470,54 @@ static s32 sxe2_hw_init(struct rte_eth_dev *dev)
return ret;
}
+s32 sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter, u32 res_type,
+ u32 item_cnt, u32 item_base)
+{
+ struct sxe2_pci_map_addr_info *addr_info = NULL;
+ s32 ret = SXE2_SUCCESS;
+
+ addr_info = &adapter->map_ctxt.addr_info[res_type];
+ if (!addr_info || addr_info->bar_idx == SXE2_PCI_MAP_BAR_INVALID) {
+ PMD_DEV_LOG_ERR(adapter, INIT, "Invalid bar index with resource type %d", res_type);
+ ret = -EFAULT;
+ goto l_end;
+ }
+
+ ret = sxe2_dev_pci_seg_map(adapter, res_type, item_cnt * addr_info->reg_width,
+ addr_info->addr_base + item_base * addr_info->reg_width);
+ if (ret != SXE2_SUCCESS) {
+ PMD_DEV_LOG_ERR(adapter, INIT, "Failed to map resource, res_type=%d", res_type);
+ goto l_end;
+ }
+l_end:
+ return ret;
+}
+
+void sxe2_dev_pci_seg_unmap(struct sxe2_adapter *adapter, u32 res_type)
+{
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ struct sxe2_pci_map_segment_info *seg_info = NULL;
+ u32 i = 0;
+
+ bar_info = sxe2_dev_get_bar_info(adapter, res_type);
+ if (bar_info == NULL) {
+ PMD_DEV_LOG_WARN(adapter, INIT, "Failed to get bar info, res_type=[%d]", res_type);
+ goto l_end;
+ }
+ seg_info = bar_info->seg_info;
+
+ for (i = 0; i < bar_info->map_cnt; i++) {
+ if (res_type == seg_info[i].type) {
+ (void)sxe2_drv_dev_munmap(adapter->cdev, seg_info[i].addr, seg_info[i].len);
+ memset(&seg_info[i], 0, sizeof(struct sxe2_pci_map_segment_info));
+ break;
+ }
+ }
+
+l_end:
+ return;
+}
+
static s32 sxe2_dev_info_init(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter =
@@ -408,6 +558,157 @@ static s32 sxe2_dev_info_init(struct rte_eth_dev *dev)
return ret;
}
+s32 sxe2_dev_pci_map_init(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev->device);
+ struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ struct sxe2_pci_map_segment_info *seg_info = NULL;
+ u16 txq_cnt = adapter->q_ctxt.qp_cnt_assign;
+ u16 txq_base = adapter->q_ctxt.base_idx_in_pf;
+ u16 rxq_cnt = adapter->q_ctxt.qp_cnt_assign;
+ u16 irq_cnt = adapter->irq_ctxt.max_cnt_hw;
+ u16 irq_base = adapter->irq_ctxt.base_idx_in_func;
+ u16 rxq_base = adapter->q_ctxt.base_idx_in_pf;
+ s32 ret = SXE2_SUCCESS;
+
+ PMD_INIT_FUNC_TRACE();
+
+ adapter->dev_info.dev_data = dev->data;
+
+ if (!pci_dev->mem_resource[0].phys_addr) {
+ PMD_LOG_ERR(INIT, "Physical address not scanned");
+ ret = -ENXIO;
+ goto l_end;
+ }
+
+ map_ctxt->bar_cnt = 2;
+
+ bar_info = rte_zmalloc(NULL, sizeof(*bar_info) * map_ctxt->bar_cnt, 0);
+ if (!bar_info) {
+ PMD_LOG_ERR(INIT, "Failed to alloc bar_info");
+ ret = -ENOMEM;
+ goto l_end;
+ }
+ bar_info[0].bar_idx = 0;
+ bar_info[0].map_cnt = SXE2_PCI_MAP_RES_MAX_COUNT;
+ seg_info = rte_zmalloc(NULL, sizeof(*seg_info) * bar_info[0].map_cnt, 0);
+ if (!seg_info) {
+ PMD_LOG_ERR(INIT, "Failed to alloc seg_info");
+ ret = -ENOMEM;
+ goto l_free_bar;
+ }
+
+ bar_info[0].seg_info = seg_info;
+
+ bar_info[1].bar_idx = 4;
+ bar_info[1].map_cnt = SXE2_PCI_MAP_RES_MAX_COUNT;
+ seg_info = rte_zmalloc(NULL, sizeof(*seg_info) * bar_info[1].map_cnt, 0);
+ if (!seg_info) {
+ PMD_LOG_ERR(INIT, "Failed to alloc seg_info");
+ ret = -ENOMEM;
+ goto l_free_seg0;
+ }
+
+ bar_info[1].seg_info = seg_info;
+ map_ctxt->bar_info = bar_info;
+
+ map_ctxt->addr_info = sxe2_net_map_addr_info_pf;
+
+ ret = sxe2_dev_pci_res_seg_map(adapter, SXE2_PCI_MAP_RES_DOORBELL_TX,
+ txq_cnt, txq_base);
+ if (!ret) {
+ PMD_LOG_ERR(INIT, "Failed to map txq doorbell addr, ret=%d", ret);
+ goto l_free_seg1;
+ }
+
+ ret = sxe2_dev_pci_res_seg_map(adapter, SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL,
+ rxq_cnt, rxq_base);
+ if (!ret) {
+ PMD_LOG_ERR(INIT, "Failed to map rxq tail doorbell addr, ret=%d", ret);
+ goto l_free_txq;
+ }
+
+ ret = sxe2_dev_pci_res_seg_map(adapter, SXE2_PCI_MAP_RES_IRQ_DYN,
+ irq_cnt, irq_base);
+ if (!ret) {
+ PMD_LOG_ERR(INIT, "Failed to map irq dyn addr, ret=%d", ret);
+ goto l_free_rxq_tail;
+ }
+
+ ret = sxe2_dev_pci_res_seg_map(adapter, SXE2_PCI_MAP_RES_IRQ_ITR,
+ irq_cnt, irq_base);
+ if (!ret) {
+ PMD_LOG_ERR(INIT, "Failed to map irq itr addr, ret=%d", ret);
+ goto l_free_irq_dyn;
+ }
+
+ ret = sxe2_dev_pci_res_seg_map(adapter, SXE2_PCI_MAP_RES_IRQ_MSIX,
+ irq_cnt, irq_base);
+ if (!ret) {
+ PMD_LOG_ERR(INIT, "Failed to map irq msix addr, ret=%d", ret);
+ goto l_free_irq_itr;
+ }
+ goto l_end;
+
+l_free_irq_itr:
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_IRQ_ITR);
+l_free_irq_dyn:
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_IRQ_DYN);
+l_free_rxq_tail:
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL);
+l_free_txq:
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_DOORBELL_TX);
+l_free_seg1:
+ if (bar_info[1].seg_info) {
+ rte_free(bar_info[1].seg_info);
+ bar_info[1].seg_info = NULL;
+ }
+l_free_seg0:
+ if (bar_info[0].seg_info) {
+ rte_free(bar_info[0].seg_info);
+ bar_info[0].seg_info = NULL;
+ }
+l_free_bar:
+ if (bar_info) {
+ rte_free(bar_info);
+ bar_info = NULL;
+ }
+l_end:
+ return ret;
+}
+
+void sxe2_dev_pci_map_uinit(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ u8 i = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL);
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_DOORBELL_TX);
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_IRQ_DYN);
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_IRQ_ITR);
+ (void)sxe2_dev_pci_seg_unmap(adapter, SXE2_PCI_MAP_RES_IRQ_MSIX);
+
+ if (map_ctxt != NULL && map_ctxt->bar_info != NULL) {
+ for (i = 0; i < map_ctxt->bar_cnt; i++) {
+ bar_info = &map_ctxt->bar_info[i];
+ if (bar_info != NULL && bar_info->seg_info != NULL) {
+ rte_free(bar_info->seg_info);
+ bar_info->seg_info = NULL;
+ }
+ }
+ rte_free(map_ctxt->bar_info);
+ map_ctxt->bar_info = NULL;
+ }
+
+ adapter->dev_info.dev_data = NULL;
+}
+
static s32 sxe2_dev_init(struct rte_eth_dev *dev, struct sxe2_dev_kvargs_info *kvargs __rte_unused)
{
s32 ret = 0;
@@ -425,6 +726,12 @@ static s32 sxe2_dev_init(struct rte_eth_dev *dev, struct sxe2_dev_kvargs_info *k
goto l_end;
}
+ ret = sxe2_dev_pci_map_init(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to pci addr map, ret=[%d]", ret);
+ goto l_end;
+ }
+
ret = sxe2_vsi_init(dev);
if (ret) {
PMD_LOG_ERR(INIT, "create main vsi failed, ret=%d", ret);
diff --git a/drivers/net/sxe2/sxe2_ethdev.h b/drivers/net/sxe2/sxe2_ethdev.h
index 412f5d2b14..698e2ee4a2 100644
--- a/drivers/net/sxe2/sxe2_ethdev.h
+++ b/drivers/net/sxe2/sxe2_ethdev.h
@@ -292,4 +292,22 @@ struct sxe2_adapter {
#define SXE2_DEV_PRIVATE_TO_ADAPTER(dev) \
((struct sxe2_adapter *)(dev)->data->dev_private)
+#define SXE2_DEV_TO_PCI(eth_dev) \
+ RTE_DEV_TO_PCI((eth_dev)->device)
+
+struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type);
+
+s32 sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type, u64 org_len, u64 org_offset);
+
+s32 sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter, u32 res_type,
+ u32 item_cnt, u32 item_base);
+
+void sxe2_dev_pci_seg_unmap(struct sxe2_adapter *adapter, u32 res_type);
+
+s32 sxe2_dev_pci_map_init(struct rte_eth_dev *dev);
+
+void sxe2_dev_pci_map_uinit(struct rte_eth_dev *dev);
+
#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v13 04/10] drivers: add base driver skeleton
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Initialize the sxe2 PMD skeleton by implementing the PCI probe and
remove functions. This includes the setup and cleanup of a character
device used for control path communication between the user space
and the hardware.
The character device provides an interface for ioctl-based management
operations, supporting device-specific configuration.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/meson.build | 15 +
drivers/common/sxe2/sxe2_common.c | 636 +++++++++++++++++++++
drivers/common/sxe2/sxe2_common.h | 86 +++
drivers/common/sxe2/sxe2_ioctl_chnl.c | 161 ++++++
drivers/common/sxe2/sxe2_ioctl_chnl.h | 141 +++++
drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 45 ++
drivers/meson.build | 1 +
7 files changed, 1085 insertions(+)
create mode 100644 drivers/common/sxe2/meson.build
create mode 100644 drivers/common/sxe2/sxe2_common.c
create mode 100644 drivers/common/sxe2/sxe2_common.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.c
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl_func.h
diff --git a/drivers/common/sxe2/meson.build b/drivers/common/sxe2/meson.build
new file mode 100644
index 0000000000..f1cc1205a0
--- /dev/null
+++ b/drivers/common/sxe2/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2023 Corigine, Inc.
+
+if is_windows
+ build = false
+ reason = 'only supported on Linux'
+ subdir_done()
+endif
+
+deps += ['bus_pci', 'net', 'eal', 'ethdev']
+
+sources = files(
+ 'sxe2_common.c',
+ 'sxe2_ioctl_chnl.c',
+)
diff --git a/drivers/common/sxe2/sxe2_common.c b/drivers/common/sxe2/sxe2_common.c
new file mode 100644
index 0000000000..62bdc93b5c
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_common.c
@@ -0,0 +1,636 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_version.h>
+#include <rte_pci.h>
+#include <rte_dev.h>
+#include <rte_devargs.h>
+#include <rte_class.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+#include <rte_fbarray.h>
+#include <rte_eal.h>
+#include <eal_private.h>
+#include <eal_memcfg.h>
+#include <bus_driver.h>
+#include <bus_pci_driver.h>
+#include <eal_export.h>
+#include <pthread.h>
+
+#include "sxe2_errno.h"
+#include "sxe2_common.h"
+#include "sxe2_common_log.h"
+#include "sxe2_ioctl_chnl_func.h"
+
+static TAILQ_HEAD(sxe2_class_drivers, sxe2_class_driver) sxe2_class_drivers_list =
+ TAILQ_HEAD_INITIALIZER(sxe2_class_drivers_list);
+
+static TAILQ_HEAD(sxe2_common_devices, sxe2_common_device) sxe2_common_devices_list =
+ TAILQ_HEAD_INITIALIZER(sxe2_common_devices_list);
+
+static pthread_mutex_t sxe2_common_devices_list_lock;
+
+static struct rte_pci_id *sxe2_common_pci_id_table;
+
+static const struct {
+ const char *name;
+ u32 class_type;
+} sxe2_class_types[] = {
+ { .name = "eth", .class_type = SXE2_CLASS_TYPE_ETH },
+ { .name = "vdpa", .class_type = SXE2_CLASS_TYPE_VDPA },
+};
+
+static u32 sxe2_class_name_to_value(const char *class_name)
+{
+ u32 class_type = SXE2_CLASS_TYPE_INVALID;
+ u32 i;
+
+ for (i = 0; i < RTE_DIM(sxe2_class_types); i++) {
+ if (strcmp(class_name, sxe2_class_types[i].name) == 0)
+ class_type = sxe2_class_types[i].class_type;
+ }
+
+ return class_type;
+}
+
+static struct sxe2_common_device *sxe2_rtedev_to_cdev(struct rte_device *rte_dev)
+{
+ struct sxe2_common_device *cdev = NULL;
+
+ TAILQ_FOREACH(cdev, &sxe2_common_devices_list, next) {
+ if (rte_dev == cdev->dev)
+ goto l_end;
+ }
+
+ cdev = NULL;
+l_end:
+ return cdev;
+}
+
+static struct sxe2_class_driver *sxe2_class_driver_get(u32 class_type)
+{
+ struct sxe2_class_driver *cdrv = NULL;
+
+ TAILQ_FOREACH(cdrv, &sxe2_class_drivers_list, next) {
+ if (cdrv->drv_class == class_type)
+ goto l_end;
+ }
+
+ cdrv = NULL;
+l_end:
+ return cdrv;
+}
+
+static s32 sxe2_kvargs_preprocessing(struct sxe2_dev_kvargs_info *kv_info,
+ const struct rte_devargs *devargs)
+{
+ const struct rte_kvargs_pair *pair;
+ struct rte_kvargs *kvlist;
+ s32 ret = SXE2_ERROR;
+ u32 i;
+
+ kvlist = rte_kvargs_parse(devargs->args, NULL);
+ if (kvlist == NULL) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ for (i = 0; i < kvlist->count; i++) {
+ pair = &kvlist->pairs[i];
+ if (pair->value == NULL || *(pair->value) == '\0') {
+ PMD_LOG_ERR(COM, "Key %s has no value.", pair->key);
+ rte_kvargs_free(kvlist);
+ ret = -EINVAL;
+ goto l_end;
+ }
+ }
+
+ kv_info->kvlist = kvlist;
+ ret = SXE2_SUCCESS;
+ PMD_LOG_DEBUG(COM, "kvargs %d preprocessing success.",
+ kv_info->kvlist->count);
+l_end:
+ return ret;
+}
+
+static void sxe2_kvargs_free(struct sxe2_dev_kvargs_info *kv_info)
+{
+ if ((kv_info != NULL) && (kv_info->kvlist != NULL)) {
+ rte_kvargs_free(kv_info->kvlist);
+ kv_info->kvlist = NULL;
+ }
+}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_kvargs_process)
+s32
+sxe2_kvargs_process(struct sxe2_dev_kvargs_info *kv_info,
+ const char *const key_match, arg_handler_t handler, void *opaque_arg)
+{
+ const struct rte_kvargs_pair *pair;
+ struct rte_kvargs *kvlist;
+ u32 i;
+ s32 ret = SXE2_SUCCESS;
+
+ if ((kv_info == NULL) || (kv_info->kvlist == NULL) ||
+ (key_match == NULL)) {
+ PMD_LOG_ERR(COM, "Failed to process kvargs, NULL parameter.");
+ ret = -EINVAL;
+ goto l_end;
+ }
+ kvlist = kv_info->kvlist;
+
+ for (i = 0; i < kvlist->count; i++) {
+ pair = &kvlist->pairs[i];
+ if (strcmp(pair->key, key_match) == 0) {
+ ret = (*handler)(pair->key, pair->value, opaque_arg);
+ if (ret)
+ goto l_end;
+
+ kv_info->is_used[i] = true;
+ break;
+ }
+ }
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_parse_class_type(const char *key, const char *value, void *args)
+{
+ u32 *class_type = (u32 *)args;
+ s32 ret = SXE2_SUCCESS;
+
+ *class_type = sxe2_class_name_to_value(value);
+ if (*class_type == SXE2_CLASS_TYPE_INVALID) {
+ ret = -EINVAL;
+ PMD_LOG_ERR(COM, "Unsupported %s type: %s", key, value);
+ }
+
+ return ret;
+}
+
+static s32 sxe2_common_device_setup(struct sxe2_common_device *cdev)
+{
+ struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(cdev->dev);
+ s32 ret = SXE2_SUCCESS;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ goto l_end;
+
+ ret = sxe2_drv_dev_open(cdev, pci_dev);
+ if (ret != SXE2_SUCCESS) {
+ PMD_LOG_ERR(COM, "Open pmd chrdev failed, ret=%d", ret);
+ goto l_end;
+ }
+
+ ret = sxe2_drv_dev_handshark(cdev);
+ if (ret != SXE2_SUCCESS) {
+ PMD_LOG_ERR(COM, "Handshark failed, ret=%d", ret);
+ goto l_close_dev;
+ }
+
+ goto l_end;
+
+l_close_dev:
+ sxe2_drv_dev_close(cdev);
+l_end:
+ return ret;
+}
+
+static void sxe2_common_device_cleanup(struct sxe2_common_device *cdev)
+{
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ return;
+
+ if (TAILQ_EMPTY(&sxe2_common_devices_list))
+ (void)rte_mem_event_callback_unregister("SXE2_MEM_ENVENT_CB", NULL);
+
+ sxe2_drv_dev_close(cdev);
+}
+
+static struct sxe2_common_device *sxe2_common_device_alloc(
+ struct rte_device *rte_dev, u32 class_type)
+{
+ struct sxe2_common_device *cdev = NULL;
+
+ cdev = rte_zmalloc("sxe2_common_device", sizeof(*cdev), 0);
+ if (cdev == NULL) {
+ PMD_LOG_ERR(COM, "Fail to alloc sxe2 common device.");
+ goto l_end;
+ }
+ cdev->dev = rte_dev;
+ cdev->class_type = class_type;
+ cdev->config.kernel_reset = false;
+ rte_ticketlock_init(&cdev->config.lock);
+
+ (void)pthread_mutex_lock(&sxe2_common_devices_list_lock);
+ TAILQ_INSERT_TAIL(&sxe2_common_devices_list, cdev, next);
+ (void)pthread_mutex_unlock(&sxe2_common_devices_list_lock);
+
+l_end:
+ return cdev;
+}
+
+static void sxe2_common_device_free(struct sxe2_common_device *cdev)
+{
+
+ (void)pthread_mutex_lock(&sxe2_common_devices_list_lock);
+ TAILQ_REMOVE(&sxe2_common_devices_list, cdev, next);
+ (void)pthread_mutex_unlock(&sxe2_common_devices_list_lock);
+
+ rte_free(cdev);
+}
+
+static bool sxe2_dev_is_pci(const struct rte_device *dev)
+{
+ return strcmp(dev->bus->name, "pci") == 0;
+}
+
+static bool sxe2_dev_pci_id_match(const struct sxe2_class_driver *cdrv,
+ const struct rte_device *dev)
+{
+ const struct rte_pci_device *pci_dev;
+ const struct rte_pci_id *id_table;
+ bool ret = false;
+
+ if (!sxe2_dev_is_pci(dev)) {
+ PMD_LOG_ERR(COM, "Device %s is not a PCI device", dev->name);
+ goto l_end;
+ }
+
+ pci_dev = RTE_DEV_TO_PCI_CONST(dev);
+ for (id_table = cdrv->id_table; id_table->vendor_id != 0;
+ id_table++) {
+
+ if (id_table->vendor_id != pci_dev->id.vendor_id &&
+ id_table->vendor_id != RTE_PCI_ANY_ID) {
+ continue;
+ }
+ if (id_table->device_id != pci_dev->id.device_id &&
+ id_table->device_id != RTE_PCI_ANY_ID) {
+ continue;
+ }
+ if (id_table->subsystem_vendor_id !=
+ pci_dev->id.subsystem_vendor_id &&
+ id_table->subsystem_vendor_id != RTE_PCI_ANY_ID) {
+ continue;
+ }
+ if (id_table->subsystem_device_id !=
+ pci_dev->id.subsystem_device_id &&
+ id_table->subsystem_device_id != RTE_PCI_ANY_ID) {
+
+ continue;
+ }
+ if (id_table->class_id != pci_dev->id.class_id &&
+ id_table->class_id != RTE_CLASS_ANY_ID) {
+ continue;
+ }
+ ret = true;
+ break;
+ }
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_classes_driver_probe(struct sxe2_common_device *cdev,
+ struct sxe2_dev_kvargs_info *kv_info, u32 class_type)
+{
+ struct sxe2_class_driver *cdrv = NULL;
+ s32 ret = SXE2_ERROR;
+
+ cdrv = sxe2_class_driver_get(class_type);
+ if (cdrv == NULL) {
+ PMD_LOG_ERR(COM, "Fail to get class type[%u] driver.", class_type);
+ goto l_end;
+ }
+
+ if (!sxe2_dev_pci_id_match(cdrv, cdev->dev)) {
+ PMD_LOG_ERR(COM, "Fail to match pci id for driver:%s.", cdrv->name);
+ goto l_end;
+ }
+
+ ret = cdrv->probe(cdev, kv_info);
+ if (ret) {
+
+ PMD_LOG_DEBUG(COM, "Fail to probe driver:%s.", cdrv->name);
+ goto l_end;
+ }
+
+ cdev->cdrv = cdrv;
+l_end:
+ return ret;
+}
+
+static s32 sxe2_classes_driver_remove(struct sxe2_common_device *cdev)
+{
+ struct sxe2_class_driver *cdrv = cdev->cdrv;
+
+ return cdrv->remove(cdev);
+}
+
+static s32 sxe2_kvargs_validate(struct sxe2_dev_kvargs_info *kv_info)
+{
+ s32 ret = SXE2_SUCCESS;
+ u32 i;
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ goto l_end;
+
+ if (kv_info == NULL)
+ goto l_end;
+
+ for (i = 0; i < kv_info->kvlist->count; i++) {
+ if (kv_info->is_used[i] == 0) {
+ PMD_LOG_ERR(COM, "Key \"%s\" is unsupported for the class driver.",
+ kv_info->kvlist->pairs[i].key);
+ ret = -EINVAL;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_common_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ struct rte_device *rte_dev = &pci_dev->device;
+ struct sxe2_common_device *cdev;
+ struct sxe2_dev_kvargs_info *kv_info_p = NULL;
+
+ u32 class_type = SXE2_CLASS_TYPE_ETH;
+ s32 ret = SXE2_ERROR;
+
+ PMD_LOG_INFO(COM, "Probe pci device: %s", pci_dev->name);
+
+ cdev = sxe2_rtedev_to_cdev(rte_dev);
+ if (cdev != NULL) {
+ PMD_LOG_ERR(COM, "Device %s already probed.", rte_dev->name);
+ ret = -EBUSY;
+ goto l_end;
+ }
+
+ if ((rte_dev->devargs != NULL) && (rte_dev->devargs->args != NULL)) {
+ kv_info_p = calloc(1, sizeof(struct sxe2_dev_kvargs_info));
+ if (!kv_info_p) {
+ PMD_LOG_ERR(COM, "Failed to allocate memory for kv_info");
+ goto l_end;
+ }
+
+ ret = sxe2_kvargs_preprocessing(kv_info_p, rte_dev->devargs);
+ if (ret < 0) {
+ PMD_LOG_ERR(COM, "Unsupported device args: %s",
+ rte_dev->devargs->args);
+ goto l_free_kvargs;
+ }
+
+ ret = sxe2_kvargs_process(kv_info_p, SXE2_DEVARGS_KEY_CLASS,
+ sxe2_parse_class_type, &class_type);
+ if (ret < 0) {
+ PMD_LOG_ERR(COM, "Unsupported sxe2 driver class: %s",
+ rte_dev->devargs->args);
+ goto l_free_args;
+ }
+
+ }
+
+ cdev = sxe2_common_device_alloc(rte_dev, class_type);
+ if (cdev == NULL) {
+ ret = -ENOMEM;
+ goto l_free_args;
+ }
+
+ ret = sxe2_common_device_setup(cdev);
+ if (ret != SXE2_SUCCESS)
+ goto l_err_setup;
+
+ ret = sxe2_classes_driver_probe(cdev, kv_info_p, class_type);
+ if (ret != SXE2_SUCCESS)
+ goto l_err_probe;
+
+ ret = sxe2_kvargs_validate(kv_info_p);
+ if (ret != SXE2_SUCCESS) {
+ PMD_LOG_ERR(COM, "Device args validate failed: %s",
+ rte_dev->devargs->args);
+ goto l_err_valid;
+ }
+ cdev->kvargs = kv_info_p;
+
+ goto l_end;
+l_err_valid:
+ (void)sxe2_classes_driver_remove(cdev);
+l_err_probe:
+ sxe2_common_device_cleanup(cdev);
+l_err_setup:
+ sxe2_common_device_free(cdev);
+l_free_args:
+ sxe2_kvargs_free(kv_info_p);
+l_free_kvargs:
+ free(kv_info_p);
+l_end:
+ return ret;
+}
+
+static s32 sxe2_common_pci_remove(struct rte_pci_device *pci_dev)
+{
+ struct sxe2_common_device *cdev;
+ s32 ret = SXE2_ERROR;
+
+ PMD_LOG_INFO(COM, "Remove pci device: %s", pci_dev->name);
+ cdev = sxe2_rtedev_to_cdev(&pci_dev->device);
+ if (cdev == NULL) {
+ ret = -ENODEV;
+ PMD_LOG_ERR(COM, "Fail to get remove device.");
+ goto l_end;
+ }
+
+ ret = sxe2_classes_driver_remove(cdev);
+ if (ret != SXE2_SUCCESS) {
+ PMD_LOG_ERR(COM, "Fail to remove device: %s", pci_dev->name);
+ goto l_end;
+ }
+
+ sxe2_common_device_cleanup(cdev);
+
+ if (cdev->kvargs != NULL) {
+ sxe2_kvargs_free(cdev->kvargs);
+ free(cdev->kvargs);
+ cdev->kvargs = NULL;
+ }
+
+ sxe2_common_device_free(cdev);
+
+l_end:
+ return ret;
+}
+
+static struct rte_pci_driver sxe2_common_pci_driver = {
+ .driver = {
+ .name = SXE2_COMMON_PCI_DRIVER_NAME,
+ },
+ .probe = sxe2_common_pci_probe,
+ .remove = sxe2_common_pci_remove,
+};
+
+static u32 sxe2_common_pci_id_table_size_get(const struct rte_pci_id *id_table)
+{
+ u32 table_size = 0;
+
+ while (id_table->vendor_id != 0) {
+ table_size++;
+ id_table++;
+ }
+
+ return table_size;
+}
+
+static bool sxe2_common_pci_id_exists(const struct rte_pci_id *id,
+ const struct rte_pci_id *id_table, u32 next_idx)
+{
+ s32 current_size = next_idx - 1;
+ s32 i;
+ bool exists = false;
+
+ for (i = 0; i < current_size; i++) {
+ if ((id->device_id == id_table[i].device_id) &&
+ (id->vendor_id == id_table[i].vendor_id) &&
+ (id->subsystem_vendor_id == id_table[i].subsystem_vendor_id) &&
+ (id->subsystem_device_id == id_table[i].subsystem_device_id)) {
+ exists = true;
+ break;
+ }
+ }
+
+ return exists;
+}
+
+static void sxe2_common_pci_id_insert(struct rte_pci_id *id_table,
+ u32 *next_idx, const struct rte_pci_id *insert_table)
+{
+ for (; insert_table->vendor_id != 0; insert_table++) {
+ if (!sxe2_common_pci_id_exists(insert_table, id_table, *next_idx)) {
+
+ id_table[*next_idx] = *insert_table;
+ (*next_idx)++;
+ }
+ }
+}
+
+static s32 sxe2_common_pci_id_table_update(const struct rte_pci_id *id_table)
+{
+ const struct rte_pci_id *id_iter;
+ struct rte_pci_id *updated_table;
+ struct rte_pci_id *old_table;
+ u32 num_ids = 0;
+ u32 i = 0;
+ s32 ret = SXE2_SUCCESS;
+
+ old_table = sxe2_common_pci_id_table;
+ if (old_table)
+ num_ids = sxe2_common_pci_id_table_size_get(old_table);
+
+ num_ids += sxe2_common_pci_id_table_size_get(id_table);
+
+ num_ids += 1;
+
+ updated_table = calloc(num_ids, sizeof(*updated_table));
+ if (!updated_table) {
+ PMD_LOG_ERR(COM, "Failed to allocate memory for PCI ID table");
+ goto l_end;
+ }
+
+ if (old_table == NULL) {
+
+ for (id_iter = id_table; id_iter->vendor_id != 0;
+ id_iter++, i++)
+ updated_table[i] = *id_iter;
+ } else {
+
+ for (id_iter = old_table; id_iter->vendor_id != 0;
+ id_iter++, i++)
+ updated_table[i] = *id_iter;
+
+ sxe2_common_pci_id_insert(updated_table, &i, id_table);
+ }
+
+ updated_table[i].vendor_id = 0;
+ sxe2_common_pci_driver.id_table = updated_table;
+ sxe2_common_pci_id_table = updated_table;
+ free(old_table);
+
+l_end:
+ return ret;
+}
+
+static void sxe2_common_driver_on_register_pci(struct sxe2_class_driver *driver)
+{
+ if (driver->id_table != NULL) {
+ if (sxe2_common_pci_id_table_update(driver->id_table) != 0)
+ return;
+ }
+
+ if (driver->intr_lsc)
+ sxe2_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_LSC;
+ if (driver->intr_rmv)
+ sxe2_common_pci_driver.drv_flags |= RTE_PCI_DRV_INTR_RMV;
+}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_class_driver_register)
+void
+sxe2_class_driver_register(struct sxe2_class_driver *driver)
+{
+ sxe2_common_driver_on_register_pci(driver);
+ TAILQ_INSERT_TAIL(&sxe2_class_drivers_list, driver, next);
+}
+
+static void sxe2_common_pci_init(void)
+{
+ const struct rte_pci_id empty_table[] = {
+ {
+ .vendor_id = 0
+ },
+ };
+ s32 ret = SXE2_ERROR;
+
+ if (sxe2_common_pci_id_table == NULL) {
+ ret = sxe2_common_pci_id_table_update(empty_table);
+ if (ret != SXE2_SUCCESS)
+ goto l_end;
+ }
+ rte_pci_register(&sxe2_common_pci_driver);
+
+l_end:
+ return;
+}
+
+static bool sxe2_commoin_inited;
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_common_init)
+void
+sxe2_common_init(void)
+{
+ if (sxe2_commoin_inited)
+ goto l_end;
+
+ pthread_mutex_init(&sxe2_common_devices_list_lock, NULL);
+ sxe2_common_pci_init();
+ sxe2_commoin_inited = true;
+
+l_end:
+ return;
+}
+
+RTE_FINI(sxe2_common_pci_finish)
+{
+ if (sxe2_common_pci_id_table != NULL) {
+ rte_pci_unregister(&sxe2_common_pci_driver);
+ free(sxe2_common_pci_id_table);
+ }
+}
+
+RTE_PMD_EXPORT_NAME(sxe2_common_pci);
+
+RTE_LOG_REGISTER_SUFFIX(sxe2_common_log, com, NOTICE);
diff --git a/drivers/common/sxe2/sxe2_common.h b/drivers/common/sxe2/sxe2_common.h
new file mode 100644
index 0000000000..d02d281a70
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_common.h
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_COMMON_H__
+#define __SXE2_COMMON_H__
+
+#include <rte_bitops.h>
+#include <rte_kvargs.h>
+#include <rte_compat.h>
+#include <rte_memory.h>
+#include <rte_ticketlock.h>
+
+#include "sxe2_type.h"
+
+#define SXE2_COMMON_PCI_DRIVER_NAME "sxe2_pci"
+
+#define SXE2_CDEV_TO_CMD_FD(cdev) \
+ ((cdev)->config.cmd_fd)
+
+#define SXE2_DEVARGS_KEY_CLASS "class"
+
+struct sxe2_class_driver;
+
+enum sxe2_class_type {
+ SXE2_CLASS_TYPE_ETH = 0,
+ SXE2_CLASS_TYPE_VDPA,
+ SXE2_CLASS_TYPE_INVALID,
+};
+
+struct sxe2_common_dev_config {
+ s32 cmd_fd;
+ bool support_iommu;
+ bool kernel_reset;
+ rte_ticketlock_t lock;
+};
+
+struct sxe2_common_device {
+ struct rte_device *dev;
+ TAILQ_ENTRY(sxe2_common_device) next;
+ struct sxe2_class_driver *cdrv;
+ enum sxe2_class_type class_type;
+ struct sxe2_common_dev_config config;
+ struct sxe2_dev_kvargs_info *kvargs;
+};
+
+struct sxe2_dev_kvargs_info {
+ struct rte_kvargs *kvlist;
+ bool is_used[RTE_KVARGS_MAX];
+};
+
+typedef s32 (sxe2_class_driver_probe_t)(struct sxe2_common_device *scdev,
+ struct sxe2_dev_kvargs_info *kvargs);
+
+typedef s32 (sxe2_class_driver_remove_t)(struct sxe2_common_device *scdev);
+
+struct sxe2_class_driver {
+ TAILQ_ENTRY(sxe2_class_driver) next;
+ enum sxe2_class_type drv_class;
+ const s8 *name;
+ sxe2_class_driver_probe_t *probe;
+ sxe2_class_driver_remove_t *remove;
+ const struct rte_pci_id *id_table;
+ u32 intr_lsc;
+ u32 intr_rmv;
+};
+
+__rte_internal
+void
+sxe2_common_mem_event_cb(enum rte_mem_event type,
+ const void *addr, size_t size, void *arg __rte_unused);
+
+__rte_internal
+void
+sxe2_class_driver_register(struct sxe2_class_driver *driver);
+
+__rte_internal
+void
+sxe2_common_init(void);
+
+__rte_internal
+s32
+sxe2_kvargs_process(struct sxe2_dev_kvargs_info *kv_info,
+ const char *const key_match, arg_handler_t handler, void *opaque_arg);
+
+#endif
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
new file mode 100644
index 0000000000..0d300e0f81
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -0,0 +1,161 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#include <inttypes.h>
+#include <rte_version.h>
+#include <eal_export.h>
+
+#include "sxe2_osal.h"
+#include "sxe2_errno.h"
+#include "sxe2_common_log.h"
+#include "sxe2_ioctl_chnl.h"
+#include "sxe2_ioctl_chnl_func.h"
+
+#define SXE2_CHR_DEV_NAME "/dev/sxe2-dpdk-"
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_cmd_close)
+void
+sxe2_drv_cmd_close(struct sxe2_common_device *cdev)
+{
+ cdev->config.kernel_reset = true;
+}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_cmd_exec)
+s32
+sxe2_drv_cmd_exec(struct sxe2_common_device *cdev,
+ struct sxe2_drv_cmd_params *cmd_params)
+{
+ s32 cmd_fd;
+ s32 ret = -EIO;
+
+ if (cdev->config.kernel_reset) {
+ ret = -EPERM;
+ PMD_LOG_WARN(COM, "kernel reset, need restart app.");
+ goto l_end;
+ }
+
+ cmd_fd = SXE2_CDEV_TO_CMD_FD(cdev);
+ if (cmd_fd < 0) {
+ ret = -EBADF;
+ PMD_LOG_ERR(COM, "Fail to exec cmd, fd[%d] error", cmd_fd);
+ goto l_end;
+ }
+
+ PMD_LOG_DEBUG(COM, "Exec drv cmd fd[%d] trace_id[0x%"PRIx64"]"
+ "opcode[0x%x] req_len[%d] resp_len[%d]",
+ cmd_fd, cmd_params->trace_id, cmd_params->opcode,
+ cmd_params->req_len, cmd_params->resp_len);
+
+ rte_ticketlock_lock(&cdev->config.lock);
+ ret = ioctl(cmd_fd, SXE2_COM_CMD_PASSTHROUGH, cmd_params);
+ if (ret < 0) {
+ PMD_LOG_ERR(COM, "Fail to exec cmd, fd[%d] opcode[0x%x] ret[%d], err:%s",
+ cmd_fd, cmd_params->opcode, ret, strerror(errno));
+ ret = -errno;
+ rte_ticketlock_unlock(&cdev->config.lock);
+ goto l_end;
+ }
+ rte_ticketlock_unlock(&cdev->config.lock);
+
+l_end:
+ return ret;
+}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_open)
+s32
+sxe2_drv_dev_open(struct sxe2_common_device *cdev, struct rte_pci_device *pci_dev)
+{
+ s32 ret = SXE2_SUCCESS;
+ s32 fd = 0;
+ char drv_name[32] = {0};
+
+ snprintf(drv_name, sizeof(drv_name),
+ "%s%04"PRIx32":%02"PRIx8":%02"PRIx8".%"PRIx8,
+ SXE2_CHR_DEV_NAME,
+ pci_dev->addr.domain,
+ pci_dev->addr.bus,
+ pci_dev->addr.devid,
+ pci_dev->addr.function);
+
+ fd = open(drv_name, O_RDWR);
+ if (fd < 0) {
+ ret = -EBADF;
+ PMD_LOG_ERR(COM, "Fail to open device:%s, ret=%d, err:%s",
+ drv_name, ret, strerror(errno));
+ goto l_end;
+ }
+
+ SXE2_CDEV_TO_CMD_FD(cdev) = fd;
+
+ PMD_LOG_INFO(COM, "Successfully opened device:%s, fd=%d",
+ drv_name, SXE2_CDEV_TO_CMD_FD(cdev));
+
+l_end:
+ return ret;
+}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_close)
+void
+sxe2_drv_dev_close(struct sxe2_common_device *cdev)
+{
+ s32 fd = SXE2_CDEV_TO_CMD_FD(cdev);
+
+ if (fd >= 0)
+ close(fd);
+ PMD_LOG_INFO(COM, "closed device fd=%d", fd);
+ SXE2_CDEV_TO_CMD_FD(cdev) = -1;
+}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_handshark)
+s32
+sxe2_drv_dev_handshark(struct sxe2_common_device *cdev)
+{
+ s32 ret = SXE2_SUCCESS;
+ s32 cmd_fd = 0;
+ struct sxe2_ioctl_cmd_common_hdr cmd_params;
+
+ if (cdev->config.kernel_reset) {
+ ret = -EPERM;
+ PMD_LOG_WARN(COM, "kernel reset, need restart app.");
+ goto l_end;
+ }
+
+ cmd_fd = SXE2_CDEV_TO_CMD_FD(cdev);
+ if (cmd_fd < 0) {
+ ret = -EBADF;
+ PMD_LOG_ERR(COM, "Failed to exec cmd, fd=%d", cmd_fd);
+ goto l_end;
+ }
+
+ PMD_LOG_DEBUG(COM, "Open fd=%d to handshark with kernel", cmd_fd);
+
+ memset(&cmd_params, 0, sizeof(struct sxe2_ioctl_cmd_common_hdr));
+ cmd_params.dpdk_ver = SXE2_COM_VER;
+ cmd_params.msg_len = sizeof(struct sxe2_ioctl_cmd_common_hdr);
+
+ rte_ticketlock_lock(&cdev->config.lock);
+ ret = ioctl(cmd_fd, SXE2_COM_CMD_HANDSHAKE, &cmd_params);
+ if (ret < 0) {
+ PMD_LOG_ERR(COM, "Failed to handshark, fd=%d, ret=%d, err:%s",
+ cmd_fd, ret, strerror(errno));
+ ret = -EIO;
+ rte_ticketlock_unlock(&cdev->config.lock);
+ goto l_end;
+ }
+ rte_ticketlock_unlock(&cdev->config.lock);
+
+ if (cmd_params.cap & BIT(SXE2_COM_CAP_IOMMU_MAP))
+ cdev->config.support_iommu = true;
+ else
+ cdev->config.support_iommu = false;
+
+l_end:
+ return ret;
+}
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.h b/drivers/common/sxe2/sxe2_ioctl_chnl.h
new file mode 100644
index 0000000000..eedb3d6693
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.h
@@ -0,0 +1,141 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_IOCTL_CHNL_H__
+#define __SXE2_IOCTL_CHNL_H__
+
+#ifdef SXE2_DPDK_DRIVER
+
+#include <rte_version.h>
+#include <bus_pci_driver.h>
+#include "sxe2_type.h"
+#endif
+
+#ifdef SXE2_LINUX_DRIVER
+#ifdef __KERNEL__
+#include <linux/types.h>
+#include <linux/ioctl.h>
+#endif
+#endif
+
+#include "sxe2_internal_ver.h"
+
+#define SXE2_COM_INVAL_U32 0xFFFFFFFF
+
+#define SXE2_COM_PCI_OFFSET_SHIFT 40
+
+#define SXE2_COM_PCI_INDEX_TO_OFFSET(index) ((u64)(index) << SXE2_COM_PCI_OFFSET_SHIFT)
+#define SXE2_COM_PCI_OFFSET_MASK (((u64)(1) << SXE2_COM_PCI_OFFSET_SHIFT) - 1)
+#define SXE2_COM_PCI_OFFSET_GEN(index, off) ((((u64)(index)) << SXE2_COM_PCI_OFFSET_SHIFT) | \
+ (((u64)(off)) & SXE2_COM_PCI_OFFSET_MASK))
+
+#define SXE2_DRV_TRACE_ID_COUNT_MASK 0x003FFFFFFFFFFFFFLLU
+
+#define SXE2_DRV_CMD_DFLT_TIMEOUT (30)
+
+#define SXE2_COM_VER_MAJOR 1
+#define SXE2_COM_VER_MINOR 0
+#define SXE2_COM_VER SXE2_MK_VER(SXE2_COM_VER_MAJOR, SXE2_COM_VER_MINOR)
+
+enum SXE2_COM_CMD {
+ SXE2_DEVICE_HANDSHAKE = 1,
+ SXE2_DEVICE_IO_IRQS_REQ,
+ SXE2_DEVICE_EVT_IRQ_REQ,
+ SXE2_DEVICE_RST_IRQ_REQ,
+ SXE2_DEVICE_EVT_CAUSE_GET,
+ SXE2_DEVICE_DMA_MAP,
+ SXE2_DEVICE_DMA_UNMAP,
+ SXE2_DEVICE_PASSTHROUGH,
+ SXE2_DEVICE_MAX,
+};
+
+#define SXE2_CMD_TYPE 'S'
+
+#define SXE2_COM_CMD_HANDSHAKE _IO(SXE2_CMD_TYPE, SXE2_DEVICE_HANDSHAKE)
+#define SXE2_COM_CMD_IO_IRQS_REQ _IO(SXE2_CMD_TYPE, SXE2_DEVICE_IO_IRQS_REQ)
+#define SXE2_COM_CMD_EVT_IRQ_REQ _IO(SXE2_CMD_TYPE, SXE2_DEVICE_EVT_IRQ_REQ)
+#define SXE2_COM_CMD_RST_IRQ_REQ _IO(SXE2_CMD_TYPE, SXE2_DEVICE_RST_IRQ_REQ)
+#define SXE2_COM_CMD_EVT_CAUSE_GET _IO(SXE2_CMD_TYPE, SXE2_DEVICE_EVT_CAUSE_GET)
+#define SXE2_COM_CMD_DMA_MAP _IO(SXE2_CMD_TYPE, SXE2_DEVICE_DMA_MAP)
+#define SXE2_COM_CMD_DMA_UNMAP _IO(SXE2_CMD_TYPE, SXE2_DEVICE_DMA_UNMAP)
+#define SXE2_COM_CMD_PASSTHROUGH _IO(SXE2_CMD_TYPE, SXE2_DEVICE_PASSTHROUGH)
+
+enum sxe2_com_cap {
+ SXE2_COM_CAP_IOMMU_MAP = 0,
+};
+
+struct sxe2_ioctl_cmd_common_hdr {
+ u32 dpdk_ver;
+ u32 drv_ver;
+ u32 msg_len;
+ u32 cap;
+ u8 reserved[32];
+};
+
+struct sxe2_drv_cmd_params {
+ u64 trace_id;
+ u32 timeout;
+ u32 opcode;
+ u16 vsi_id;
+ u16 repr_id;
+ u32 req_len;
+ u32 resp_len;
+ void *req_data;
+ void *resp_data;
+ u8 resv[32];
+};
+
+struct sxe2_ioctl_irq_set {
+ u32 cnt;
+ u8 resv[4];
+ u32 base_irq_in_com;
+ s32 *event_fd;
+};
+
+enum sxe2_com_event_cause {
+ SXE2_COM_EC_LINK_CHG = 0,
+ SXE2_COM_SW_MODE_LEGACY,
+ SXE2_COM_SW_MODE_SWITCHDEV,
+ SXE2_COM_FC_ST_CHANGE,
+
+ SXE2_COM_EC_RESET = 62,
+ SXE2_COM_EC_MAX = 63,
+};
+
+struct sxe2_ioctl_other_evt_set {
+ s32 eventfd;
+ u8 resv[4];
+ u64 filter_table;
+};
+
+struct sxe2_ioctl_other_evt_get {
+ u64 evt_cause;
+ u8 resv[8];
+};
+
+struct sxe2_ioctl_reset_sub_set {
+ s32 eventfd;
+ u8 resv[4];
+};
+
+struct sxe2_ioctl_iommu_dma_map {
+ u64 vaddr;
+ u64 iova;
+ u64 size;
+ u8 resv[4];
+};
+
+struct sxe2_ioctl_iommu_dma_unmap {
+ u64 iova;
+};
+
+union sxe2_drv_trace_info {
+ u64 id;
+ struct {
+ u64 count : 54;
+ u64 cpu_id : 10;
+ } sxe2_drv_trace_id_param;
+};
+
+#endif
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl_func.h b/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
new file mode 100644
index 0000000000..0c3cb9caea
--- /dev/null
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_IOCTL_CHNL_FUNC_H__
+#define __SXE2_IOCTL_CHNL_FUNC_H__
+
+#include <rte_version.h>
+#include <bus_pci_driver.h>
+
+#include "sxe2_type.h"
+#include "sxe2_common.h"
+#include "sxe2_ioctl_chnl.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+__rte_internal
+void
+sxe2_drv_cmd_close(struct sxe2_common_device *cdev);
+
+__rte_internal
+s32
+sxe2_drv_cmd_exec(struct sxe2_common_device *cdev,
+ struct sxe2_drv_cmd_params *cmd_params);
+
+__rte_internal
+s32
+sxe2_drv_dev_open(struct sxe2_common_device *cdev,
+ struct rte_pci_device *pci_dev);
+
+__rte_internal
+void
+sxe2_drv_dev_close(struct sxe2_common_device *cdev);
+
+__rte_internal
+s32
+sxe2_drv_dev_handshark(struct sxe2_common_device *cdev);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/drivers/meson.build b/drivers/meson.build
index 6ae102e943..d4ae512bae 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -12,6 +12,7 @@ subdirs = [
'common/qat', # depends on bus.
'common/sfc_efx', # depends on bus.
'common/zsda', # depends on bus.
+ 'common/sxe2', # depends on bus.
'mempool', # depends on common and bus.
'dma', # depends on common and bus.
'net', # depends on common, bus, mempool
--
2.47.3
^ permalink raw reply related
* [PATCH v13 07/10] common/sxe2: add ioctl interface for DMA map and unmap
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512113700.837744-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Implement DMA mapping and unmapping functionality using ioctl
calls. This allows the driver to configure the hardware's IOMMU/DMA
tables, ensuring the device can safely access memory buffers
allocated by the userspace.
The mapping is established during device initialization or queue
setup and is revoked during device closure to prevent memory
leaks and ensure hardware security.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_common.c | 50 +++++++++-
drivers/common/sxe2/sxe2_ioctl_chnl.c | 106 ++++++++++++++++++++-
drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 9 ++
3 files changed, 163 insertions(+), 2 deletions(-)
diff --git a/drivers/common/sxe2/sxe2_common.c b/drivers/common/sxe2/sxe2_common.c
index 62bdc93b5c..63873afe4a 100644
--- a/drivers/common/sxe2/sxe2_common.c
+++ b/drivers/common/sxe2/sxe2_common.c
@@ -443,7 +443,7 @@ static s32 sxe2_common_pci_remove(struct rte_pci_device *pci_dev)
cdev = sxe2_rtedev_to_cdev(&pci_dev->device);
if (cdev == NULL) {
ret = -ENODEV;
- PMD_LOG_ERR(COM, "Fail to get remove device.");
+ PMD_LOG_ERR(COM, "Fail to get device when remove.");
goto l_end;
}
@@ -467,12 +467,60 @@ static s32 sxe2_common_pci_remove(struct rte_pci_device *pci_dev)
return ret;
}
+static s32 sxe2_common_pci_dma_map(struct rte_pci_device *pci_dev,
+ void *addr, u64 iova, size_t len)
+{
+ struct sxe2_common_device *cdev;
+ s32 ret = SXE2_ERROR;
+
+ cdev = sxe2_rtedev_to_cdev(&pci_dev->device);
+ if (cdev == NULL) {
+ ret = -ENODEV;
+ PMD_LOG_ERR(COM, "Fail to get device when dma map.");
+ goto l_end;
+ }
+
+ ret = sxe2_drv_dev_dma_map(cdev, (u64)(uintptr_t)addr, iova, len);
+ if (ret) {
+ PMD_LOG_ERR(COM, "Fail to map dma map, ret=%d", ret);
+ goto l_end;
+ }
+
+l_end:
+ return ret;
+}
+
+static s32 sxe2_common_pci_dma_unmap(struct rte_pci_device *pci_dev,
+ void *addr __rte_unused, u64 iova, size_t len __rte_unused)
+{
+ struct sxe2_common_device *cdev;
+ s32 ret = SXE2_ERROR;
+
+ cdev = sxe2_rtedev_to_cdev(&pci_dev->device);
+ if (cdev == NULL) {
+ ret = -ENODEV;
+ PMD_LOG_ERR(COM, "Fail to get device when dma unmap.");
+ goto l_end;
+ }
+
+ ret = sxe2_drv_dev_dma_unmap(cdev, iova);
+ if (ret) {
+ PMD_LOG_ERR(COM, "Fail to unmap dma map, ret=%d", ret);
+ goto l_end;
+ }
+
+l_end:
+ return ret;
+}
+
static struct rte_pci_driver sxe2_common_pci_driver = {
.driver = {
.name = SXE2_COMMON_PCI_DRIVER_NAME,
},
.probe = sxe2_common_pci_probe,
.remove = sxe2_common_pci_remove,
+ .dma_map = sxe2_common_pci_dma_map,
+ .dma_unmap = sxe2_common_pci_dma_unmap,
};
static u32 sxe2_common_pci_id_table_size_get(const struct rte_pci_id *id_table)
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
index 80fccc6a11..4dfc4fd0fa 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl.c
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -202,7 +202,7 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
if (cdev->config.kernel_reset) {
ret = -EPERM;
- PMD_LOG_WARN(COM, "kernel reseted, need restart app.");
+ PMD_LOG_WARN(COM, "kernel reset, need restart app.");
goto l_end;
}
@@ -220,3 +220,107 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
l_end:
return ret;
}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_dma_map)
+s32
+sxe2_drv_dev_dma_map(struct sxe2_common_device *cdev, u64 vaddr,
+ u64 iova, u64 size)
+{
+ struct sxe2_ioctl_iommu_dma_map cmd_params;
+ enum rte_iova_mode iova_mode;
+ s32 ret = SXE2_SUCCESS;
+ s32 cmd_fd = 0;
+
+ if (cdev->config.kernel_reset) {
+ ret = -EPERM;
+ PMD_LOG_WARN(COM, "kernel reset, need restart app.");
+ goto l_end;
+ }
+
+ iova_mode = rte_eal_iova_mode();
+ if (iova_mode == RTE_IOVA_PA) {
+ if (cdev->config.support_iommu) {
+ PMD_LOG_ERR(COM, "iommu not support pa mode");
+ ret = -EIO;
+ }
+ goto l_end;
+ } else if (iova_mode == RTE_IOVA_VA) {
+ if (!cdev->config.support_iommu) {
+ PMD_LOG_ERR(COM, "no iommu not support va mode, please use pa mode.");
+ ret = -EIO;
+ goto l_end;
+ }
+ }
+
+ cmd_fd = SXE2_CDEV_TO_CMD_FD(cdev);
+ if (cmd_fd < 0) {
+ ret = -EBADF;
+ PMD_LOG_ERR(COM, "Failed to exec cmd, fd=%d", cmd_fd);
+ goto l_end;
+ }
+
+ memset(&cmd_params, 0, sizeof(struct sxe2_ioctl_iommu_dma_map));
+ cmd_params.vaddr = vaddr;
+ cmd_params.iova = iova;
+ cmd_params.size = size;
+
+ rte_ticketlock_lock(&cdev->config.lock);
+ ret = ioctl(cmd_fd, SXE2_COM_CMD_DMA_MAP, &cmd_params);
+ if (ret < 0) {
+ PMD_LOG_ERR(COM, "Failed to dma map, fd=%d, ret=%d, err:%s",
+ cmd_fd, ret, strerror(errno));
+ ret = -EIO;
+ rte_ticketlock_unlock(&cdev->config.lock);
+ goto l_end;
+ }
+ rte_ticketlock_unlock(&cdev->config.lock);
+
+l_end:
+ return ret;
+}
+
+RTE_EXPORT_INTERNAL_SYMBOL(sxe2_drv_dev_dma_unmap)
+s32
+sxe2_drv_dev_dma_unmap(struct sxe2_common_device *cdev, u64 iova)
+{
+ s32 ret = SXE2_SUCCESS;
+ s32 cmd_fd = 0;
+ struct sxe2_ioctl_iommu_dma_unmap cmd_params;
+
+ if (cdev->config.kernel_reset) {
+ ret = -EPERM;
+ PMD_LOG_WARN(COM, "kernel reset, need restart app.");
+ goto l_end;
+ }
+
+ if (!cdev->config.support_iommu)
+ goto l_end;
+
+ cmd_fd = SXE2_CDEV_TO_CMD_FD(cdev);
+ if (cmd_fd < 0) {
+ ret = -EBADF;
+ PMD_LOG_ERR(COM, "Failed to exec cmd, fd=%d", cmd_fd);
+ goto l_end;
+ }
+
+ PMD_LOG_DEBUG(COM, "fd %d dma unmap iova=0x%"PRIX64"",
+ cmd_fd, iova);
+
+ memset(&cmd_params, 0, sizeof(struct sxe2_ioctl_iommu_dma_unmap));
+ cmd_params.iova = iova;
+
+ rte_ticketlock_lock(&cdev->config.lock);
+ ret = ioctl(cmd_fd, SXE2_COM_CMD_DMA_UNMAP, &cmd_params);
+ if (ret < 0) {
+ PMD_LOG_INFO(COM, "Failed to dma unmap, fd=%d, ret=%d, err:%s",
+ cmd_fd, ret, strerror(errno));
+ ret = -EIO;
+ rte_ticketlock_unlock(&cdev->config.lock);
+ goto l_end;
+ }
+ rte_ticketlock_unlock(&cdev->config.lock);
+
+l_end:
+ return ret;
+}
+
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl_func.h b/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
index 376c5e3ac7..e8f983e40e 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl_func.h
@@ -47,6 +47,15 @@ __rte_internal
s32
sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len);
+__rte_internal
+s32
+sxe2_drv_dev_dma_map(struct sxe2_common_device *cdev, u64 vaddr,
+ u64 iova, u64 size);
+
+__rte_internal
+s32
+sxe2_drv_dev_dma_unmap(struct sxe2_common_device *cdev, u64 iova);
+
#ifdef __cplusplus
}
#endif
--
2.47.3
^ permalink raw reply related
* [PATCH v13 00/10] net/sxe2: fix logic errors and address feedback
From: liujie5 @ 2026-05-12 11:36 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512080616.454171-11-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch set addresses the feedback received on the v10 submission
for the sxe2 PMD. The primary focus is on fixing vector path selection,
ensuring memory safety during mbuf initialization, and cleaning up
redundant logic in the configuration functions.
v13 Changes:
- Fixed vector Rx burst function being overwritten by scalar selection.
- Refactored Rx/Tx mode set functions to seed flags from caps first,eliminating tautological checks.
- Added memset for mbuf_def in vector init to avoid uninitialized reads.
- Converted pci_map_addr_info to designated initializers.
- Removed dead Windows-only code in meson.build.
- Added NULL checks for mbuf free for driver-wide consistency.
- Updated burst_mode_get to accurately report AVX paths.
- Adjusted SXE2_ETH_OVERHEAD to match actual VLAN capabilities.
Jie Liu (10):
mailmap: add Jie Liu
doc: add sxe2 guide and release notes
common/sxe2: add sxe2 basic structures
drivers: add base driver skeleton
drivers: add base driver probe skeleton
drivers: support PCI BAR mapping
common/sxe2: add ioctl interface for DMA map and unmap
net/sxe2: support queue setup and control
drivers: add data path for Rx and Tx
net/sxe2: add vectorized Rx and Tx
.mailmap | 1 +
doc/guides/nics/features/sxe2.ini | 30 +
doc/guides/nics/index.rst | 1 +
doc/guides/nics/sxe2.rst | 34 +
doc/guides/rel_notes/release_26_07.rst | 4 +
drivers/common/sxe2/meson.build | 15 +
drivers/common/sxe2/sxe2_common.c | 685 +++++++++++++++
drivers/common/sxe2/sxe2_common.h | 86 ++
drivers/common/sxe2/sxe2_common_log.h | 83 ++
drivers/common/sxe2/sxe2_errno.h | 110 +++
drivers/common/sxe2/sxe2_host_regs.h | 707 +++++++++++++++
drivers/common/sxe2/sxe2_internal_ver.h | 33 +
drivers/common/sxe2/sxe2_ioctl_chnl.c | 326 +++++++
drivers/common/sxe2/sxe2_ioctl_chnl.h | 141 +++
drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 63 ++
drivers/common/sxe2/sxe2_osal.h | 584 +++++++++++++
drivers/common/sxe2/sxe2_type.h | 60 ++
drivers/meson.build | 1 +
drivers/net/meson.build | 1 +
drivers/net/sxe2/meson.build | 32 +
drivers/net/sxe2/sxe2_cmd_chnl.c | 319 +++++++
drivers/net/sxe2/sxe2_cmd_chnl.h | 33 +
drivers/net/sxe2/sxe2_drv_cmd.h | 389 +++++++++
drivers/net/sxe2/sxe2_ethdev.c | 941 ++++++++++++++++++++
drivers/net/sxe2/sxe2_ethdev.h | 315 +++++++
drivers/net/sxe2/sxe2_irq.h | 49 ++
drivers/net/sxe2/sxe2_queue.c | 67 ++
drivers/net/sxe2/sxe2_queue.h | 194 +++++
drivers/net/sxe2/sxe2_rx.c | 579 +++++++++++++
drivers/net/sxe2/sxe2_rx.h | 34 +
drivers/net/sxe2/sxe2_tx.c | 447 ++++++++++
drivers/net/sxe2/sxe2_tx.h | 32 +
drivers/net/sxe2/sxe2_txrx.c | 372 ++++++++
drivers/net/sxe2/sxe2_txrx.h | 22 +
drivers/net/sxe2/sxe2_txrx_common.h | 541 ++++++++++++
drivers/net/sxe2/sxe2_txrx_poll.c | 945 +++++++++++++++++++++
drivers/net/sxe2/sxe2_txrx_poll.h | 17 +
drivers/net/sxe2/sxe2_txrx_vec.c | 197 +++++
drivers/net/sxe2/sxe2_txrx_vec.h | 72 ++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 235 +++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 545 ++++++++++++
drivers/net/sxe2/sxe2_vsi.c | 212 +++++
drivers/net/sxe2/sxe2_vsi.h | 205 +++++
43 files changed, 9759 insertions(+)
create mode 100644 doc/guides/nics/features/sxe2.ini
create mode 100644 doc/guides/nics/sxe2.rst
create mode 100644 drivers/common/sxe2/meson.build
create mode 100644 drivers/common/sxe2/sxe2_common.c
create mode 100644 drivers/common/sxe2/sxe2_common.h
create mode 100644 drivers/common/sxe2/sxe2_common_log.h
create mode 100644 drivers/common/sxe2/sxe2_errno.h
create mode 100644 drivers/common/sxe2/sxe2_host_regs.h
create mode 100644 drivers/common/sxe2/sxe2_internal_ver.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.c
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl_func.h
create mode 100644 drivers/common/sxe2/sxe2_osal.h
create mode 100644 drivers/common/sxe2/sxe2_type.h
create mode 100644 drivers/net/sxe2/meson.build
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.c
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.h
create mode 100644 drivers/net/sxe2/sxe2_drv_cmd.h
create mode 100644 drivers/net/sxe2/sxe2_ethdev.c
create mode 100644 drivers/net/sxe2/sxe2_ethdev.h
create mode 100644 drivers/net/sxe2/sxe2_irq.h
create mode 100644 drivers/net/sxe2/sxe2_queue.c
create mode 100644 drivers/net/sxe2/sxe2_queue.h
create mode 100644 drivers/net/sxe2/sxe2_rx.c
create mode 100644 drivers/net/sxe2/sxe2_rx.h
create mode 100644 drivers/net/sxe2/sxe2_tx.c
create mode 100644 drivers/net/sxe2/sxe2_tx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx.c
create mode 100644 drivers/net/sxe2/sxe2_txrx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_sse.c
create mode 100644 drivers/net/sxe2/sxe2_vsi.c
create mode 100644 drivers/net/sxe2/sxe2_vsi.h
--
2.47.3
^ permalink raw reply
* [PATCH v2 2/4] ethdev: introduce generic cache stash API
From: Chengwen Feng @ 2026-05-12 9:23 UTC (permalink / raw)
To: thomas, stephen; +Cc: dev, wathsala.vithanage, liuyonglong
In-Reply-To: <20260512092302.23735-1-fengchengwen@huawei.com>
Introduce a generic Ethernet device API for cache stashing,
supporting hardware mechanisms such as PCIe TPH (Transaction
Processing Hints).
The API provides:
- Generic query for supported stashing types and objects
- Device-wide stashing enable/disable
- Per-queue stashing with target lcore and data objects
(Rx/Tx descriptor, header, payload)
The design allows a device to support multiple stashing types
while only allowing one active type at runtime.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
lib/ethdev/ethdev_driver.h | 37 +++++++++++++++
lib/ethdev/rte_ethdev.c | 40 ++++++++++++++++
lib/ethdev/rte_ethdev.h | 95 ++++++++++++++++++++++++++++++++++++++
3 files changed, 172 insertions(+)
diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
index 1255cd6f2c..e7b4167939 100644
--- a/lib/ethdev/ethdev_driver.h
+++ b/lib/ethdev/ethdev_driver.h
@@ -1409,6 +1409,38 @@ enum rte_eth_dev_operation {
typedef uint64_t (*eth_get_restore_flags_t)(struct rte_eth_dev *dev,
enum rte_eth_dev_operation op);
+/**
+ * @internal
+ * Get cache stash capabilities of the Ethernet device.
+ *
+ * @param dev
+ * Port (ethdev) handle.
+ * @param capa
+ * Pointer to capability structure to store supported stash types and objects.
+ *
+ * @return
+ * Negative on error, 0 on success.
+ */
+typedef int (*eth_cache_stash_get_t)(struct rte_eth_dev *dev,
+ struct rte_eth_cache_stash_capability *capa);
+
+/**
+ * @internal
+ * Configure cache stash for the Ethernet device or queue.
+ *
+ * @param dev
+ * Port (ethdev) handle.
+ * @param op
+ * Cache stash operation type (enable/disable device or queue).
+ * @param config
+ * Cache stash configuration (device or queue level).
+ *
+ * @return
+ * Negative on error, 0 on success.
+ */
+typedef int (*eth_cache_stash_set_t)(struct rte_eth_dev *dev, enum rte_eth_cache_stash_op op,
+ struct rte_eth_cache_stash_config *config);
+
/**
* @internal A structure containing the functions exported by an Ethernet driver.
*/
@@ -1661,6 +1693,11 @@ struct eth_dev_ops {
/** Get configuration which ethdev should restore */
eth_get_restore_flags_t get_restore_flags;
+
+ /** Cache stash get ops */
+ eth_cache_stash_get_t cache_stash_get;
+ /** Cache stash set ops */
+ eth_cache_stash_set_t cache_stash_set;
};
/**
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index 2edc7a362e..4398aca5e1 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -7460,5 +7460,45 @@ int rte_eth_dev_map_aggr_tx_affinity(uint16_t port_id, uint16_t tx_queue_id,
return ret;
}
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_cache_stash_get, 26.03)
+int
+rte_eth_cache_stash_get(uint16_t port_id,
+ struct rte_eth_cache_stash_capability *capa)
+{
+ struct rte_eth_dev *dev;
+
+ RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+
+ if (capa == NULL)
+ return -EINVAL;
+
+ dev = &rte_eth_devices[port_id];
+
+ if (*dev->dev_ops->cache_stash_get == NULL)
+ return -ENOTSUP;
+
+ return eth_err(port_id, (*dev->dev_ops->cache_stash_get)(dev, capa));
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_cache_stash_set, 26.03)
+int
+rte_eth_cache_stash_set(uint16_t port_id, enum rte_eth_cache_stash_op op,
+ struct rte_eth_cache_stash_config *config)
+{
+ struct rte_eth_dev *dev;
+
+ RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
+
+ if (op != RTE_ETH_CACHE_STASH_OP_DEV_DISABLE && config == NULL)
+ return -EINVAL;
+
+ dev = &rte_eth_devices[port_id];
+
+ if (*dev->dev_ops->cache_stash_set == NULL)
+ return -ENOTSUP;
+
+ return eth_err(port_id, (*dev->dev_ops->cache_stash_set)(dev, op, config));
+}
+
RTE_EXPORT_SYMBOL(rte_eth_dev_logtype)
RTE_LOG_REGISTER_DEFAULT(rte_eth_dev_logtype, INFO);
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index 0d8e2d0236..014f0c21b6 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -7178,6 +7178,101 @@ rte_eth_tx_queue_count(uint16_t port_id, uint16_t queue_id)
return rc;
}
+/**
+ * Cache stash mechanisms (bitmask).
+ * A device may support multiple types but can only enable one at a time.
+ */
+enum rte_eth_cache_stash_type {
+ RTE_ETH_CACHE_STASH_TYPE_TPH = RTE_BIT32(0),
+};
+
+/**
+ * Cache stash objects (bitmask).
+ */
+#define RTE_ETH_CACHE_STASH_OBJ_RX_DESC RTE_BIT64(0)
+#define RTE_ETH_CACHE_STASH_OBJ_TX_DESC RTE_BIT64(1)
+#define RTE_ETH_CACHE_STASH_OBJ_RX_HEADER RTE_BIT64(2)
+#define RTE_ETH_CACHE_STASH_OBJ_TX_HEADER RTE_BIT64(3)
+#define RTE_ETH_CACHE_STASH_OBJ_RX_PAYLOAD RTE_BIT64(4)
+#define RTE_ETH_CACHE_STASH_OBJ_TX_PAYLOAD RTE_BIT64(5)
+
+/**
+ * Cache stash capability.
+ */
+struct rte_eth_cache_stash_capability {
+ uint32_t supported_types; /**< Bitmask of rte_eth_cache_stash_type */
+ uint64_t supported_objects;/**< Bitmask of RTE_ETH_CACHE_STASH_OBJ_* */
+};
+
+/**
+ * Cache stash operations.
+ */
+enum rte_eth_cache_stash_op {
+ RTE_ETH_CACHE_STASH_OP_DEV_ENABLE, /**< Enable device-wide cache stash */
+ RTE_ETH_CACHE_STASH_OP_DEV_DISABLE, /**< Disable device-wide cache stash */
+ RTE_ETH_CACHE_STASH_OP_QUEUE_ENABLE, /**< Enable cache stash for a queue */
+ RTE_ETH_CACHE_STASH_OP_QUEUE_DISABLE, /**< Disable cache stash for a queue */
+ RTE_ETH_CACHE_STASH_OP_BUTT
+};
+
+/**
+ * Cache stash configuration.
+ * The used union member is determined by the operation.
+ */
+struct rte_eth_cache_stash_config {
+ union {
+ /**
+ * Device-level configuration (used with DEV_ENABLE).
+ */
+ struct {
+ uint32_t type; /**< Selected stash mechanism type */
+ } dev;
+ /**
+ * Queue-level configuration (used with QUEUE_ENABLE/QUEUE_DISABLE).
+ */
+ struct {
+ uint32_t lcore_id; /**< Target CPU core ID */
+ uint32_t queue_id; /**< Queue ID */
+ uint64_t objects; /**< Objects bitmask to stash */
+ } queue;
+ };
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get cache stash capabilities of a port.
+ *
+ * @param port_id
+ * The port identifier.
+ * @param capa
+ * Output: supported stash types and objects.
+ * @return
+ * 0 on success, negative on error.
+ */
+__rte_experimental
+int rte_eth_cache_stash_get(uint16_t port_id, struct rte_eth_cache_stash_capability *capa);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Configure cache stash.
+ *
+ * @param port_id
+ * The port identifier.
+ * @param op
+ * Configuration operation.
+ * @param config
+ * Configuration structure.
+ * @return
+ * 0 on success, negative on error.
+ */
+__rte_experimental
+int rte_eth_cache_stash_set(uint16_t port_id, enum rte_eth_cache_stash_op op,
+ struct rte_eth_cache_stash_config *config);
+
#ifdef __cplusplus
}
#endif
--
2.17.1
^ permalink raw reply related
* [PATCH v2 1/4] bus/pci: introduce PCIe TPH support
From: Chengwen Feng @ 2026-05-12 9:22 UTC (permalink / raw)
To: thomas, stephen; +Cc: dev, wathsala.vithanage, liuyonglong
In-Reply-To: <20260512092302.23735-1-fengchengwen@huawei.com>
Add experimental API and Linux VFIO implementation for PCIe TLP
Processing Hints (TPH). Support device capability query, TPH
enable/disable, and steering tag get/set operations.
Stub functions are added for BSD and Windows.
The API includes:
- rte_pci_tph_query: Query device TPH capabilities
- rte_pci_tph_enable: Enable TPH with specified mode
- rte_pci_tph_disable: Disable TPH on device
- rte_pci_tph_st_get: Get steering tags for CPUs
- rte_pci_tph_st_set: Program steering tags into device's ST table
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
drivers/bus/pci/bsd/pci.c | 51 ++++++++++
drivers/bus/pci/linux/pci.c | 48 +++++++++
drivers/bus/pci/linux/pci_init.h | 9 ++
drivers/bus/pci/linux/pci_vfio.c | 164 +++++++++++++++++++++++++++++++
drivers/bus/pci/rte_bus_pci.h | 114 +++++++++++++++++++++
drivers/bus/pci/windows/pci.c | 51 ++++++++++
kernel/linux/uapi/linux/vfio.h | 41 ++++++++
lib/pci/rte_pci.h | 1 +
8 files changed, 479 insertions(+)
diff --git a/drivers/bus/pci/bsd/pci.c b/drivers/bus/pci/bsd/pci.c
index aba44492e0..15b65091b3 100644
--- a/drivers/bus/pci/bsd/pci.c
+++ b/drivers/bus/pci/bsd/pci.c
@@ -667,3 +667,54 @@ rte_pci_ioport_unmap(struct rte_pci_ioport *p)
return ret;
}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_query, 26.07)
+int
+rte_pci_tph_query(const struct rte_pci_device *dev, uint32_t *supported_modes,
+ uint32_t *st_table_sz)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(supported_modes);
+ RTE_SET_USED(st_table_sz);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_enable, 26.07)
+int
+rte_pci_tph_enable(const struct rte_pci_device *dev, uint32_t mode)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(mode);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_disable, 26.07)
+int
+rte_pci_tph_disable(const struct rte_pci_device *dev)
+{
+ RTE_SET_USED(dev);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_st_get, 26.07)
+int
+rte_pci_tph_st_get(const struct rte_pci_device *dev,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(ents);
+ RTE_SET_USED(count);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_st_set, 26.07)
+int
+rte_pci_tph_st_set(const struct rte_pci_device *dev, uint16_t index,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(index);
+ RTE_SET_USED(ents);
+ RTE_SET_USED(count);
+ return -ENOTSUP;
+}
diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c
index 5f263f8b28..824be47b36 100644
--- a/drivers/bus/pci/linux/pci.c
+++ b/drivers/bus/pci/linux/pci.c
@@ -791,3 +791,51 @@ rte_pci_ioport_unmap(struct rte_pci_ioport *p)
return ret;
}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_query, 26.07)
+int
+rte_pci_tph_query(const struct rte_pci_device *dev, uint32_t *supported_modes,
+ uint32_t *st_table_sz)
+{
+ if (dev->kdrv == RTE_PCI_KDRV_VFIO && pci_vfio_is_enabled())
+ return pci_vfio_tph_query(dev, supported_modes, st_table_sz);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_enable, 26.07)
+int
+rte_pci_tph_enable(const struct rte_pci_device *dev, uint32_t mode)
+{
+ if (dev->kdrv == RTE_PCI_KDRV_VFIO && pci_vfio_is_enabled())
+ return pci_vfio_tph_enable(dev, mode);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_disable, 26.07)
+int
+rte_pci_tph_disable(const struct rte_pci_device *dev)
+{
+ if (dev->kdrv == RTE_PCI_KDRV_VFIO && pci_vfio_is_enabled())
+ return pci_vfio_tph_disable(dev);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_st_get, 26.07)
+int
+rte_pci_tph_st_get(const struct rte_pci_device *dev,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ if (dev->kdrv == RTE_PCI_KDRV_VFIO && pci_vfio_is_enabled())
+ return pci_vfio_tph_st_get(dev, ents, count);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_st_set, 26.07)
+int
+rte_pci_tph_st_set(const struct rte_pci_device *dev, uint16_t index,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ if (dev->kdrv == RTE_PCI_KDRV_VFIO && pci_vfio_is_enabled())
+ return pci_vfio_tph_st_set(dev, index, ents, count);
+ return -ENOTSUP;
+}
diff --git a/drivers/bus/pci/linux/pci_init.h b/drivers/bus/pci/linux/pci_init.h
index 6949dd57d9..c475a4cfbf 100644
--- a/drivers/bus/pci/linux/pci_init.h
+++ b/drivers/bus/pci/linux/pci_init.h
@@ -73,4 +73,13 @@ int pci_vfio_unmap_resource(struct rte_pci_device *dev);
int pci_vfio_is_enabled(void);
+int pci_vfio_tph_query(const struct rte_pci_device *dev, uint32_t *supported_modes,
+ uint32_t *st_table_sz);
+int pci_vfio_tph_enable(const struct rte_pci_device *dev, uint32_t mode);
+int pci_vfio_tph_disable(const struct rte_pci_device *dev);
+int pci_vfio_tph_st_get(const struct rte_pci_device *dev,
+ struct rte_pci_tph_entry *ents, uint32_t count);
+int pci_vfio_tph_st_set(const struct rte_pci_device *dev, uint16_t index,
+ struct rte_pci_tph_entry *ents, uint32_t count);
+
#endif /* EAL_PCI_INIT_H_ */
diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c
index bc5c5c2499..dac745cd74 100644
--- a/drivers/bus/pci/linux/pci_vfio.c
+++ b/drivers/bus/pci/linux/pci_vfio.c
@@ -1308,3 +1308,167 @@ pci_vfio_is_enabled(void)
}
return status;
}
+
+#define PCI_TPH_CAP_OFF 0x4
+#define PCI_TPH_CAP_ST_NS 0x00000001 /* No ST Mode Supported */
+#define PCI_TPH_CAP_ST_IV 0x00000002 /* Interrupt Vector Mode Supported */
+#define PCI_TPH_CAP_ST_DS 0x00000004 /* Device Specific Mode Supported */
+#define PCI_TPH_CAP_LOC_MASK 0x00000600 /* ST Table Location */
+#define PCI_TPH_LOC_NONE 0x00000000 /* Not present */
+#define PCI_TPH_LOC_CAP 0x00000200 /* In capability */
+#define PCI_TPH_LOC_MSIX 0x00000400 /* In MSI-X */
+#define PCI_TPH_CTRL_OFF 0x8
+#define PCI_TPH_ST_NS_MODE 0x0 /* No ST Mode */
+#define PCI_TPH_ST_IV_MODE 0x1 /* Interrupt Vector Mode */
+#define PCI_TPH_ST_DS_MODE 0x2 /* Device Specific Mode */
+
+int
+pci_vfio_tph_query(const struct rte_pci_device *dev, uint32_t *supported_modes,
+ uint32_t *st_table_sz)
+{
+ off_t off = rte_pci_find_ext_capability(dev, RTE_PCI_EXT_CAP_ID_TPH);
+ uint32_t cap, loc;
+ int ret;
+
+ if (off <= 0)
+ return -ENOTSUP;
+
+ ret = rte_pci_read_config(dev, &cap, 4, off + PCI_TPH_CAP_OFF);
+ if (ret != 4)
+ return -EIO;
+
+ *supported_modes = 0;
+ if (cap & PCI_TPH_CAP_ST_IV)
+ *supported_modes |= RTE_PCI_TPH_MODE_IV;
+ if (cap & PCI_TPH_CAP_ST_DS)
+ *supported_modes |= RTE_PCI_TPH_MODE_DS;
+ loc = cap & PCI_TPH_CAP_LOC_MASK;
+ if (loc == PCI_TPH_LOC_CAP || loc == PCI_TPH_LOC_MSIX)
+ *st_table_sz = RTE_FIELD_GET32(PCI_TPH_LOC_MSIX, cap) + 1;
+ else
+ *st_table_sz = 0;
+
+ return 0;
+}
+
+static int
+pci_vfio_tph_ctrl(const struct rte_pci_device *dev, uint32_t mode)
+{
+ off_t off = rte_pci_find_ext_capability(dev, RTE_PCI_EXT_CAP_ID_TPH);
+ int ret;
+
+ if (off <= 0)
+ return -ENOTSUP;
+
+ ret = rte_pci_write_config(dev, &mode, 4, off + PCI_TPH_CTRL_OFF);
+ if (ret != 4)
+ return -EIO;
+
+ return 0;
+}
+
+int
+pci_vfio_tph_enable(const struct rte_pci_device *dev, uint32_t mode)
+{
+ uint32_t st_mode;
+
+ if (mode == RTE_PCI_TPH_MODE_IV)
+ st_mode = PCI_TPH_ST_IV_MODE;
+ else if (mode == RTE_PCI_TPH_MODE_DS)
+ st_mode = PCI_TPH_ST_DS_MODE;
+ else
+ return -EINVAL;
+
+ return pci_vfio_tph_ctrl(dev, st_mode);
+}
+
+int
+pci_vfio_tph_disable(const struct rte_pci_device *dev)
+{
+ return pci_vfio_tph_ctrl(dev, PCI_TPH_ST_NS_MODE);
+}
+
+int
+pci_vfio_tph_st_get(const struct rte_pci_device *dev,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ struct vfio_device_feature_tph_st *tph_st;
+ struct vfio_device_feature *feature;
+ int vfio_dev_fd, ret;
+ size_t argsz;
+ uint32_t i;
+
+ if (count > VFIO_TPH_ST_MAX_COUNT)
+ return -EINVAL;
+
+ vfio_dev_fd = rte_intr_dev_fd_get(dev->intr_handle);
+ if (vfio_dev_fd < 0)
+ return -1;
+
+ argsz = sizeof(struct vfio_device_feature) +
+ sizeof(struct vfio_device_feature_tph_st) +
+ count * sizeof(uint32_t);
+ feature = (struct vfio_device_feature *)calloc(1, argsz);
+ if (feature == NULL)
+ return -ENOMEM;
+ tph_st = (struct vfio_device_feature_tph_st *)feature->data;
+
+ feature->argsz = argsz;
+ feature->flags = VFIO_DEVICE_FEATURE_TPH_ST;
+ feature->flags |= VFIO_DEVICE_FEATURE_GET;
+ for (i = 0; i < count; i++)
+ tph_st->data[i] = ents[i].cpu;
+ tph_st->flags = VFIO_TPH_ST_MEM_TYPE_VM;
+ tph_st->index = 0;
+ tph_st->count = count;
+ ret = ioctl(vfio_dev_fd, VFIO_DEVICE_FEATURE, feature);
+ if (ret) {
+ free(feature);
+ return ret;
+ }
+
+ for (i = 0; i < count; i++)
+ ents[i].st = tph_st->data[i];
+ free(feature);
+
+ return 0;
+}
+
+int
+pci_vfio_tph_st_set(const struct rte_pci_device *dev, uint16_t index,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ struct vfio_device_feature_tph_st *tph_st;
+ struct vfio_device_feature *feature;
+ int vfio_dev_fd, ret;
+ size_t argsz;
+ uint32_t i;
+
+ if (count > VFIO_TPH_ST_MAX_COUNT)
+ return -EINVAL;
+
+ vfio_dev_fd = rte_intr_dev_fd_get(dev->intr_handle);
+ if (vfio_dev_fd < 0)
+ return -1;
+
+ argsz = sizeof(struct vfio_device_feature) +
+ sizeof(struct vfio_device_feature_tph_st) +
+ count * sizeof(uint32_t);
+ feature = (struct vfio_device_feature *)calloc(1, argsz);
+ if (feature == NULL)
+ return -ENOMEM;
+ tph_st = (struct vfio_device_feature_tph_st *)feature->data;
+
+ feature->argsz = argsz;
+ feature->flags = VFIO_DEVICE_FEATURE_TPH_ST;
+ feature->flags |= VFIO_DEVICE_FEATURE_SET;
+ tph_st->flags = VFIO_TPH_ST_MEM_TYPE_VM;
+ tph_st->index = index;
+ tph_st->count = count;
+ for (i = 0; i < count; i++)
+ tph_st->data[i] = ents[i].cpu;
+ ret = ioctl(vfio_dev_fd, VFIO_DEVICE_FEATURE, feature);
+ free(feature);
+
+ return ret;
+}
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 19a7b15b99..623c80e855 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -312,6 +312,120 @@ void rte_pci_ioport_read(struct rte_pci_ioport *p,
void rte_pci_ioport_write(struct rte_pci_ioport *p,
const void *data, size_t len, off_t offset);
+#define RTE_PCI_TPH_MODE_IV (1u << 0) /* Interrupt vector */
+#define RTE_PCI_TPH_MODE_DS (1u << 1) /* Device specific */
+
+/**
+ * @struct rte_pci_tph_entry
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice.
+ *
+ * An entry used for TPH Steering Tag (ST) get/set operations.
+ */
+struct rte_pci_tph_entry {
+ /**
+ * CPU ID used for both get and set operations.
+ * For set operation: if set to U32_MAX, clear the ST entry at
+ * specified index.
+ */
+ uint32_t cpu;
+ /**
+ * Steering tag value, only used for get operation to return result.
+ */
+ uint16_t st;
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Query PCIe TLP Processing Hints (TPH) capabilities of a device.
+ *
+ * @param dev
+ * A pointer to a rte_pci_device structure describing the device to query.
+ * @param supported_modes
+ * Output: supported TPH modes (RTE_PCI_TPH_MODE_*).
+ * @param st_table_sz
+ * Output: number of entries in the ST table; 0 means no table present.
+ * @return
+ * 0 on success, negative value on error.
+ */
+__rte_experimental
+int rte_pci_tph_query(const struct rte_pci_device *dev, uint32_t *supported_modes,
+ uint32_t *st_table_sz);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Enable PCIe TLP Processing Hints (TPH) on a device with specified mode.
+ *
+ * @param dev
+ * A pointer to a rte_pci_device structure describing the device to enable.
+ * @param mode
+ * TPH operating mode (RTE_PCI_TPH_MODE_*).
+ * @return
+ * 0 on success, negative value on error.
+ */
+__rte_experimental
+int rte_pci_tph_enable(const struct rte_pci_device *dev, uint32_t mode);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Disable PCIe TLP Processing Hints (TPH) on a device.
+ *
+ * @param dev
+ * A pointer to a rte_pci_device structure describing the device to disable.
+ * @return
+ * 0 on success, negative value on error.
+ */
+__rte_experimental
+int rte_pci_tph_disable(const struct rte_pci_device *dev);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get steering tags for given CPU IDs from the device.
+ * Only valid when TPH is enabled in Device-Specific (DS) mode.
+ *
+ * @param dev
+ * A pointer to a rte_pci_device structure describing the device.
+ * @param ents
+ * Array of entries with CPU IDs as input; steering tags are returned
+ * as output.
+ * @param count
+ * Number of entries in the array.
+ * @return
+ * 0 on success, negative value on error.
+ */
+__rte_experimental
+int rte_pci_tph_st_get(const struct rte_pci_device *dev,
+ struct rte_pci_tph_entry *ents, uint32_t count);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Program steering tags into the device's ST table.
+ *
+ * @param dev
+ * A pointer to a rte_pci_device structure describing the device.
+ * @index
+ * Start ST table index to program.
+ * @param ents
+ * Array of entries with CPU IDs and index indices to program.
+ * @param count
+ * Number of entries in the array.
+ * @return
+ * 0 on success, negative errno value on error.
+ */
+__rte_experimental
+int rte_pci_tph_st_set(const struct rte_pci_device *dev, uint16_t index,
+ struct rte_pci_tph_entry *ents, uint32_t count);
+
#ifdef __cplusplus
}
#endif
diff --git a/drivers/bus/pci/windows/pci.c b/drivers/bus/pci/windows/pci.c
index 549319ad5b..53d7bebde9 100644
--- a/drivers/bus/pci/windows/pci.c
+++ b/drivers/bus/pci/windows/pci.c
@@ -207,6 +207,57 @@ pci_uio_remap_resource(struct rte_pci_device *dev __rte_unused)
return -1;
}
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_query, 26.07)
+int
+rte_pci_tph_query(const struct rte_pci_device *dev, uint32_t *supported_modes,
+ uint32_t *st_table_sz)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(supported_modes);
+ RTE_SET_USED(st_table_sz);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_enable, 26.07)
+int
+rte_pci_tph_enable(const struct rte_pci_device *dev, uint32_t mode)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(mode);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_disable, 26.07)
+int
+rte_pci_tph_disable(const struct rte_pci_device *dev)
+{
+ RTE_SET_USED(dev);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_st_get, 26.07)
+int
+rte_pci_tph_st_get(const struct rte_pci_device *dev,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(ents);
+ RTE_SET_USED(count);
+ return -ENOTSUP;
+}
+
+RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_pci_tph_st_set, 26.07)
+int
+rte_pci_tph_st_set(const struct rte_pci_device *dev, uint16_t index,
+ struct rte_pci_tph_entry *ents, uint32_t count)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(index);
+ RTE_SET_USED(ents);
+ RTE_SET_USED(count);
+ return -ENOTSUP;
+}
+
static int
get_device_pci_address(HDEVINFO dev_info,
PSP_DEVINFO_DATA device_info_data, struct rte_pci_addr *addr)
diff --git a/kernel/linux/uapi/linux/vfio.h b/kernel/linux/uapi/linux/vfio.h
index 79bf8c0cc5..f0676310bf 100644
--- a/kernel/linux/uapi/linux/vfio.h
+++ b/kernel/linux/uapi/linux/vfio.h
@@ -1468,6 +1468,47 @@ struct vfio_device_feature_bus_master {
};
#define VFIO_DEVICE_FEATURE_BUS_MASTER 10
+/**
+ * VFIO_DEVICE_FEATURE_TPH_ST - Get/Set PCIe TPH Steering Tag (ST) entries
+ *
+ * Provides userspace interface to manage PCIe TPH ST table entries.
+ * This feature is only available when device TPH is enabled.
+ *
+ * Upon VFIO_DEVICE_FEATURE_SET:
+ * Program contiguous ST table entries from the starting @index.
+ * Valid only for hardware with ST table located in TPH Capability
+ * space or MSI-X table. If an entry CPU ID is specified as U32_MAX,
+ * the corresponding ST entry will be cleared. @index and @count define
+ * the contiguous entry range to be programmed.
+ * If any entry programming fails, the operation will roll back and
+ * clear all entries that were successfully programmed before the error.
+ *
+ * Upon VFIO_DEVICE_FEATURE_GET:
+ * Retrieve the ST value mapped to each given CPU ID in the @data array.
+ * Userspace fills @data with CPU ID array, the interface returns each
+ * CPU's corresponding ST value back in place.
+ * Valid only when TPH DS mode is enabled.
+ *
+ * @flags: Operation flags (VFIO_TPH_ST_MEM_TYPE_*).
+ * @index: Starting ST entry index, only valid for FEATURE_SET.
+ * @count: Number of contiguous entries to access.
+ * @data: Array of CPU IDs for both SET and GET. On SET it programs ST for
+ * each CPU; on GET it returns the mapped ST value of each CPU.
+ *
+ * This feature is gated by enable_unsafe_tph module parameter.
+ */
+#define VFIO_DEVICE_FEATURE_TPH_ST 13
+
+struct vfio_device_feature_tph_st {
+ __u32 flags;
+#define VFIO_TPH_ST_MEM_TYPE_VM (0U << 0)
+#define VFIO_TPH_ST_MEM_TYPE_PM (1U << 0)
+ __u16 index;
+ __u16 count;
+#define VFIO_TPH_ST_MAX_COUNT 2048
+ __u32 data[];
+};
+
/* -------- API for Type1 VFIO IOMMU -------- */
/**
diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
index 9d04978a0f..f9129723eb 100644
--- a/lib/pci/rte_pci.h
+++ b/lib/pci/rte_pci.h
@@ -111,6 +111,7 @@ extern "C" {
#define RTE_PCI_EXT_CAP_ID_ACS 0x0d /* Access Control Services */
#define RTE_PCI_EXT_CAP_ID_SRIOV 0x10 /* SR-IOV */
#define RTE_PCI_EXT_CAP_ID_PRI 0x13 /* Page Request Interface */
+#define RTE_PCI_EXT_CAP_ID_TPH 0x17 /* TLP Processing Hints */
#define RTE_PCI_EXT_CAP_ID_PASID 0x1b /* Process Address Space ID */
/* Advanced Error Reporting (RTE_PCI_EXT_CAP_ID_ERR) */
--
2.17.1
^ permalink raw reply related
* [PATCH v2 4/4] app/testpmd: add TPH stash objects configuration
From: Chengwen Feng @ 2026-05-12 9:23 UTC (permalink / raw)
To: thomas, stephen; +Cc: dev, wathsala.vithanage, liuyonglong
In-Reply-To: <20260512092302.23735-1-fengchengwen@huawei.com>
Add --tph-stash-objects command line option to configure PCIe TPH cache
stash objects for Rx queues. Implement enable/disable logic during
packet forwarding start/stop with device capability checks.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
app/test-pmd/parameters.c | 20 ++++++++++
app/test-pmd/testpmd.c | 81 +++++++++++++++++++++++++++++++++++++++
app/test-pmd/testpmd.h | 1 +
3 files changed, 102 insertions(+)
diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index ecbd618f00..ae31d9ba4c 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -79,6 +79,8 @@ enum {
TESTPMD_OPT_NO_NUMA_NUM,
#define TESTPMD_OPT_MP_ANON "mp-anon"
TESTPMD_OPT_MP_ANON_NUM,
+#define TESTPMD_OPT_TPH_STASH_OBJECTS "tph-stash-objects"
+ TESTPMD_OPT_TPH_STASH_OBJECTS_NUM,
#define TESTPMD_OPT_PORT_NUMA_CONFIG "port-numa-config"
TESTPMD_OPT_PORT_NUMA_CONFIG_NUM,
#define TESTPMD_OPT_RING_NUMA_CONFIG "ring-numa-config"
@@ -289,6 +291,7 @@ static const struct option long_options[] = {
NO_ARG(TESTPMD_OPT_NUMA),
NO_ARG(TESTPMD_OPT_NO_NUMA),
NO_ARG(TESTPMD_OPT_MP_ANON), /* deprecated */
+ REQUIRED_ARG(TESTPMD_OPT_TPH_STASH_OBJECTS),
REQUIRED_ARG(TESTPMD_OPT_PORT_NUMA_CONFIG),
REQUIRED_ARG(TESTPMD_OPT_RING_NUMA_CONFIG),
REQUIRED_ARG(TESTPMD_OPT_SOCKET_NUM),
@@ -1140,6 +1143,23 @@ launch_args_parse(int argc, char** argv)
"native, anon, xmem or xmemhuge\n",
optarg);
break;
+ case TESTPMD_OPT_TPH_STASH_OBJECTS_NUM:
+ if (!strcmp(optarg, "rxdesc"))
+ tph_stash_objects = RTE_ETH_CACHE_STASH_OBJ_RX_DESC;
+ else if (!strcmp(optarg, "rxheader"))
+ tph_stash_objects = RTE_ETH_CACHE_STASH_OBJ_RX_HEADER;
+ else if (!strcmp(optarg, "rxpayload"))
+ tph_stash_objects = RTE_ETH_CACHE_STASH_OBJ_RX_PAYLOAD;
+ else if (!strcmp(optarg, "rxdata"))
+ tph_stash_objects = RTE_ETH_CACHE_STASH_OBJ_RX_HEADER |
+ RTE_ETH_CACHE_STASH_OBJ_RX_PAYLOAD;
+ else if (!strcmp(optarg, "rxall"))
+ tph_stash_objects = RTE_ETH_CACHE_STASH_OBJ_RX_DESC |
+ RTE_ETH_CACHE_STASH_OBJ_RX_HEADER |
+ RTE_ETH_CACHE_STASH_OBJ_RX_PAYLOAD;
+ else
+ rte_exit(EXIT_FAILURE, "unknown tph stash mode %s\n", optarg);
+ break;
case TESTPMD_OPT_PORT_NUMA_CONFIG_NUM:
if (parse_portnuma_config(optarg))
rte_exit(EXIT_FAILURE,
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index e2569d9e30..6f94dd3b76 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -136,6 +136,11 @@ uint8_t socket_num = UMA_NO_CONFIG;
*/
uint8_t mp_alloc_type = MP_ALLOC_NATIVE;
+/*
+ * PCIE TPH stash objects.
+ */
+uint64_t tph_stash_objects;
+
/*
* Store specified sockets on which memory pool to be used by ports
* is allocated.
@@ -2540,6 +2545,77 @@ update_queue_state(portid_t pid)
}
}
+static void
+start_tph_stash(void)
+{
+ struct rte_eth_cache_stash_capability capa;
+ struct rte_eth_cache_stash_config config;
+ struct fwd_config *cfg = &cur_fwd_config;
+ struct fwd_stream *fs;
+ lcoreid_t lc_id;
+ streamid_t sm_id;
+ portid_t pt_id;
+ int ret;
+ int i;
+
+ for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {
+ pt_id = fwd_ports_ids[i];
+ ret = rte_eth_cache_stash_get(pt_id, &capa);
+ if (ret != 0 || (capa.supported_types & RTE_ETH_CACHE_STASH_TYPE_TPH) == 0) {
+ fprintf(stderr, "%s: (port %u) don't support tph stash!\n",
+ __func__, pt_id);
+ return;
+ }
+ if ((capa.supported_objects & tph_stash_objects) != tph_stash_objects) {
+ fprintf(stderr, "%s: (port %u) don't support objects=0x%"PRIx64" 0x%"PRIx64"\n",
+ __func__, pt_id, tph_stash_objects, capa.supported_objects);
+ return;
+ }
+ memset(&config, 0, sizeof(config));
+ config.dev.type = RTE_ETH_CACHE_STASH_TYPE_TPH;
+ ret = rte_eth_cache_stash_set(pt_id, RTE_ETH_CACHE_STASH_OP_DEV_ENABLE, &config);
+ if (ret != 0) {
+ fprintf(stderr, "%s: (port %u) enable tph failed! ret=%d\n",
+ __func__, pt_id, ret);
+ return;
+ }
+ }
+
+ for (lc_id = 0; lc_id < cfg->nb_fwd_lcores; lc_id++) {
+ for (sm_id = 0; sm_id < fwd_lcores[lc_id]->stream_nb; sm_id++) {
+ fs = fwd_streams[fwd_lcores[lc_id]->stream_idx + sm_id];
+ memset(&config, 0, sizeof(config));
+ config.queue.lcore_id = fwd_lcores_cpuids[lc_id];
+ config.queue.queue_id = fs->rx_queue;
+ config.queue.objects = tph_stash_objects;
+ ret = rte_eth_cache_stash_set(fs->rx_port,
+ RTE_ETH_CACHE_STASH_OP_QUEUE_ENABLE, &config);
+ if (ret != 0)
+ fprintf(stderr, "%s: (port %u) enable rx-queue=%u cpu=%u stash ret=%d\n",
+ __func__, fs->rx_port, fs->rx_queue,
+ fwd_lcores_cpuids[lc_id], ret);
+ }
+ }
+}
+
+static void
+stop_tph_stash(void)
+{
+ struct rte_eth_cache_stash_config config;
+ portid_t pt_id;
+ int ret;
+ int i;
+
+ for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {
+ pt_id = fwd_ports_ids[i];
+ memset(&config, 0, sizeof(config));
+ ret = rte_eth_cache_stash_set(pt_id, RTE_ETH_CACHE_STASH_OP_DEV_DISABLE, &config);
+ if (ret != 0)
+ fprintf(stderr, "%s: (port %u) disable tph stash ret=%d\n",
+ __func__, pt_id, ret);
+ }
+}
+
/*
* Launch packet forwarding configuration.
*/
@@ -2614,6 +2690,9 @@ start_packet_forwarding(int with_tx_first)
if(!no_flush_rx)
flush_fwd_rx_queues();
+ if (tph_stash_objects > 0)
+ start_tph_stash();
+
rxtx_config_display();
fwd_stats_reset();
@@ -2649,6 +2728,8 @@ stop_packet_forwarding(void)
fwd_lcores[lc_id]->stopped = 1;
printf("\nWaiting for lcores to finish...\n");
rte_eal_mp_wait_lcore();
+ if (tph_stash_objects > 0)
+ stop_tph_stash();
port_fwd_end = cur_fwd_config.fwd_eng->port_fwd_end;
if (port_fwd_end != NULL) {
for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 9b60ebd7fc..4124d40fda 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -535,6 +535,7 @@ extern uint8_t flow_isolate_all; /**< set by "--flow-isolate-all */
extern uint8_t no_flow_flush; /**< set by "--disable-flow-flush" parameter */
extern uint8_t mp_alloc_type;
/**< set by "--mp-anon" or "--mp-alloc" parameter */
+extern uint64_t tph_stash_objects; /**< set by "--tph-stash-objects" parameter */
extern uint32_t eth_link_speed;
extern uint8_t no_link_check; /**<set by "--disable-link-check" parameter */
extern uint8_t no_device_start; /**<set by "--disable-device-start" parameter */
--
2.17.1
^ permalink raw reply related
* [PATCH v2 0/4] Introduce generic cache stash API and PCIe TPH implementation
From: Chengwen Feng @ 2026-05-12 9:22 UTC (permalink / raw)
To: thomas, stephen; +Cc: dev, wathsala.vithanage, liuyonglong
In-Reply-To: <20260508092855.51987-1-fengchengwen@huawei.com>
This patch set introduces a generic ethdev cache stash framework,
provides PCIe TPH (Transaction Processing Hints) implementation
on PCI/VFIO bus, enables TPH cache stash support for e1000/igb PMD,
and adds testpmd command line configuration.
The patchset is organized as follows:
- Patch 1/4: bus/pci: introduce PCIe TPH support
Add generic PCIe TPH APIs and VFIO backend implementation,
with stub implementations for BSD and Windows.
- Patch 2/4: ethdev: introduce generic cache stash API
Define unified cache stash capability/operation/config abstraction,
support device-level and per-queue stash management.
- Patch 3/4: net/e1000: add cache stash support via TPH
Implement ethdev cache stash ops for igb PMD based on PCIe TPH.
- Patch 4/4: app/testpmd: add TPH stash objects configuration
Add testpmd CLI option to configure stash objects and
automatically apply stash enable/disable on forwarding start/stop.
Note:
1. This implementation references the Linux kernel VFIO TPH uapi
series (v9 version), which is not yet merged upstream:
https://lore.kernel.org/all/20260512080329.42593-1-fengchengwen@huawei.com/
2. The API design and cache stash model learn from the prior work
by Wathsala Vithanage: "An API for Cache Stashing with TPH".
3. The e1000/igb PMD is enabled as the first user of this API
because it is the only PCIe TPH capable hardware available
for testing at this time. The generic API design is extensible
to other NICs and future cache stash mechanisms.
--
v2: Sync kernel side v9
Move vfio interface to kernel/linux/uapi/linux/vfio.h
Fix CI compile error
Chengwen Feng (4):
bus/pci: introduce PCIe TPH support
ethdev: introduce generic cache stash API
net/e1000: add cache stash support via TPH
app/testpmd: add TPH stash objects configuration
app/test-pmd/parameters.c | 20 +++
app/test-pmd/testpmd.c | 81 ++++++++++
app/test-pmd/testpmd.h | 1 +
drivers/bus/pci/bsd/pci.c | 51 ++++++
drivers/bus/pci/linux/pci.c | 48 ++++++
drivers/bus/pci/linux/pci_init.h | 9 ++
drivers/bus/pci/linux/pci_vfio.c | 164 +++++++++++++++++++
drivers/bus/pci/rte_bus_pci.h | 114 +++++++++++++
drivers/bus/pci/windows/pci.c | 51 ++++++
drivers/net/intel/e1000/base/e1000_hw.h | 2 +
drivers/net/intel/e1000/base/e1000_vf.h | 2 +
drivers/net/intel/e1000/igb_ethdev.c | 205 ++++++++++++++++++++++++
kernel/linux/uapi/linux/vfio.h | 41 +++++
lib/ethdev/ethdev_driver.h | 37 +++++
lib/ethdev/rte_ethdev.c | 40 +++++
lib/ethdev/rte_ethdev.h | 95 +++++++++++
lib/pci/rte_pci.h | 1 +
17 files changed, 962 insertions(+)
--
2.17.1
^ permalink raw reply
* [PATCH v2 3/4] net/e1000: add cache stash support via TPH
From: Chengwen Feng @ 2026-05-12 9:23 UTC (permalink / raw)
To: thomas, stephen; +Cc: dev, wathsala.vithanage, liuyonglong
In-Reply-To: <20260512092302.23735-1-fengchengwen@huawei.com>
Implement ethdev generic cache stash operations for igb PMD
using PCIe TPH (Transaction Processing Hints) as the underlying
mechanism.
The implementation supports:
- Query TPH based stash capabilities
- Device-level global enable/disable of TPH
- Per-queue TPH stashing configuration with target lcore
- Configure stashing objects (Rx descriptor, payload, etc.)
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
drivers/net/intel/e1000/base/e1000_hw.h | 2 +
drivers/net/intel/e1000/base/e1000_vf.h | 2 +
drivers/net/intel/e1000/igb_ethdev.c | 205 ++++++++++++++++++++++++
3 files changed, 209 insertions(+)
diff --git a/drivers/net/intel/e1000/base/e1000_hw.h b/drivers/net/intel/e1000/base/e1000_hw.h
index 9b1fafd75c..3e8e21b194 100644
--- a/drivers/net/intel/e1000/base/e1000_hw.h
+++ b/drivers/net/intel/e1000/base/e1000_hw.h
@@ -1092,6 +1092,8 @@ struct e1000_hw {
u16 vendor_id;
u8 revision_id;
+
+ u8 tph_mode;
};
#include "e1000_82541.h"
diff --git a/drivers/net/intel/e1000/base/e1000_vf.h b/drivers/net/intel/e1000/base/e1000_vf.h
index 4bec21c935..dd3cef254e 100644
--- a/drivers/net/intel/e1000/base/e1000_vf.h
+++ b/drivers/net/intel/e1000/base/e1000_vf.h
@@ -248,6 +248,8 @@ struct e1000_hw {
u16 vendor_id;
u8 revision_id;
+
+ u8 tph_mode;
};
enum e1000_promisc_type {
diff --git a/drivers/net/intel/e1000/igb_ethdev.c b/drivers/net/intel/e1000/igb_ethdev.c
index ef1599ac38..0f06139786 100644
--- a/drivers/net/intel/e1000/igb_ethdev.c
+++ b/drivers/net/intel/e1000/igb_ethdev.c
@@ -241,6 +241,10 @@ static int igb_tx_burst_mode_get(struct rte_eth_dev *dev,
__rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
static int igb_rx_burst_mode_get(struct rte_eth_dev *dev,
__rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
+static int eth_igb_cache_stash_get(struct rte_eth_dev *dev,
+ struct rte_eth_cache_stash_capability *capa);
+static int eth_igb_cache_stash_set(struct rte_eth_dev *dev, enum rte_eth_cache_stash_op op,
+ struct rte_eth_cache_stash_config *config);
/*
* Define VF Stats MACRO for Non "cleared on read" register
@@ -402,6 +406,8 @@ static const struct eth_dev_ops eth_igb_ops = {
.timesync_read_time = igb_timesync_read_time,
.timesync_write_time = igb_timesync_write_time,
.read_clock = eth_igb_read_clock,
+ .cache_stash_get = eth_igb_cache_stash_get,
+ .cache_stash_set = eth_igb_cache_stash_set,
};
/*
@@ -5692,6 +5698,205 @@ igb_filter_restore(struct rte_eth_dev *dev)
return 0;
}
+static int
+eth_igb_cache_stash_get(struct rte_eth_dev *dev, struct rte_eth_cache_stash_capability *capa)
+{
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ uint32_t support_modes, st_table_sz;
+ int ret;
+
+ ret = rte_pci_tph_query(pci_dev, &support_modes, &st_table_sz);
+ if (ret != 0)
+ return -ENOTSUP;
+ capa->supported_types = RTE_ETH_CACHE_STASH_TYPE_TPH;
+ capa->supported_objects = RTE_ETH_CACHE_STASH_OBJ_RX_DESC |
+ RTE_ETH_CACHE_STASH_OBJ_RX_HEADER |
+ RTE_ETH_CACHE_STASH_OBJ_RX_PAYLOAD |
+ RTE_ETH_CACHE_STASH_OBJ_TX_DESC |
+ RTE_ETH_CACHE_STASH_OBJ_TX_HEADER |
+ RTE_ETH_CACHE_STASH_OBJ_TX_PAYLOAD;
+ return 0;
+}
+
+static int
+eth_igb_cache_stash_dev_enable(struct rte_eth_dev *dev,
+ struct rte_eth_cache_stash_config *config)
+{
+ struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ uint32_t support_modes, st_table_sz;
+ uint32_t mode;
+ int ret;
+
+ if (hw->tph_mode != 0) {
+ PMD_DRV_LOG(ERR, "Already enable tph_mode=%u", hw->tph_mode);
+ return -EINVAL;
+ }
+
+ if (config->dev.type != RTE_ETH_CACHE_STASH_TYPE_TPH) {
+ PMD_DRV_LOG(ERR, "Unsupported stash type=%u!", config->dev.type);
+ return -ENOTSUP;
+ }
+
+ ret = rte_pci_tph_query(pci_dev, &support_modes, &st_table_sz);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Query TPH failed! ret=%d", ret);
+ return -ENOTSUP;
+ }
+
+ if (support_modes & RTE_PCI_TPH_MODE_IV)
+ mode = RTE_PCI_TPH_MODE_IV;
+ else if (support_modes & RTE_PCI_TPH_MODE_DS)
+ mode = RTE_PCI_TPH_MODE_DS;
+ else
+ return -ENOTSUP;
+ ret = rte_pci_tph_enable(pci_dev, mode);
+ if (ret == 0)
+ hw->tph_mode = mode;
+
+ return ret;
+}
+
+static void
+eth_igb_cache_stash_dev_clear(struct rte_eth_dev *dev)
+{
+ uint32_t nb_queues = RTE_MAX(dev->data->nb_rx_queues, dev->data->nb_tx_queues);
+ struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ struct rte_pci_tph_entry ent = {0};
+ uint32_t reg;
+ uint32_t i;
+
+ for (i = 0; i < nb_queues; i++) {
+ ent.cpu = UINT32_MAX;
+ rte_pci_tph_st_set(pci_dev, i, &ent, 1);
+ reg = E1000_READ_REG(hw, E1000_RXCTL(i));
+ reg &= ~RTE_BIT32(0);
+ reg &= ~RTE_BIT32(2);
+ reg &= ~RTE_BIT32(3);
+ reg &= ~RTE_SHIFT_VAL32(0xFF, 24);
+ E1000_WRITE_REG(hw, E1000_RXCTL(i), reg);
+ }
+}
+
+static int
+eth_igb_cache_stash_dev_disable(struct rte_eth_dev *dev)
+{
+ struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ int ret;
+
+ if (hw->tph_mode == 0) {
+ PMD_DRV_LOG(ERR, "Already disable TPH!");
+ return -EINVAL;
+ }
+
+ eth_igb_cache_stash_dev_clear(dev);
+ ret = rte_pci_tph_disable(pci_dev);
+ if (ret == 0)
+ hw->tph_mode = 0;
+
+ return ret;
+}
+
+static int
+eth_igb_cache_stash_queue_enable(struct rte_eth_dev *dev,
+ struct rte_eth_cache_stash_config *config)
+{
+ struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
+ uint32_t queue_id = config->queue.queue_id;
+ struct rte_pci_tph_entry ent = {0};
+ uint32_t reg;
+ uint16_t st;
+ int ret;
+
+ if (hw->tph_mode == 0) {
+ PMD_DRV_LOG(ERR, "Device TPH was not enabled!");
+ return -EINVAL;
+ }
+
+ ent.cpu = config->queue.lcore_id;
+ ret = rte_pci_tph_st_get(pci_dev, &ent, 1);
+ if (ret != 0) {
+ PMD_DRV_LOG(ERR, "Failed to get device queue=%u st of cpu=%u ret=%u",
+ queue_id, config->queue.lcore_id, ret);
+ return ret;
+ }
+ st = ent.st;
+
+ reg = E1000_READ_REG(hw, E1000_RXCTL(queue_id));
+ PMD_DRV_LOG(DEBUG, "[Enable] Queue=%u rxctl register init val=0x%x", queue_id, reg);
+ if (config->queue.objects & RTE_ETH_CACHE_STASH_OBJ_RX_DESC)
+ reg |= RTE_BIT32(0);
+ else
+ reg &= ~RTE_BIT32(0);
+ if (config->queue.objects & RTE_ETH_CACHE_STASH_OBJ_RX_HEADER)
+ reg |= RTE_BIT32(2);
+ else
+ reg &= ~RTE_BIT32(2);
+ if (config->queue.objects & RTE_ETH_CACHE_STASH_OBJ_RX_PAYLOAD)
+ reg |= RTE_BIT32(3);
+ else
+ reg &= ~RTE_BIT32(3);
+ /* I350 must encoding st in high 8bit, and ST in config-space is no needed! */
+ reg |= RTE_SHIFT_VAL32(st, 24);
+ E1000_WRITE_REG(hw, E1000_RXCTL(queue_id), reg);
+ PMD_DRV_LOG(DEBUG, "[Enable] Queue=%u rxctl register after val=0x%x", queue_id, reg);
+
+ PMD_DRV_LOG(DEBUG, "[Enable] Enable device queue=%u st of cpu=%u success!",
+ queue_id, config->queue.lcore_id);
+
+ return 0;
+}
+
+static int
+eth_igb_cache_stash_queue_disable(struct rte_eth_dev *dev,
+ struct rte_eth_cache_stash_config *config)
+{
+ struct e1000_hw *hw = E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ uint32_t queue_id = config->queue.queue_id;
+ uint32_t reg;
+
+ if (hw->tph_mode == 0) {
+ PMD_DRV_LOG(ERR, "Device TPH was not enabled!");
+ return -EINVAL;
+ }
+
+ reg = E1000_READ_REG(hw, E1000_RXCTL(queue_id));
+ PMD_DRV_LOG(DEBUG, "[Disable] Queue=%u rxctl register init val=0x%x", queue_id, reg);
+ reg &= ~RTE_BIT32(0);
+ reg &= ~RTE_BIT32(1);
+ reg &= ~RTE_BIT32(2);
+ reg &= ~RTE_BIT32(3);
+ reg &= ~RTE_SHIFT_VAL32(0xFF, 24);
+ E1000_WRITE_REG(hw, E1000_RXCTL(queue_id), reg);
+ PMD_DRV_LOG(DEBUG, "[Disable] Queue=%u rxctl register after val=0x%x", queue_id, reg);
+
+ PMD_DRV_LOG(DEBUG, "[Disable] disable device queue=%u st of cpu=%u success!",
+ queue_id, config->queue.lcore_id);
+
+ return 0;
+}
+
+static int
+eth_igb_cache_stash_set(struct rte_eth_dev *dev, enum rte_eth_cache_stash_op op,
+ struct rte_eth_cache_stash_config *config)
+{
+ switch (op) {
+ case RTE_ETH_CACHE_STASH_OP_DEV_ENABLE:
+ return eth_igb_cache_stash_dev_enable(dev, config);
+ case RTE_ETH_CACHE_STASH_OP_DEV_DISABLE:
+ return eth_igb_cache_stash_dev_disable(dev);
+ case RTE_ETH_CACHE_STASH_OP_QUEUE_ENABLE:
+ return eth_igb_cache_stash_queue_enable(dev, config);
+ case RTE_ETH_CACHE_STASH_OP_QUEUE_DISABLE:
+ return eth_igb_cache_stash_queue_disable(dev, config);
+ default:
+ return -ENOTSUP;
+ }
+}
+
RTE_PMD_REGISTER_PCI(net_e1000_igb, rte_igb_pmd);
RTE_PMD_REGISTER_PCI_TABLE(net_e1000_igb, pci_id_igb_map);
RTE_PMD_REGISTER_KMOD_DEP(net_e1000_igb, "* igb_uio | uio_pci_generic | vfio-pci");
--
2.17.1
^ permalink raw reply related
* Re: [PATCH 0/6] net/gve: add hardware timestamping support
From: Stephen Hemminger @ 2026-05-12 8:26 UTC (permalink / raw)
To: Mark Blasko; +Cc: dev
In-Reply-To: <20260512005404.946979-1-blasko@google.com>
On Tue, 12 May 2026 00:53:47 +0000
Mark Blasko <blasko@google.com> wrote:
> This patch series introduces support for GVE hardware timestamping on DQO
> queues. To support concurrent access, a mutex lock is introduced to protect
> admin queue operations. A mechanism is then added to periodically synchronize
> the NIC clock via AdminQ, and support is introduced for the read_clock ethdev
> operation. Finally, the RX datapath is updated to reconstruct full 64-bit
> timestamps from the 32-bit values in DQO descriptors.
>
> Mark Blasko (6):
> net/gve: add thread safety to admin queue
> net/gve: add device option support for HW timestamps
> net/gve: add AdminQ command for NIC timestamps
> net/gve: add periodic NIC clock synchronization
> net/gve: support read clock ethdev op
> net/gve: reconstruct HW timestamps from DQO
>
> .mailmap | 1 +
> doc/guides/nics/features/gve.ini | 1 +
> doc/guides/nics/gve.rst | 18 +++
> doc/guides/rel_notes/release_26_07.rst | 3 +
> drivers/net/gve/base/gve_adminq.c | 127 +++++++++++++++++----
> drivers/net/gve/base/gve_adminq.h | 29 +++++
> drivers/net/gve/base/gve_desc_dqo.h | 8 +-
> drivers/net/gve/gve_ethdev.c | 148 ++++++++++++++++++++++++-
> drivers/net/gve/gve_ethdev.h | 39 +++++++
> drivers/net/gve/gve_rx_dqo.c | 26 +++++
> 10 files changed, 378 insertions(+), 22 deletions(-)
>
Please rate limit submissions to mailing list to one series per day.
^ permalink raw reply
* Re: [PATCH 6/6] net/gve: reconstruct HW timestamps from DQO
From: Stephen Hemminger @ 2026-05-12 8:26 UTC (permalink / raw)
To: mark-blasko; +Cc: dev, Joshua Washington, Jasper Tran O'Leary
In-Reply-To: <20260512005057.944672-7-blasko@google.com>
On Tue, 12 May 2026 00:50:55 +0000
mark-blasko <blasko@google.com> wrote:
> diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
> index 1b012c4776..db886f19cf 100644
> --- a/doc/guides/rel_notes/release_26_07.rst
> +++ b/doc/guides/rel_notes/release_26_07.rst
> @@ -62,6 +62,9 @@ New Features
> * ``-A`` or ``--no-auto-probing`` disable the initial bus probing: no device is probed during
> ``rte_eal_init`` and the application is responsible for probing each device,
> * ``--auto-probing`` enables the initial bus probing, which is the current default behavior.
> +* **Updated Google GVE net driver.**
> +
> + * Added hardware timestamping support on DQO queues.
>
> * **Updated PCAP ethernet driver.**
Need blank line between release note entries.
Run doc build to see errors like this
^ permalink raw reply
* [PATCH v12 09/10] drivers: add data path for Rx and Tx
From: liujie5 @ 2026-05-12 8:06 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512080616.454171-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Implement receive and transmit burst functions for sxe2 PMD.
Add sxe2_recv_pkts and sxe2_xmit_pkts as the primary data path
interfaces.
The implementation includes:
- Efficient descriptor fetching and mbuf allocation for Rx.
- Descriptor setup and checksum offload handling for Tx.
- Buffer recycling and hardware tail pointer updates.
- Performance-oriented loop unrolling and prefetching where applicable.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_common.c | 1 +
drivers/common/sxe2/sxe2_common_log.h | 1 -
drivers/common/sxe2/sxe2_errno.h | 3 -
drivers/common/sxe2/sxe2_ioctl_chnl.c | 8 +-
drivers/common/sxe2/sxe2_osal.h | 4 +-
drivers/common/sxe2/sxe2_type.h | 1 -
drivers/net/sxe2/meson.build | 2 +
drivers/net/sxe2/sxe2_ethdev.c | 22 +-
drivers/net/sxe2/sxe2_txrx.c | 247 +++++++
drivers/net/sxe2/sxe2_txrx.h | 21 +
drivers/net/sxe2/sxe2_txrx_poll.c | 945 ++++++++++++++++++++++++++
11 files changed, 1238 insertions(+), 17 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx.c
create mode 100644 drivers/net/sxe2/sxe2_txrx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.c
diff --git a/drivers/common/sxe2/sxe2_common.c b/drivers/common/sxe2/sxe2_common.c
index e04982e92f..73b288d5d8 100644
--- a/drivers/common/sxe2/sxe2_common.c
+++ b/drivers/common/sxe2/sxe2_common.c
@@ -664,6 +664,7 @@ sxe2_common_init(void)
goto l_end;
pthread_mutex_init(&sxe2_common_devices_list_lock, NULL);
+
sxe2_common_pci_init();
sxe2_commoin_inited = true;
diff --git a/drivers/common/sxe2/sxe2_common_log.h b/drivers/common/sxe2/sxe2_common_log.h
index a7d2157610..cbb53263b5 100644
--- a/drivers/common/sxe2/sxe2_common_log.h
+++ b/drivers/common/sxe2/sxe2_common_log.h
@@ -81,4 +81,3 @@ extern s32 sxe2_log_hw;
#define PMD_INIT_FUNC_TRACE() PMD_LOG_DEBUG(INIT, " >>")
#endif /* __SXE2_COMMON_LOG_H__ */
-
diff --git a/drivers/common/sxe2/sxe2_errno.h b/drivers/common/sxe2/sxe2_errno.h
index 89a715eaef..1257319edf 100644
--- a/drivers/common/sxe2/sxe2_errno.h
+++ b/drivers/common/sxe2/sxe2_errno.h
@@ -50,9 +50,6 @@ enum sxe2_status {
SXE2_ERR_NOLCK = -ENOLCK,
SXE2_ERR_NOSYS = -ENOSYS,
SXE2_ERR_NOTEMPTY = -ENOTEMPTY,
- SXE2_ERR_ILSEQ = -EILSEQ,
- SXE2_ERR_NODATA = -ENODATA,
- SXE2_ERR_CANCELED = -ECANCELED,
SXE2_ERR_TIMEDOUT = -ETIMEDOUT,
SXE2_ERROR = -150,
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
index 4dfc4fd0fa..b9224cf197 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl.c
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -178,13 +178,13 @@ void
goto l_err;
}
- PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=0x%zx, src=0x%"PRIx64", offset=0x%"PRIx64"",
+ PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=%"PRIu64", src=0x%"PRIx64", offset=0x%"PRIx64"",
bar_idx, cmd_fd, len, offset, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
virt = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_SHARED, cmd_fd, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
if (virt == MAP_FAILED) {
- PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=0x%zx, offset=0x%"PRIx64", err:%s",
+ PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=%"PRIu64", offset=0x%"PRIx64", err:%s",
cmd_fd, len, offset, strerror(errno));
goto l_err;
}
@@ -206,12 +206,12 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, u64 len)
goto l_end;
}
- PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%zx",
+ PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%"PRIx64"",
virt, len);
ret = munmap(virt, len);
if (ret < 0) {
- PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=0x%zx, err:%s",
+ PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=%"PRIu64", err:%s",
virt, len, strerror(errno));
ret = -EIO;
goto l_end;
diff --git a/drivers/common/sxe2/sxe2_osal.h b/drivers/common/sxe2/sxe2_osal.h
index e0f4b753b2..20d1accd5f 100644
--- a/drivers/common/sxe2/sxe2_osal.h
+++ b/drivers/common/sxe2/sxe2_osal.h
@@ -29,8 +29,6 @@
#define BIT_ULL(a) (1ULL << (a))
#endif
-#define MIN(a, b) ((a) < (b) ? (a) : (b))
-
#define BITS_PER_BYTE 8
#define IS_UNICAST_ETHER_ADDR(addr) \
@@ -88,7 +86,7 @@
(((n) + (typeof(n))(d) - (typeof(n))1) / (typeof(n))(d))
#endif
-#define usleep_range(min, max) msleep(DIV_ROUND_UP(min, 1000))
+#define usleep_range(min) msleep(DIV_ROUND_UP(min, 1000))
#define __bf_shf(x) ((uint32_t)rte_bsf64(x))
diff --git a/drivers/common/sxe2/sxe2_type.h b/drivers/common/sxe2/sxe2_type.h
index 86923adf6f..e4ef6ed2ce 100644
--- a/drivers/common/sxe2/sxe2_type.h
+++ b/drivers/common/sxe2/sxe2_type.h
@@ -8,7 +8,6 @@
#include <sys/time.h>
#include <stdlib.h>
-#include <stdio.h>
#include <errno.h>
#include <stdarg.h>
#include <unistd.h>
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 8638244d80..b348dd71a1 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -18,6 +18,8 @@ sources += files(
'sxe2_queue.c',
'sxe2_tx.c',
'sxe2_rx.c',
+ 'sxe2_txrx_poll.c',
+ 'sxe2_txrx.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index bb51b9fb71..38e3967c56 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -27,6 +27,7 @@
#include "sxe2_cmd_chnl.h"
#include "sxe2_tx.h"
#include "sxe2_rx.h"
+#include "sxe2_txrx.h"
#include "sxe2_common.h"
#include "sxe2_common_log.h"
#include "sxe2_host_regs.h"
@@ -132,6 +133,9 @@ static s32 sxe2_dev_start(struct rte_eth_dev *dev)
goto l_end;
}
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
+
ret = sxe2_queues_start(dev);
if (ret) {
PMD_LOG_ERR(INIT, "enable queues failed");
@@ -349,8 +353,8 @@ void __iomem *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
for (i = 0; i < bar_info->map_cnt; i++) {
seg_info = &bar_info->seg_info[i];
if (res_type == seg_info->type) {
- addr = (void __iomem *)((uintptr_t)seg_info->addr +
- seg_info->page_inner_offset + reg_width * idx_in_func);
+ addr = (uint8_t __iomem *)seg_info->addr +
+ seg_info->page_inner_offset + reg_width * idx_in_func;
goto l_end;
}
}
@@ -461,8 +465,9 @@ s32 sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
map_addr = sxe2_drv_dev_mmap(adapter->cdev, bar_info->bar_idx, aligned_len, aligned_offset);
if (!map_addr) {
- PMD_LOG_ERR(INIT, "Failed to mmap BAR space, type=%d, len=%zu, page_size=%zu",
- res_type, org_len, page_size);
+ PMD_LOG_ERR(INIT, "Failed to mmap BAR space, type=%d, len=%" PRIu64
+ ", offset=%" PRIu64 ", page_size=%zu",
+ res_type, org_len, org_offset, page_size);
ret = -EFAULT;
goto l_end;
}
@@ -746,10 +751,17 @@ static s32 sxe2_dev_init(struct rte_eth_dev *dev, struct sxe2_dev_kvargs_info *k
PMD_INIT_FUNC_TRACE();
+ sxe2_set_common_function(dev);
+
dev->dev_ops = &sxe2_eth_dev_ops;
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
+ if (ret != SXE2_SUCCESS)
+ PMD_LOG_ERR(INIT, "Failed to mp init (secondary), ret=%d", ret);
goto l_end;
+ }
ret = sxe2_hw_init(dev);
if (ret) {
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
new file mode 100644
index 0000000000..a7b94e8967
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -0,0 +1,247 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include <unistd.h>
+
+#include "sxe2_txrx.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_ethdev.h"
+
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+#include "sxe2_osal.h"
+#include "sxe2_cmd_chnl.h"
+#if defined(RTE_ARCH_ARM64)
+#include <rte_cpuflags.h>
+#endif
+
+static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ s32 ret;
+ u16 desc_idx;
+
+ if (unlikely(offset >= txq->ring_depth)) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+ desc_idx = txq->next_use + offset;
+ desc_idx = DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
+ if (desc_idx >= txq->ring_depth) {
+ desc_idx -= txq->ring_depth;
+ if (desc_idx >= txq->ring_depth)
+ desc_idx -= txq->ring_depth;
+ }
+
+ if (desc_idx == 0)
+ desc_idx = txq->rs_thresh - 1;
+ else
+ desc_idx -= 1;
+
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
+ (txq->desc_ring[desc_idx].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
+ ret = RTE_ETH_TX_DESC_DONE;
+ else
+ ret = RTE_ETH_TX_DESC_FULL;
+
+l_end:
+ return ret;
+}
+
+static inline s32 sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
+{
+ struct rte_mbuf *m_seg = mbuf;
+
+ while (m_seg != NULL) {
+ if (m_seg->data_len == 0)
+ return SXE2_ERR_INVAL;
+ m_seg = m_seg->next;
+ }
+
+ return SXE2_SUCCESS;
+}
+
+u16 sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct rte_mbuf *mbuf;
+ u64 ol_flags = 0;
+ s32 ret = SXE2_SUCCESS;
+ s32 i = 0;
+
+ for (i = 0; i < nb_pkts; i++) {
+ mbuf = tx_pkts[i];
+ if (!mbuf)
+ continue;
+ ol_flags = mbuf->ol_flags;
+ if (!(ol_flags & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))) {
+ if (mbuf->nb_segs > SXE2_TX_MTU_SEG_MAX ||
+ mbuf->pkt_len > SXE2_FRAME_SIZE_MAX) {
+ rte_errno = -SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ } else if ((mbuf->tso_segsz < SXE2_MIN_TSO_MSS) ||
+ (mbuf->tso_segsz > SXE2_MAX_TSO_MSS) ||
+ (mbuf->nb_segs > txq->ring_depth) ||
+ (mbuf->pkt_len > SXE2_TX_TSO_PKTLEN_MAX)) {
+ rte_errno = -SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+ if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
+ rte_errno = -SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_TX
+ ret = rte_validate_tx_offload(mbuf);
+ if (ret != SXE2_SUCCESS) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+#endif
+ ret = rte_net_intel_cksum_prepare(mbuf);
+ if (ret != SXE2_SUCCESS) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+
+ ret = sxe2_tx_mbuf_empty_check(mbuf);
+ if (ret != SXE2_SUCCESS) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return i;
+}
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ u32 tx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
+ PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
+ tx_mode_flags, dev->data->port_id);
+}
+
+static s32 sxe2_rx_desciptor_status(void *rx_queue, u16 offset)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ s32 ret;
+
+ if (unlikely(offset >= rxq->ring_depth)) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+
+ if (offset >= rxq->ring_depth - rxq->hold_num) {
+ ret = RTE_ETH_RX_DESC_UNAVAIL;
+ goto l_end;
+ }
+
+ if (rxq->processing_idx + offset >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
+ else
+ desc = &rxq->desc_ring[rxq->processing_idx + offset];
+
+ if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
+ ret = RTE_ETH_RX_DESC_DONE;
+ else
+ ret = RTE_ETH_RX_DESC_AVAIL;
+
+l_end:
+ PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
+ offset, ret, rxq->queue_id, rxq->port_id);
+ return ret;
+}
+
+static s32 sxe2_rx_queue_count(void *rx_queue)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ u16 done_num = 0;
+
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ while ((done_num < rxq->ring_depth) &&
+ (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK)) {
+ done_num += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ if (rxq->processing_idx + done_num >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + done_num - rxq->ring_depth];
+ else
+ desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ }
+
+ PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
+ done_num, rxq->queue_id, rxq->port_id);
+
+ return done_num;
+}
+
+static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ u16 i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (0 != (rxq->offloads & offload)) {
+ en = true;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return en;
+}
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ u32 rx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
+ else
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
+
+ PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
+ rx_mode_flags, dev->data->port_id);
+ adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
+}
+
+void sxe2_set_common_function(struct rte_eth_dev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+
+ dev->rx_queue_count = sxe2_rx_queue_count;
+ dev->rx_descriptor_status = sxe2_rx_desciptor_status;
+
+ dev->tx_descriptor_status = sxe2_tx_desciptor_status;
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
new file mode 100644
index 0000000000..e6f671e3dc
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef SXE2_TXRX_H
+#define SXE2_TXRX_H
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+
+void sxe2_set_common_function(struct rte_eth_dev *dev);
+
+u16 sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts);
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
+
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.c b/drivers/net/sxe2/sxe2_txrx_poll.c
new file mode 100644
index 0000000000..02533abfd5
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_poll.c
@@ -0,0 +1,945 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include <unistd.h>
+
+#include "sxe2_osal.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_txrx.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+
+static __rte_always_inline s32
+sxe2_tx_bufs_free(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX];
+ s32 ret;
+ u32 i;
+ u16 rs_thresh;
+ u16 free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - rs_thresh + 1];
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {
+ if (likely(rs_thresh <= SXE2_TX_FREE_BUFFER_SIZE_MAX)) {
+ mbuf = buffer[0].mbuf;
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = buffer[i].mbuf;
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ rte_mempool_put(buffer->mbuf->pool, buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline s32 sxe2_tx_cleanup(struct sxe2_tx_queue *txq)
+{
+ s32 ret = SXE2_SUCCESS;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ struct sxe2_tx_buffer *buffer_ring = txq->buffer_ring;
+ u16 ring_depth = txq->ring_depth;
+ u16 next_clean = txq->next_clean;
+ u16 clean_last;
+ u16 clean_num;
+
+ clean_last = next_clean + txq->rs_thresh;
+ if (clean_last >= ring_depth)
+ clean_last = clean_last - ring_depth;
+
+ clean_last = buffer_ring[clean_last].last_id;
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) !=
+ (txq->desc_ring[clean_last].wb.dd & rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK))) {
+ PMD_LOG_DEBUG(TX, "desc[%u] is not done.port_id=%u queue_id=%u val=0x%" PRIx64,
+ clean_last, txq->port_id,
+ txq->queue_id, txq->desc_ring[clean_last].wb.dd);
+ ret = SXE2_ERR_DESC_NO_DONE;
+ goto l_end;
+ }
+
+ if (clean_last > next_clean)
+ clean_num = clean_last - next_clean;
+ else
+ clean_num = ring_depth - next_clean + clean_last;
+
+ desc_ring[clean_last].wb.dd = 0;
+
+ txq->next_clean = clean_last;
+ txq->desc_free_num += clean_num;
+
+ ret = SXE2_SUCCESS;
+
+l_end:
+ return ret;
+}
+
+static __rte_always_inline u16
+sxe2_tx_pkt_data_desc_count(struct rte_mbuf *tx_pkt)
+{
+ struct rte_mbuf *m_seg = tx_pkt;
+ u16 count = 0;
+
+ while (m_seg != NULL) {
+ count += DIV_ROUND_UP(m_seg->data_len,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC);
+ m_seg = m_seg->next;
+ }
+
+ return count;
+}
+
+static __rte_always_inline void
+sxe2_tx_desc_checksum_fill(u64 offloads, u32 *desc_cmd, u32 *desc_offset,
+ union sxe2_tx_offload_info ol_info)
+{
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ }
+
+ if (offloads & RTE_MBUF_F_TX_TCP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_UDP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ default:
+
+ break;
+ }
+
+l_end:
+ return;
+}
+
+static __rte_always_inline u64
+sxe2_tx_data_desc_build_cobt(u32 cmd, u32 offset, u16 buf_size, u16 l2tag)
+{
+ return rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DATA |
+ (((u64)cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT) |
+ (((u64)offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT) |
+ (((u64)buf_size) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT) |
+ (((u64)l2tag) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT));
+}
+
+u16 sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct sxe2_tx_buffer *buffer_ring;
+ struct sxe2_tx_buffer *buffer;
+ struct sxe2_tx_buffer *next_buffer;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ volatile union sxe2_tx_data_desc *desc_ring;
+ volatile union sxe2_tx_data_desc *desc;
+ volatile struct sxe2_tx_context_desc *ctxt_desc;
+ union sxe2_tx_offload_info ol_info;
+ struct sxe2_vsi *vsi = txq->vsi;
+ rte_iova_t buf_dma_addr;
+ u64 offloads;
+ u64 desc_type_cmd_tso_mss;
+ u32 desc_cmd;
+ u32 desc_offset;
+ u32 desc_tag;
+ u32 desc_tunneling_params;
+ u16 ipsec_offset;
+ u16 ctxt_desc_num;
+ u16 desc_sum_num;
+ u16 tx_num;
+ u16 seg_len;
+ u16 next_use;
+ u16 last_use;
+ u16 desc_l2tag2;
+
+ buffer_ring = txq->buffer_ring;
+ desc_ring = txq->desc_ring;
+ next_use = txq->next_use;
+ buffer = &buffer_ring[next_use];
+
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_cleanup(txq);
+
+ for (tx_num = 0; tx_num < nb_pkts; tx_num++) {
+ tx_pkt = *tx_pkts++;
+ desc_cmd = 0;
+ desc_offset = 0;
+ desc_tag = 0;
+ desc_tunneling_params = 0;
+ ipsec_offset = 0;
+ offloads = tx_pkt->ol_flags;
+ ol_info.l2_len = tx_pkt->l2_len;
+ ol_info.l3_len = tx_pkt->l3_len;
+ ol_info.l4_len = tx_pkt->l4_len;
+ ol_info.tso_segsz = tx_pkt->tso_segsz;
+ ol_info.outer_l2_len = tx_pkt->outer_l2_len;
+ ol_info.outer_l3_len = tx_pkt->outer_l3_len;
+
+ ctxt_desc_num = (offloads &
+ SXE2_TX_OFFLOAD_CTXT_NEEDCK_MASK) ? 1 : 0;
+ if (unlikely(vsi->vsi_type == SXE2_VSI_T_DPDK_ESW))
+ ctxt_desc_num = 1;
+
+ if (offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))
+ desc_sum_num = sxe2_tx_pkt_data_desc_count(tx_pkt) + ctxt_desc_num;
+ else
+ desc_sum_num = tx_pkt->nb_segs + ctxt_desc_num;
+
+ last_use = next_use + desc_sum_num - 1;
+ if (last_use >= txq->ring_depth)
+ last_use = last_use - txq->ring_depth;
+
+ if (desc_sum_num > txq->desc_free_num) {
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_exit_logic;
+
+ if (unlikely(desc_sum_num > txq->rs_thresh)) {
+ while (desc_sum_num > txq->desc_free_num)
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_exit_logic;
+ }
+ }
+
+ desc_offset |= SXE2_TX_DATA_DESC_MACLEN_VAL(ol_info.l2_len);
+
+ if (offloads & SXE2_TX_OFFLOAD_CKSUM_MASK) {
+ sxe2_tx_desc_checksum_fill(offloads, &desc_cmd,
+ &desc_offset, ol_info);
+ }
+
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ desc_tag = tx_pkt->vlan_tci;
+ }
+
+ if (ctxt_desc_num) {
+ ctxt_desc = (volatile struct sxe2_tx_context_desc *)
+ &desc_ring[next_use];
+ desc_l2tag2 = 0;
+ desc_type_cmd_tso_mss = SXE2_TX_DESC_DTYPE_CTXT;
+
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_QINQ) {
+ desc_l2tag2 = tx_pkt->vlan_tci_outer;
+ desc_type_cmd_tso_mss |= SXE2_TX_CTXT_DESC_CMD_IL2TAG2_MASK;
+ }
+
+ ctxt_desc->tunneling_params =
+ rte_cpu_to_le_32(desc_tunneling_params);
+ ctxt_desc->l2tag2 = rte_cpu_to_le_16(desc_l2tag2);
+ ctxt_desc->type_cmd_tso_mss = rte_cpu_to_le_64(desc_type_cmd_tso_mss);
+ ctxt_desc->ipsec_offset = rte_cpu_to_le_64(ipsec_offset);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ }
+
+ m_seg = tx_pkt;
+
+ do {
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ buffer->mbuf = m_seg;
+ seg_len = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ while ((offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG)) &&
+ unlikely(seg_len > SXE2_TX_MAX_DATA_NUM_PER_DESC)) {
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd, desc_offset,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC,
+ desc_tag);
+ buf_dma_addr += SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ seg_len -= SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ }
+
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd,
+ desc_offset, seg_len, desc_tag);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_EOP;
+ txq->desc_used_num += desc_sum_num;
+ txq->desc_free_num -= desc_sum_num;
+
+ if (txq->desc_used_num >= txq->rs_thresh) {
+ PMD_LOG_DEBUG(TX, "Tx pkts set RS bit."
+ "last_use=%u port_id=%u, queue_id=%u",
+ last_use, txq->port_id, txq->queue_id);
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_RS;
+ txq->desc_used_num = 0;
+ }
+
+ desc->read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(((u64)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT);
+ }
+
+l_exit_logic:
+ if (tx_num == 0)
+ goto l_end;
+ goto l_end_of_tx;
+l_end_of_tx:
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, tx_num);
+
+ txq->next_use = next_use;
+
+l_end:
+ return tx_num;
+}
+
+static __rte_always_inline void
+sxe2_tx_data_desc_fill(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ u32 desc_offset;
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset, (*tx_pkts)->data_len, 0);
+}
+static __rte_always_inline void
+sxe2_tx_data_desc_fill_batch(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ u32 i;
+ u32 desc_offset;
+ for (i = 0; i < SXE2_TX_FILL_PER_LOOP; ++i, ++desc, ++tx_pkts) {
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset,
+ (*tx_pkts)->data_len,
+ 0);
+ }
+}
+
+static inline void sxe2_tx_ring_fill(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_buffer *buffer = &txq->buffer_ring[txq->next_use];
+ volatile union sxe2_tx_data_desc *desc = &txq->desc_ring[txq->next_use];
+ u32 i, j;
+ u32 mainpart;
+ u32 leftover;
+ mainpart = nb_pkts & ((u32)~SXE2_TX_FILL_PER_LOOP_MASK);
+ leftover = nb_pkts & ((u32)SXE2_TX_FILL_PER_LOOP_MASK);
+ for (i = 0; i < mainpart; i += SXE2_TX_FILL_PER_LOOP) {
+ for (j = 0; j < SXE2_TX_FILL_PER_LOOP; ++j)
+ (buffer + i + j)->mbuf = *(tx_pkts + i + j);
+ sxe2_tx_data_desc_fill_batch(desc + i, tx_pkts + i);
+ }
+ if (unlikely(leftover > 0)) {
+ for (i = 0; i < leftover; ++i) {
+ (buffer + mainpart + i)->mbuf = *(tx_pkts + mainpart + i);
+ sxe2_tx_data_desc_fill(desc + mainpart + i,
+ tx_pkts + mainpart + i);
+ }
+ }
+}
+
+static inline u16 sxe2_tx_pkts_batch(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ u16 res_num = 0;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ txq->desc_free_num -= nb_pkts;
+ if ((txq->next_use + nb_pkts) > txq->ring_depth) {
+ res_num = txq->ring_depth - txq->next_use;
+ sxe2_tx_ring_fill(txq, tx_pkts, res_num);
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs = txq->rs_thresh - 1;
+ txq->next_use = 0;
+ }
+ sxe2_tx_ring_fill(txq, tx_pkts + res_num, nb_pkts - res_num);
+ txq->next_use = txq->next_use + (nb_pkts - res_num);
+ if (txq->next_use > txq->next_rs) {
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ if (txq->next_rs >= txq->ring_depth)
+ txq->next_rs = txq->rs_thresh - 1;
+ }
+ if (txq->next_use >= txq->ring_depth)
+ txq->next_use = 0;
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, txq->next_use, nb_pkts);
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, txq->next_use);
+l_end:
+ return nb_pkts;
+}
+
+u16 sxe2_tx_pkts_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ u16 tx_done_num;
+ u16 tx_once_num;
+ u16 tx_need_num;
+ if (likely(nb_pkts <= SXE2_TX_PKTS_BURST_BATCH_NUM)) {
+ tx_done_num = sxe2_tx_pkts_batch(tx_queue,
+ tx_pkts, nb_pkts);
+ goto l_end;
+ }
+ tx_done_num = 0;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, SXE2_TX_PKTS_BURST_BATCH_NUM);
+ tx_once_num = sxe2_tx_pkts_batch(tx_queue,
+ &tx_pkts[tx_done_num], tx_need_num);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+l_end:
+ return tx_done_num;
+}
+
+static inline void
+sxe2_update_rx_tail(struct sxe2_rx_queue *rxq, u16 hold_num, u16 rx_id)
+{
+ hold_num += rxq->hold_num;
+
+ if (hold_num > rxq->rx_free_thresh) {
+ rx_id = (u16)((rx_id == 0) ? (rxq->ring_depth - 1) : (rx_id - 1));
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, rx_id);
+ hold_num = 0;
+ }
+ rxq->hold_num = hold_num;
+}
+
+static inline u64
+sxe2_rx_desc_error_para(__rte_unused struct sxe2_rx_queue *rxq,
+ union sxe2_rx_desc *desc)
+{
+ u64 flags = 0;
+ u64 desc_qw1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (unlikely(0 == (desc_qw1 & SXE2_RX_DESC_STATUS_L3L4_P_MASK)))
+ goto l_end;
+
+ if (likely(0 == (desc->wb.rxdid_src & SXE2_RX_DESC_EUDPE_MASK)))
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+ else
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_QW1_ERRORS_MASK))) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD);
+ goto l_end;
+ }
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_IPE_MASK)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_L4_MASK)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+
+ if (unlikely(0 != (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_EIP_MASK)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+l_end:
+ return flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_mbuf_common_fields_fill(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ u32 *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ u64 qword1;
+ u64 pkt_flags;
+ qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+
+ mbuf->ol_flags = 0;
+ mbuf->packet_type = ptype_tbl[SXE2_RX_DESC_PTYPE_VAL_GET(qword1)];
+
+ pkt_flags = sxe2_rx_desc_error_para(rxq, rxd);
+
+ mbuf->ol_flags |= pkt_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_sw_stats_update(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ u64 qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ switch (SXE2_RX_DESC_STATUS_UMBCAST_VAL_GET(qword1)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+}
+
+u16 sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ u64 qword1;
+ u16 done_num;
+ u16 hold_num;
+ u16 cur_idx;
+ u16 pkt_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ PMD_LOG_INFO(RX, "Rx new_mbuf alloc failed port_id:%u "
+ "queue_id:%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+
+ rte_prefetch0(buffer_ring[cur_idx]);
+
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+
+ cur_mbuf = *cur_buffer;
+
+ *cur_buffer = new_mbuf;
+
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+
+ if (0 == (qword1 & SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len + pkt_len -
+ RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+ first_seg->port = rxq->port_id;
+
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
+
+u16 sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *cur_mbuf_pay;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *new_mbuf_pay = NULL;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ u64 qword1;
+ u16 done_num;
+ u16 hold_num;
+ u16 cur_idx;
+ u16 pkt_len;
+ u16 hdr_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ new_mbuf = NULL;
+
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) == 0 ||
+ first_seg == NULL) {
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ break;
+ }
+ }
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) {
+ new_mbuf_pay = rte_mbuf_raw_alloc(rxq->rx_seg[1].mp);
+ if (unlikely(new_mbuf_pay == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ if (new_mbuf != NULL)
+ rte_pktmbuf_free(new_mbuf);
+ new_mbuf = NULL;
+ break;
+ }
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+ rte_prefetch0(buffer_ring[cur_idx]);
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+ cur_mbuf = *cur_buffer;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ *cur_buffer = new_mbuf;
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ } else {
+ if (first_seg == NULL) {
+ *cur_buffer = new_mbuf;
+ new_mbuf->next = new_mbuf_pay;
+ new_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ } else {
+ cur_mbuf_pay = cur_mbuf->next;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ cur_mbuf->next = new_mbuf_pay;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(cur_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ cur_mbuf = cur_mbuf_pay;
+ }
+ }
+ new_mbuf = NULL;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ } else {
+ if (first_seg == NULL) {
+ cur_mbuf->nb_segs = 2;
+ cur_mbuf->next->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ hdr_len = SXE2_RX_DESC_HDR_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = hdr_len;
+ cur_mbuf->pkt_len = hdr_len + pkt_len;
+ cur_mbuf->next->data_len = pkt_len;
+ first_seg = cur_mbuf;
+ cur_mbuf = cur_mbuf->next;
+ last_seg = cur_mbuf;
+ } else {
+ cur_mbuf->nb_segs = 1;
+ cur_mbuf->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_RX
+
+ rte_pktmbuf_dump(stdout, first_seg, rte_pktmbuf_pkt_len(first_seg));
+#endif
+
+ if (0 == (rte_le_to_cpu_64(desc_tmp.wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len +
+ pkt_len - RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ first_seg->port = rxq->port_id;
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
* [PATCH v12 00/10] net/sxe2: fix logic errors and address feedback
From: liujie5 @ 2026-05-12 8:06 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260506113557.1151250-11-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch set addresses the feedback received on the v10 submission
for the sxe2 PMD. The primary focus is on fixing vector path selection,
ensuring memory safety during mbuf initialization, and cleaning up
redundant logic in the configuration functions.
v12 Changes:
- Fixed vector Rx burst function being overwritten by scalar selection.
- Refactored Rx/Tx mode set functions to seed flags from caps first,eliminating tautological checks.
- Added memset for mbuf_def in vector init to avoid uninitialized reads.
- Converted pci_map_addr_info to designated initializers.
- Removed dead Windows-only code in meson.build.
- Added NULL checks for mbuf free for driver-wide consistency.
- Updated burst_mode_get to accurately report AVX paths.
- Adjusted SXE2_ETH_OVERHEAD to match actual VLAN capabilities.
Jie Liu (10):
mailmap: add Jie Liu
doc: add sxe2 guide and release notes
common/sxe2: add sxe2 basic structures
drivers: add base driver skeleton
drivers: add base driver probe skeleton
drivers: support PCI BAR mapping
common/sxe2: add ioctl interface for DMA map and unmap
net/sxe2: support queue setup and control
drivers: add data path for Rx and Tx
net/sxe2: add vectorized Rx and Tx
.mailmap | 1 +
doc/guides/nics/features/sxe2.ini | 30 +
doc/guides/nics/index.rst | 1 +
doc/guides/nics/sxe2.rst | 34 +
doc/guides/rel_notes/release_26_07.rst | 4 +
drivers/common/sxe2/meson.build | 15 +
drivers/common/sxe2/sxe2_common.c | 685 +++++++++++++++
drivers/common/sxe2/sxe2_common.h | 86 ++
drivers/common/sxe2/sxe2_common_log.h | 83 ++
drivers/common/sxe2/sxe2_errno.h | 110 +++
drivers/common/sxe2/sxe2_host_regs.h | 707 +++++++++++++++
drivers/common/sxe2/sxe2_internal_ver.h | 33 +
drivers/common/sxe2/sxe2_ioctl_chnl.c | 326 +++++++
drivers/common/sxe2/sxe2_ioctl_chnl.h | 141 +++
drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 63 ++
drivers/common/sxe2/sxe2_osal.h | 584 +++++++++++++
drivers/common/sxe2/sxe2_type.h | 60 ++
drivers/meson.build | 1 +
drivers/net/meson.build | 1 +
drivers/net/sxe2/meson.build | 32 +
drivers/net/sxe2/sxe2_cmd_chnl.c | 319 +++++++
drivers/net/sxe2/sxe2_cmd_chnl.h | 33 +
drivers/net/sxe2/sxe2_drv_cmd.h | 389 +++++++++
drivers/net/sxe2/sxe2_ethdev.c | 942 ++++++++++++++++++++
drivers/net/sxe2/sxe2_ethdev.h | 315 +++++++
drivers/net/sxe2/sxe2_irq.h | 49 ++
drivers/net/sxe2/sxe2_queue.c | 67 ++
drivers/net/sxe2/sxe2_queue.h | 194 +++++
drivers/net/sxe2/sxe2_rx.c | 579 +++++++++++++
drivers/net/sxe2/sxe2_rx.h | 34 +
drivers/net/sxe2/sxe2_tx.c | 447 ++++++++++
drivers/net/sxe2/sxe2_tx.h | 32 +
drivers/net/sxe2/sxe2_txrx.c | 372 ++++++++
drivers/net/sxe2/sxe2_txrx.h | 22 +
drivers/net/sxe2/sxe2_txrx_common.h | 541 ++++++++++++
drivers/net/sxe2/sxe2_txrx_poll.c | 945 +++++++++++++++++++++
drivers/net/sxe2/sxe2_txrx_poll.h | 17 +
drivers/net/sxe2/sxe2_txrx_vec.c | 197 +++++
drivers/net/sxe2/sxe2_txrx_vec.h | 72 ++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 235 +++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 545 ++++++++++++
drivers/net/sxe2/sxe2_vsi.c | 212 +++++
drivers/net/sxe2/sxe2_vsi.h | 205 +++++
43 files changed, 9760 insertions(+)
create mode 100644 doc/guides/nics/features/sxe2.ini
create mode 100644 doc/guides/nics/sxe2.rst
create mode 100644 drivers/common/sxe2/meson.build
create mode 100644 drivers/common/sxe2/sxe2_common.c
create mode 100644 drivers/common/sxe2/sxe2_common.h
create mode 100644 drivers/common/sxe2/sxe2_common_log.h
create mode 100644 drivers/common/sxe2/sxe2_errno.h
create mode 100644 drivers/common/sxe2/sxe2_host_regs.h
create mode 100644 drivers/common/sxe2/sxe2_internal_ver.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.c
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl_func.h
create mode 100644 drivers/common/sxe2/sxe2_osal.h
create mode 100644 drivers/common/sxe2/sxe2_type.h
create mode 100644 drivers/net/sxe2/meson.build
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.c
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.h
create mode 100644 drivers/net/sxe2/sxe2_drv_cmd.h
create mode 100644 drivers/net/sxe2/sxe2_ethdev.c
create mode 100644 drivers/net/sxe2/sxe2_ethdev.h
create mode 100644 drivers/net/sxe2/sxe2_irq.h
create mode 100644 drivers/net/sxe2/sxe2_queue.c
create mode 100644 drivers/net/sxe2/sxe2_queue.h
create mode 100644 drivers/net/sxe2/sxe2_rx.c
create mode 100644 drivers/net/sxe2/sxe2_rx.h
create mode 100644 drivers/net/sxe2/sxe2_tx.c
create mode 100644 drivers/net/sxe2/sxe2_tx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx.c
create mode 100644 drivers/net/sxe2/sxe2_txrx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_sse.c
create mode 100644 drivers/net/sxe2/sxe2_vsi.c
create mode 100644 drivers/net/sxe2/sxe2_vsi.h
--
2.47.3
^ permalink raw reply
* [PATCH v12 10/10] net/sxe2: add vectorized Rx and Tx
From: liujie5 @ 2026-05-12 8:06 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260512080616.454171-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch implements the vectorized data path for the sxe2 PMD.
It utilizes SIMD instructions (e.g., SSE) to process multiple
packets simultaneously, significantly improving throughput for
small packet processing.
The implementation includes:
* Vectorized Rx burst function for bulk descriptor processing.
* Vectorized Tx burst function with optimized resource cleanup.
* Capability flags update to reflect vectorized path support.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/meson.build | 7 +
drivers/net/sxe2/sxe2_ethdev.c | 35 +-
drivers/net/sxe2/sxe2_ethdev.h | 1 -
drivers/net/sxe2/sxe2_queue.c | 28 ++
drivers/net/sxe2/sxe2_queue.h | 3 +
drivers/net/sxe2/sxe2_txrx.c | 221 +++++++---
drivers/net/sxe2/sxe2_txrx.h | 11 +-
drivers/net/sxe2/sxe2_txrx_poll.h | 3 +-
drivers/net/sxe2/sxe2_txrx_vec.c | 197 +++++++++
drivers/net/sxe2/sxe2_txrx_vec.h | 72 ++++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 235 ++++++++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 545 ++++++++++++++++++++++++
12 files changed, 1276 insertions(+), 82 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_sse.c
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index b348dd71a1..3df57aee8c 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -11,6 +11,12 @@ cflags += ['-g']
deps += ['common_sxe2', 'hash','cryptodev','security']
+includes += include_directories('../../common/sxe2')
+
+if arch_subdir == 'x86'
+ sources += files('sxe2_txrx_vec_sse.c')
+endif
+
sources += files(
'sxe2_ethdev.c',
'sxe2_cmd_chnl.c',
@@ -20,6 +26,7 @@ sources += files(
'sxe2_rx.c',
'sxe2_txrx_poll.c',
'sxe2_txrx.c',
+ 'sxe2_txrx_vec.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 38e3967c56..dc4a33901d 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -59,17 +59,11 @@ static const struct rte_pci_id pci_id_sxe2_tbl[] = {
};
static struct sxe2_pci_map_addr_info sxe2_net_map_addr_info_pf[SXE2_PCI_MAP_RES_MAX_COUNT] = {
- /* SXE2_PCI_MAP_RES_INVALID */
{0, 0, 0},
- /* SXE2_PCI_MAP_RES_DOORBELL_TX */
{ SXE2_TXQ_LEGACY_DBLL(0), 0, 4},
- /* SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL */
{ SXE2_RXQ_TAIL(0), 0, 4},
- /* SXE2_PCI_MAP_RES_IRQ_DYN */
{ SXE2_VF_DYN_CTL(0), 0, 4},
- /* SXE2_PCI_MAP_RES_IRQ_ITR(默认使用ITR0) */
{ SXE2_VF_INT_ITR(0, 0), 0, 4},
- /* SXE2_PCI_MAP_RES_IRQ_MSIX */
{ SXE2_BAR4_MSIX_CTL(0), 4, 0x10},
};
@@ -102,25 +96,6 @@ static s32 sxe2_dev_stop(struct rte_eth_dev *dev)
return ret;
}
-static s32 sxe2_queues_start(struct rte_eth_dev *dev)
-{
- s32 ret = SXE2_SUCCESS;
- ret = sxe2_txqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start tx queue.");
- goto l_end;
- }
-
- ret = sxe2_rxqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start rx queue.");
- sxe2_txqs_all_stop(dev);
- }
-
-l_end:
- return ret;
-}
-
static s32 sxe2_dev_start(struct rte_eth_dev *dev)
{
s32 ret = SXE2_SUCCESS;
@@ -153,7 +128,7 @@ static s32 sxe2_dev_start(struct rte_eth_dev *dev)
static s32 sxe2_dev_close(struct rte_eth_dev *dev)
{
(void)sxe2_dev_stop(dev);
-
+ (void)sxe2_queues_release(dev);
sxe2_vsi_uninit(dev);
sxe2_dev_pci_map_uinit(dev);
@@ -291,13 +266,19 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_close = sxe2_dev_close,
.dev_infos_get = sxe2_dev_infos_get,
+ .rx_queue_start = sxe2_rx_queue_start,
+ .rx_queue_stop = sxe2_rx_queue_stop,
+ .tx_queue_start = sxe2_tx_queue_start,
+ .tx_queue_stop = sxe2_tx_queue_stop,
.rx_queue_setup = sxe2_rx_queue_setup,
- .tx_queue_setup = sxe2_tx_queue_setup,
.rx_queue_release = sxe2_rx_queue_release,
+ .tx_queue_setup = sxe2_tx_queue_setup,
.tx_queue_release = sxe2_tx_queue_release,
.rxq_info_get = sxe2_rx_queue_info_get,
.txq_info_get = sxe2_tx_queue_info_get,
+ .rx_burst_mode_get = sxe2_rx_burst_mode_get,
+ .tx_burst_mode_get = sxe2_tx_burst_mode_get,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
diff --git a/drivers/net/sxe2/sxe2_ethdev.h b/drivers/net/sxe2/sxe2_ethdev.h
index 4ef7854479..43148f9b03 100644
--- a/drivers/net/sxe2/sxe2_ethdev.h
+++ b/drivers/net/sxe2/sxe2_ethdev.h
@@ -11,7 +11,6 @@
#include <rte_tm_driver.h>
#include <rte_io.h>
-#include "sxe2_common.h"
#include "sxe2_errno.h"
#include "sxe2_type.h"
#include "sxe2_vsi.h"
diff --git a/drivers/net/sxe2/sxe2_queue.c b/drivers/net/sxe2/sxe2_queue.c
index 98343679f6..b1860490aa 100644
--- a/drivers/net/sxe2/sxe2_queue.c
+++ b/drivers/net/sxe2/sxe2_queue.c
@@ -6,6 +6,8 @@
#include "sxe2_queue.h"
#include "sxe2_common_log.h"
#include "sxe2_errno.h"
+#include "sxe2_tx.h"
+#include "sxe2_rx.h"
void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
struct sxe2_drv_queue_caps *q_caps)
@@ -37,3 +39,29 @@ s32 sxe2_queues_init(struct rte_eth_dev *dev)
return ret;
}
+
+s32 sxe2_queues_start(struct rte_eth_dev *dev)
+{
+ s32 ret = SXE2_SUCCESS;
+
+ ret = sxe2_txqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start tx queue.");
+ goto l_end;
+ }
+
+ ret = sxe2_rxqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start rx queue.");
+ sxe2_txqs_all_stop(dev);
+ }
+l_end:
+ return ret;
+}
+
+void sxe2_queues_release(struct rte_eth_dev *dev)
+{
+ sxe2_all_rxqs_release(dev);
+
+ sxe2_all_txqs_release(dev);
+}
diff --git a/drivers/net/sxe2/sxe2_queue.h b/drivers/net/sxe2/sxe2_queue.h
index 7fa22e2820..93402186c7 100644
--- a/drivers/net/sxe2/sxe2_queue.h
+++ b/drivers/net/sxe2/sxe2_queue.h
@@ -188,4 +188,7 @@ void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
s32 sxe2_queues_init(struct rte_eth_dev *dev);
+s32 sxe2_queues_start(struct rte_eth_dev *dev);
+
+void sxe2_queues_release(struct rte_eth_dev *dev);
#endif
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
index a7b94e8967..d81f0c8c98 100644
--- a/drivers/net/sxe2/sxe2_txrx.c
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -9,12 +9,11 @@
#include <rte_memzone.h>
#include <ethdev_driver.h>
#include <unistd.h>
-
#include "sxe2_txrx.h"
#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_vec.h"
#include "sxe2_txrx_poll.h"
#include "sxe2_ethdev.h"
-
#include "sxe2_common_log.h"
#include "sxe2_errno.h"
#include "sxe2_osal.h"
@@ -22,18 +21,38 @@
#if defined(RTE_ARCH_ARM64)
#include <rte_cpuflags.h>
#endif
-
+s32 __rte_cold
+sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ u32 *batch_flags)
+{
+ struct sxe2_tx_queue *txq;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ if (txq->offloads != (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) ||
+ txq->rs_thresh < SXE2_TX_PKTS_BURST_BATCH_NUM) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ }
+ *batch_flags = SXE2_TX_MODE_SIMPLE_BATCH;
+l_end:
+ return ret;
+}
static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
{
struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
s32 ret;
u16 desc_idx;
-
if (unlikely(offset >= txq->ring_depth)) {
ret = SXE2_ERR_INVAL;
goto l_end;
}
-
desc_idx = txq->next_use + offset;
desc_idx = DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
if (desc_idx >= txq->ring_depth) {
@@ -41,19 +60,16 @@ static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
if (desc_idx >= txq->ring_depth)
desc_idx -= txq->ring_depth;
}
-
if (desc_idx == 0)
desc_idx = txq->rs_thresh - 1;
else
desc_idx -= 1;
-
if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
(txq->desc_ring[desc_idx].wb.dd &
rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
ret = RTE_ETH_TX_DESC_DONE;
else
ret = RTE_ETH_TX_DESC_FULL;
-
l_end:
return ret;
}
@@ -61,13 +77,11 @@ static s32 sxe2_tx_desciptor_status(void *tx_queue, u16 offset)
static inline s32 sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
{
struct rte_mbuf *m_seg = mbuf;
-
while (m_seg != NULL) {
if (m_seg->data_len == 0)
return SXE2_ERR_INVAL;
m_seg = m_seg->next;
}
-
return SXE2_SUCCESS;
}
@@ -79,7 +93,6 @@ u16 sxe2_tx_pkts_prepare(void *tx_queue,
u64 ol_flags = 0;
s32 ret = SXE2_SUCCESS;
s32 i = 0;
-
for (i = 0; i < nb_pkts; i++) {
mbuf = tx_pkts[i];
if (!mbuf)
@@ -98,12 +111,10 @@ u16 sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -SXE2_ERR_INVAL;
goto l_end;
}
-
if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
rte_errno = -SXE2_ERR_INVAL;
goto l_end;
}
-
#ifdef RTE_ETHDEV_DEBUG_TX
ret = rte_validate_tx_offload(mbuf);
if (ret != SXE2_SUCCESS) {
@@ -116,14 +127,12 @@ u16 sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -ret;
goto l_end;
}
-
ret = sxe2_tx_mbuf_empty_check(mbuf);
if (ret != SXE2_SUCCESS) {
rte_errno = -ret;
goto l_end;
}
}
-
l_end:
return i;
}
@@ -132,42 +141,117 @@ void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
u32 tx_mode_flags = 0;
-
+ s32 ret;
+ u32 vec_flags;
+ u32 batch_flags;
+ RTE_SET_USED(vec_flags);
PMD_INIT_FUNC_TRACE();
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = sxe2_tx_vec_support_check(dev, &vec_flags);
+ if (ret == SXE2_SUCCESS &&
+ (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128)) {
+ tx_mode_flags = vec_flags;
+#ifdef RTE_ARCH_X86
+ if ((rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1)) {
+ PMD_LOG_INFO(TX, "AVX512 is not supported in build env.");
+ }
+ if (((tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) == 0) &&
+ ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1) ||
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1)) &&
+ (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256)) {
+ PMD_LOG_INFO(TX, "AVX2 is not supported in build env.");
+ }
- dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
- dev->tx_pkt_burst = sxe2_tx_pkts;
+ if (((tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) == 0))
+ tx_mode_flags |= SXE2_TX_MODE_VEC_SSE;
+#endif
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ ret = sxe2_tx_queues_vec_prepare(dev);
+ if (ret != SXE2_SUCCESS)
+ tx_mode_flags &= (~SXE2_TX_MODE_VEC_SET_MASK);
+ }
+ }
+ ret = sxe2_tx_simple_batch_support_check(dev, &batch_flags);
+ if (ret == SXE2_SUCCESS && batch_flags == SXE2_TX_MODE_SIMPLE_BATCH)
+ tx_mode_flags |= SXE2_TX_MODE_SIMPLE_BATCH;
+ }
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ dev->tx_pkt_prepare = NULL;
+#ifdef RTE_ARCH_X86
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_OFFLOAD) {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse;
+ } else {
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse_simple;
+ }
+#endif
+ } else {
+ if (tx_mode_flags & SXE2_TX_MODE_SIMPLE_BATCH) {
+ dev->tx_pkt_prepare = NULL;
+ dev->tx_pkt_burst = sxe2_tx_pkts_simple;
+ } else {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ }
+ }
adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
tx_mode_flags, dev->data->port_id);
}
+static const struct {
+ eth_tx_burst_t tx_burst;
+ const char *info;
+} sxe2_tx_burst_infos[] = {
+ { sxe2_tx_pkts, "Scalar" },
+#ifdef RTE_ARCH_X86
+ { sxe2_tx_pkts_vec_sse, "Vector SSE" },
+ { sxe2_tx_pkts_vec_sse_simple, "Vector SSE Simple" },
+#endif
+};
+
+s32 sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_tx_burst_t pkt_burst = dev->tx_pkt_burst;
+ s32 ret = SXE2_ERR_INVAL;
+ u32 i;
+ u32 size;
+ size = RTE_DIM(sxe2_tx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_tx_burst_infos[i].tx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_tx_burst_infos[i].info);
+ ret = SXE2_SUCCESS;
+ break;
+ }
+ }
+ return ret;
+}
+
static s32 sxe2_rx_desciptor_status(void *rx_queue, u16 offset)
{
struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
volatile union sxe2_rx_desc *desc;
s32 ret;
-
if (unlikely(offset >= rxq->ring_depth)) {
ret = SXE2_ERR_INVAL;
goto l_end;
}
-
if (offset >= rxq->ring_depth - rxq->hold_num) {
ret = RTE_ETH_RX_DESC_UNAVAIL;
goto l_end;
}
-
if (rxq->processing_idx + offset >= rxq->ring_depth)
desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
else
desc = &rxq->desc_ring[rxq->processing_idx + offset];
-
if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
ret = RTE_ETH_RX_DESC_DONE;
else
ret = RTE_ETH_RX_DESC_AVAIL;
-
l_end:
PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
offset, ret, rxq->queue_id, rxq->port_id);
@@ -179,7 +263,6 @@ static s32 sxe2_rx_queue_count(void *rx_queue)
struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
volatile union sxe2_rx_desc *desc;
u16 done_num = 0;
-
desc = &rxq->desc_ring[rxq->processing_idx];
while ((done_num < rxq->ring_depth) &&
(rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
@@ -190,55 +273,97 @@ static s32 sxe2_rx_queue_count(void *rx_queue)
else
desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
}
-
PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
done_num, rxq->queue_id, rxq->port_id);
-
return done_num;
}
-static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload)
-{
- struct sxe2_rx_queue *rxq;
- bool en = false;
- u16 i;
-
- for (i = 0; i < dev->data->nb_rx_queues; ++i) {
- rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
- if (rxq == NULL)
- continue;
-
- if (0 != (rxq->offloads & offload)) {
- en = true;
- goto l_end;
- }
- }
-
-l_end:
- return en;
-}
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
u32 rx_mode_flags = 0;
+ s32 ret;
+ u32 vec_flags;
PMD_INIT_FUNC_TRACE();
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = sxe2_rx_vec_support_check(dev, &vec_flags);
+ if (ret == SXE2_SUCCESS &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ rx_mode_flags = vec_flags;
+#ifdef RTE_ARCH_X86
+ if ((rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1) &&
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) == 1))
+ PMD_LOG_INFO(RX, "AVX512 is not supported in build env");
+
+ if (((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) == 0) &&
+ ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) == 1) ||
+ (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) == 1)) &&
+ (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256))
+ PMD_LOG_INFO(RX, "AVX2 is not supported in build env");
+
+ if (((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) == 0) &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128)
+ rx_mode_flags |= SXE2_RX_MODE_VEC_SSE;
+#endif
+ if ((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) != 0) {
+ ret = sxe2_rx_queues_vec_prepare(dev);
+ if (ret != SXE2_SUCCESS)
+ rx_mode_flags &= (~SXE2_RX_MODE_VEC_SET_MASK);
+ }
+ }
+ }
+#ifdef RTE_ARCH_X86
+ if (rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) {
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_vec_sse_offload;
+ goto l_end;
+ }
+#endif
if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
else
dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
+l_end:
PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
rx_mode_flags, dev->data->port_id);
adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
}
+static const struct {
+ eth_rx_burst_t rx_burst;
+ const char *info;
+} sxe2_rx_burst_infos[] = {
+ { sxe2_rx_pkts_scattered, "Scalar Scattered" },
+ { sxe2_rx_pkts_scattered_split, "Scalar Scattered split" },
+#ifdef RTE_ARCH_X86
+ { sxe2_rx_pkts_scattered_vec_sse_offload, "Vector SSE Scattered" },
+#endif
+};
+
+s32 sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused u16 queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_rx_burst_t pkt_burst = dev->rx_pkt_burst;
+ s32 ret = SXE2_ERR_INVAL;
+ u32 i, size;
+ size = RTE_DIM(sxe2_rx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_rx_burst_infos[i].rx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_rx_burst_infos[i].info);
+ ret = SXE2_SUCCESS;
+ break;
+ }
+ }
+ return ret;
+}
+
void sxe2_set_common_function(struct rte_eth_dev *dev)
{
PMD_INIT_FUNC_TRACE();
-
dev->rx_queue_count = sxe2_rx_queue_count;
dev->rx_descriptor_status = sxe2_rx_desciptor_status;
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
index e6f671e3dc..8f929c4f19 100644
--- a/drivers/net/sxe2/sxe2_txrx.h
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -6,16 +6,17 @@
#define SXE2_TXRX_H
#include <ethdev_driver.h>
#include "sxe2_queue.h"
-
void sxe2_set_common_function(struct rte_eth_dev *dev);
+s32 __rte_cold sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ u32 *batch_flags);
u16 sxe2_tx_pkts_prepare(void *tx_queue,
struct rte_mbuf **tx_pkts, u16 nb_pkts);
-
void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
-
void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
-
+s32 sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
+s32 sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused u16 queue_id, struct rte_eth_burst_mode *mode);
#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.h b/drivers/net/sxe2/sxe2_txrx_poll.h
index 4924b0f41f..67da08e58e 100644
--- a/drivers/net/sxe2/sxe2_txrx_poll.h
+++ b/drivers/net/sxe2/sxe2_txrx_poll.h
@@ -8,7 +8,8 @@
#include "sxe2_queue.h"
u16 sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
-
+u16 sxe2_tx_pkts_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts);
u16 sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts);
u16 sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, u16 nb_pkts);
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.c b/drivers/net/sxe2/sxe2_txrx_vec.c
new file mode 100644
index 0000000000..30e1468020
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.c
@@ -0,0 +1,197 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_errno.h"
+
+s32 __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags)
+{
+ struct sxe2_rx_queue *rxq;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ *vec_flags = SXE2_RX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ if (!rte_is_power_of_2(rxq->ring_depth)) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if (rxq->rx_free_thresh < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC &&
+ (rxq->ring_depth % rxq->rx_free_thresh) != 0) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_RX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ u16 i;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+ if ((rxq->offloads & offload) != 0) {
+ en = true;
+ goto l_end;
+ }
+ }
+l_end:
+ return en;
+}
+
+static inline void sxe2_rx_queue_mbufs_release_vec(struct sxe2_rx_queue *rxq)
+{
+ const u16 mask = rxq->ring_depth - 1;
+ u16 i;
+ if (unlikely(!rxq->buffer_ring)) {
+ PMD_LOG_DEBUG(RX, "Rx queue release mbufs vec, buffer_ring if NULL."
+ "port_id:%u queue_id:%u", rxq->port_id, rxq->queue_id);
+ return;
+ }
+ if (rxq->realloc_num >= rxq->ring_depth)
+ return;
+ if (rxq->realloc_num == 0) {
+ for (i = 0; i < rxq->ring_depth; ++i) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ } else {
+ for (i = rxq->processing_idx;
+ i != rxq->realloc_start;
+ i = (i + 1) & mask) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ }
+ rxq->realloc_num = rxq->ring_depth;
+ memset(rxq->buffer_ring, 0, rxq->ring_depth * sizeof(rxq->buffer_ring[0]));
+}
+
+static inline void sxe2_rx_queue_vec_init(struct sxe2_rx_queue *rxq)
+{
+ uintptr_t data;
+ struct rte_mbuf mbuf_def;
+
+ memset(&mbuf_def, 0, sizeof(mbuf_def));
+ mbuf_def.buf_addr = 0;
+ mbuf_def.nb_segs = 1;
+ mbuf_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mbuf_def, 1);
+ rte_compiler_barrier();
+ data = (uintptr_t)&mbuf_def.rearm_data;
+ rxq->mbuf_init_value = *(u64 *)data;
+}
+
+s32 __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_rx_queue *rxq = NULL;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ PMD_LOG_INFO(RX, "Failed to prepare rx queue, rxq[%d] is NULL", i);
+ continue;
+ }
+ rxq->ops.mbufs_release = sxe2_rx_queue_mbufs_release_vec;
+ sxe2_rx_queue_vec_init(rxq);
+ }
+ return ret;
+}
+
+s32 __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags)
+{
+ struct sxe2_tx_queue *txq;
+ s32 ret = SXE2_SUCCESS;
+ u32 i;
+ *vec_flags = SXE2_TX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = SXE2_ERR_INVAL;
+ goto l_end;
+ }
+ if (txq->rs_thresh < SXE2_TX_RS_THRESH_MIN_VEC ||
+ txq->rs_thresh > SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = SXE2_ERR_NOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_TX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+static void sxe2_tx_queue_mbufs_release_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ u16 i;
+
+ if (unlikely(txq == NULL || txq->buffer_ring == NULL)) {
+ PMD_LOG_ERR(TX, "Tx release mbufs vec, invalid params.");
+ return;
+ }
+ i = txq->next_dd - (txq->rs_thresh - 1);
+ buffer = txq->buffer_ring;
+ if (txq->next_use < i) {
+ for ( ; i < txq->ring_depth; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+ i = 0;
+ }
+ for (; i < txq->next_use; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+}
+
+s32 __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_tx_queue *txq = NULL;
+ s32 ret = SXE2_SUCCESS;
+ u16 i;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL) {
+ PMD_LOG_INFO(TX, "Failed to prepare tx queue, txq[%d] is NULL", i);
+ continue;
+ }
+ txq->ops.mbufs_release = sxe2_tx_queue_mbufs_release_vec;
+ }
+ return ret;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.h b/drivers/net/sxe2/sxe2_txrx_vec.h
new file mode 100644
index 0000000000..cb6a3dd3b8
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef _SXE2_TXRX_VEC_H_
+#define _SXE2_TXRX_VEC_H_
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+#include "sxe2_type.h"
+#define SXE2_RX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_RX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_RX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_RX_MODE_VEC_AVX2 RTE_BIT32(3)
+#define SXE2_RX_MODE_VEC_AVX512 RTE_BIT32(4)
+#define SXE2_RX_MODE_VEC_NEON RTE_BIT32(5)
+#define SXE2_RX_MODE_BATCH_ALLOC RTE_BIT32(10)
+#define SXE2_RX_MODE_VEC_SET_MASK (SXE2_RX_MODE_VEC_SIMPLE | \
+ SXE2_RX_MODE_VEC_OFFLOAD | SXE2_RX_MODE_VEC_SSE | \
+ SXE2_RX_MODE_VEC_AVX2 | SXE2_RX_MODE_VEC_AVX512 | \
+ SXE2_RX_MODE_VEC_NEON)
+#define SXE2_TX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_TX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_TX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_TX_MODE_VEC_AVX2 RTE_BIT32(3)
+#define SXE2_TX_MODE_VEC_AVX512 RTE_BIT32(4)
+#define SXE2_TX_MODE_VEC_NEON RTE_BIT32(5)
+#define SXE2_TX_MODE_SIMPLE_BATCH RTE_BIT32(10)
+#define SXE2_TX_MODE_VEC_SET_MASK (SXE2_TX_MODE_VEC_SIMPLE | \
+ SXE2_TX_MODE_VEC_OFFLOAD | SXE2_TX_MODE_VEC_SSE | \
+ SXE2_TX_MODE_VEC_AVX2 | SXE2_TX_MODE_VEC_AVX512 | \
+ SXE2_TX_MODE_VEC_NEON)
+#define SXE2_TX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS | \
+ RTE_ETH_TX_OFFLOAD_QINQ_INSERT | \
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_SECURITY | \
+ RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM)
+#define SXE2_TX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_VLAN_INSERT | \
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM)
+#define SXE2_RX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP | \
+ RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT | \
+ RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_SECURITY | \
+ RTE_ETH_RX_OFFLOAD_QINQ_STRIP)
+#define SXE2_RX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_CHECKSUM | \
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_VLAN_STRIP | \
+ RTE_ETH_RX_OFFLOAD_VLAN_FILTER | \
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)
+#ifdef RTE_ARCH_X86
+u16 sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
+u16 sxe2_tx_pkts_vec_sse_simple(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts);
+u16 sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts);
+#endif
+s32 __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags);
+s32 __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev);
+s32 __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, u32 *vec_flags);
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, u64 offload);
+s32 __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev);
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_common.h b/drivers/net/sxe2/sxe2_txrx_vec_common.h
new file mode 100644
index 0000000000..c0405c9a59
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_common.h
@@ -0,0 +1,235 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TXRX_VEC_COMMON_H__
+#define __SXE2_TXRX_VEC_COMMON_H__
+#include <rte_atomic.h>
+#ifdef PCLINT
+#include "avx_stub.h"
+#endif
+#include "sxe2_rx.h"
+#include "sxe2_queue.h"
+#include "sxe2_tx.h"
+#include "sxe2_vsi.h"
+#include "sxe2_ethdev.h"
+#define SXE2_RX_NUM_PER_LOOP_SSE 4
+#define SXE2_RX_NUM_PER_LOOP_AVX 8
+#define SXE2_RX_NUM_PER_LOOP_NEON 4
+#define SXE2_RX_REARM_THRESH_VEC 64
+#define SXE2_RX_PKTS_BURST_BATCH_NUM_VEC 32
+#define SXE2_TX_RS_THRESH_MIN_VEC 32
+#define SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC 64
+
+static __rte_always_inline void
+sxe2_tx_pkts_mbuf_fill(struct sxe2_tx_buffer *buffer,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ u16 i;
+ for (i = 0; i < nb_pkts; ++i)
+ buffer[i].mbuf = tx_pkts[i];
+}
+
+static __rte_always_inline s32
+sxe2_tx_bufs_free_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC];
+ s32 ret;
+ u32 i;
+ u16 rs_thresh;
+ u16 free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - (rs_thresh - 1)];
+ mbuf = rte_pktmbuf_prefree_seg(buffer[0].mbuf);
+ if (likely(mbuf)) {
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (likely(mbuf)) {
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline void
+sxe2_tx_desc_fill_offloads(struct rte_mbuf *mbuf, u64 *desc_qw1)
+{
+ u64 offloads = mbuf->ol_flags;
+ u32 desc_cmd = 0;
+ u32 desc_offset = 0;
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ }
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ default:
+ break;
+ }
+ *desc_qw1 |= ((u64)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ *desc_qw1 |= ((u64)mbuf->vlan_tci) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT;
+ }
+ *desc_qw1 |= ((u64)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT;
+}
+#define SXE2_RX_UMBCAST_FLAGS_VAL_GET(_flags) \
+ (((_flags) & 0x30) >> 4)
+
+static inline void sxe2_vf_rx_vec_sw_stats_cnt(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf *mbuf, u8 umbcast_flag)
+{
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN, rte_memory_order_relaxed);
+ switch (SXE2_RX_UMBCAST_FLAGS_VAL_GET(umbcast_flag)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static inline u16
+sxe2_rx_pkts_refactor(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **mbuf_bufs, u16 mbuf_num,
+ u8 *split_rxe_flags, u8 *umbcast_flags)
+{
+ struct rte_mbuf *done_pkts[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ struct rte_mbuf *first_seg = rxq->pkt_first_seg;
+ struct rte_mbuf *last_seg = rxq->pkt_last_seg;
+ struct rte_mbuf *tmp_seg;
+ u16 done_num, buf_idx;
+ done_num = 0;
+ for (buf_idx = 0; buf_idx < mbuf_num; buf_idx++) {
+ if (last_seg) {
+ last_seg->next = mbuf_bufs[buf_idx];
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ first_seg->nb_segs++;
+ first_seg->pkt_len += mbuf_bufs[buf_idx]->data_len;
+ last_seg = last_seg->next;
+ if (split_rxe_flags[buf_idx] == 0) {
+ first_seg->hash = last_seg->hash;
+ first_seg->vlan_tci = last_seg->vlan_tci;
+ first_seg->ol_flags = last_seg->ol_flags;
+ first_seg->pkt_len -= rxq->crc_len;
+ if (last_seg->data_len > rxq->crc_len) {
+ last_seg->data_len -= rxq->crc_len;
+ } else {
+ tmp_seg = first_seg;
+ first_seg->nb_segs--;
+ while (tmp_seg->next != last_seg)
+ tmp_seg = tmp_seg->next;
+ tmp_seg->data_len -= (rxq->crc_len - last_seg->data_len);
+ tmp_seg->next = NULL;
+ rte_pktmbuf_free_seg(last_seg);
+ last_seg = NULL;
+ }
+ done_pkts[done_num++] = first_seg;
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, first_seg, umbcast_flags[buf_idx]);
+ first_seg = NULL;
+ last_seg = NULL;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ continue;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ last_seg = NULL;
+ continue;
+ }
+ } else {
+ if (split_rxe_flags[buf_idx] == 0) {
+ done_pkts[done_num++] = mbuf_bufs[buf_idx];
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, mbuf_bufs[buf_idx],
+ umbcast_flags[buf_idx]);
+ continue;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ first_seg = mbuf_bufs[buf_idx];
+ last_seg = first_seg;
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ mbuf_bufs[buf_idx]->pkt_len += rxq->crc_len;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ mbuf_bufs[buf_idx]->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free_seg(mbuf_bufs[buf_idx]);
+ continue;
+ }
+ }
+ }
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+ rte_memcpy(mbuf_bufs, done_pkts, done_num * (sizeof(struct rte_mbuf *)));
+ return done_num;
+}
+#endif
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_sse.c b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
new file mode 100644
index 0000000000..8cf11849d6
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
@@ -0,0 +1,545 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <ethdev_driver.h>
+#include <rte_bitops.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_vect.h>
+#include "rte_common.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_queue.h"
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_vsi.h"
+
+static __rte_always_inline void
+sxe2_tx_desc_fill_one_sse(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf *pkt,
+ u64 desc_cmd, bool with_offloads)
+{
+ __m128i data_desc;
+ u64 desc_qw1;
+ u32 desc_offset;
+ desc_qw1 = (SXE2_TX_DESC_DTYPE_DATA |
+ ((u64)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT |
+ ((u64)pkt->data_len) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL(pkt->l2_len);
+ desc_qw1 |= ((u64)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (with_offloads)
+ sxe2_tx_desc_fill_offloads(pkt, &desc_qw1);
+ data_desc = _mm_set_epi64x(desc_qw1, rte_pktmbuf_iova(pkt));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, desc), data_desc);
+}
+
+static __rte_always_inline u16
+sxe2_tx_pkts_vec_sse_batch(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ u16 nb_pkts, bool with_offloads)
+{
+ volatile union sxe2_tx_data_desc *desc;
+ struct sxe2_tx_buffer *buffer;
+ u16 next_use;
+ u16 res_num;
+ u16 tx_num;
+ u16 i;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free_vec(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx pkts sse batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ tx_num = nb_pkts;
+ next_use = txq->next_use;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ txq->desc_free_num -= nb_pkts;
+ res_num = txq->ring_depth - txq->next_use;
+ if (tx_num >= res_num) {
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, res_num);
+ for (i = 0; i < res_num - 1; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts++,
+ (SXE2_TX_DATA_DESC_CMD_EOP | SXE2_TX_DATA_DESC_CMD_RS),
+ with_offloads);
+ tx_num -= res_num;
+ next_use = 0;
+ txq->next_rs = txq->rs_thresh - 1;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ }
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, tx_num);
+ for (i = 0; i < tx_num; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ next_use += tx_num;
+ if (next_use > txq->next_rs) {
+ txq->desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ }
+ txq->next_use = next_use;
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, nb_pkts);
+l_end:
+ return nb_pkts;
+}
+
+static __rte_always_inline u16
+sxe2_tx_pkts_vec_sse_common(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ u16 nb_pkts, bool with_offloads)
+{
+ u16 tx_done_num = 0;
+ u16 tx_once_num;
+ u16 tx_need_num;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, txq->rs_thresh);
+ tx_once_num = sxe2_tx_pkts_vec_sse_batch(txq,
+ tx_pkts + tx_done_num,
+ tx_need_num, with_offloads);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+ return tx_done_num;
+}
+
+u16 sxe2_tx_pkts_vec_sse_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, false);
+}
+u16 sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, u16 nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, true);
+}
+
+static inline void sxe2_rx_queue_rearm_sse(struct sxe2_rx_queue *rxq)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ struct rte_mbuf *mbuf0, *mbuf1;
+ __m128i dma_addr0, dma_addr1;
+ __m128i virt_addr0, virt_addr1;
+ __m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
+ RTE_PKTMBUF_HEADROOM);
+ s32 ret;
+ u16 i;
+ u16 new_tail;
+ buffer = &rxq->buffer_ring[rxq->realloc_start];
+ desc = &rxq->desc_ring[rxq->realloc_start];
+ ret = rte_mempool_get_bulk(rxq->mb_pool, (void *)buffer,
+ SXE2_RX_REARM_THRESH_VEC);
+ if (ret != 0) {
+ PMD_LOG_INFO(RX, "Rx mbuf vec alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ if ((rxq->realloc_num + SXE2_RX_REARM_THRESH_VEC) >= rxq->ring_depth) {
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < SXE2_RX_NUM_PER_LOOP_SSE; ++i) {
+ buffer[i] = &rxq->fake_mbuf;
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc[i].read),
+ dma_addr0);
+ }
+ }
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed +=
+ SXE2_RX_REARM_THRESH_VEC;
+ goto l_end;
+ }
+ for (i = 0; i < SXE2_RX_REARM_THRESH_VEC; i += 2, buffer += 2) {
+ mbuf0 = buffer[0];
+ mbuf1 = buffer[1];
+#if RTE_IOVA_IN_MBUF
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+#endif
+ virt_addr0 = _mm_loadu_si128((__m128i *)&mbuf0->buf_addr);
+ virt_addr1 = _mm_loadu_si128((__m128i *)&mbuf1->buf_addr);
+#if RTE_IOVA_IN_MBUF
+ dma_addr0 = _mm_unpackhi_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpackhi_epi64(virt_addr1, virt_addr1);
+#else
+ dma_addr0 = _mm_unpacklo_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpacklo_epi64(virt_addr1, virt_addr1);
+#endif
+ dma_addr0 = _mm_add_epi64(dma_addr0, hdr_room);
+ dma_addr1 = _mm_add_epi64(dma_addr1, hdr_room);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr0);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr1);
+ }
+ rxq->realloc_start += SXE2_RX_REARM_THRESH_VEC;
+ if (rxq->realloc_start >= rxq->ring_depth)
+ rxq->realloc_start = 0;
+ rxq->realloc_num -= SXE2_RX_REARM_THRESH_VEC;
+ new_tail = (rxq->realloc_start == 0) ?
+ (rxq->ring_depth - 1) : (rxq->realloc_start - 1);
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, new_tail);
+l_end:
+ return;
+}
+
+static __rte_always_inline __m128i
+sxe2_rx_desc_fnav_flags_sse(__m128i descs_arr[4])
+{
+ __m128i descs_tmp1, descs_tmp2;
+ __m128i descs_fnav_vld;
+ __m128i v_zeros, v_ffff, v_u32_one;
+ __m128i m_flags;
+ const __m128i fdir_flags = _mm_set1_epi32(RTE_MBUF_F_RX_FDIR | RTE_MBUF_F_RX_FDIR_ID);
+ descs_tmp1 = _mm_unpacklo_epi32(descs_arr[0], descs_arr[1]);
+ descs_tmp2 = _mm_unpacklo_epi32(descs_arr[2], descs_arr[3]);
+ descs_fnav_vld = _mm_unpacklo_epi64(descs_tmp1, descs_tmp2);
+ descs_fnav_vld = _mm_slli_epi32(descs_fnav_vld, 26);
+ descs_fnav_vld = _mm_srli_epi32(descs_fnav_vld, 31);
+ v_zeros = _mm_setzero_si128();
+ v_ffff = _mm_cmpeq_epi32(v_zeros, v_zeros);
+ v_u32_one = _mm_srli_epi32(v_ffff, 31);
+ m_flags = _mm_cmpeq_epi32(descs_fnav_vld, v_u32_one);
+ m_flags = _mm_and_si128(m_flags, fdir_flags);
+ return m_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_desc_offloads_para_fill_sse(struct sxe2_rx_queue *rxq,
+ volatile union sxe2_rx_desc *desc __rte_unused,
+ __m128i descs_arr[4],
+ struct rte_mbuf **rx_pkts)
+{
+ const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_init_value);
+ __m128i rearm_arr[4];
+ __m128i tmp_desc_lo, tmp_desc_hi, flags, tmp_flags;
+ const __m128i desc_flags_mask = _mm_set_epi32(0x00001C04, 0x00001C04,
+ 0x00001C04, 0x00001C04);
+ const __m128i desc_flags_rss_mask = _mm_set_epi32(0x20000000, 0x20000000,
+ 0x20000000, 0x20000000);
+ const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ 0, 0, 0, 0);
+ const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, RTE_MBUF_F_RX_RSS_HASH,
+ 0, 0, 0, 0);
+ const __m128i cksum_flags =
+ _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1));
+ const __m128i cksum_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD);
+ const __m128i vlan_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED);
+ flags = _mm_unpackhi_epi32(descs_arr[0], descs_arr[1]);
+ tmp_flags = _mm_unpackhi_epi32(descs_arr[2], descs_arr[3]);
+ tmp_desc_lo = _mm_unpacklo_epi64(flags, tmp_flags);
+ tmp_desc_hi = _mm_unpackhi_epi64(flags, tmp_flags);
+ tmp_desc_lo = _mm_and_si128(tmp_desc_lo, desc_flags_mask);
+ tmp_desc_hi = _mm_and_si128(tmp_desc_hi, desc_flags_rss_mask);
+ tmp_flags = _mm_shuffle_epi8(vlan_flags, tmp_desc_lo);
+ flags = _mm_and_si128(tmp_flags, vlan_mask);
+ tmp_desc_lo = _mm_srli_epi32(tmp_desc_lo, 10);
+ tmp_flags = _mm_shuffle_epi8(cksum_flags, tmp_desc_lo);
+ tmp_flags = _mm_slli_epi32(tmp_flags, 1);
+ tmp_flags = _mm_and_si128(tmp_flags, cksum_mask);
+ flags = _mm_or_si128(flags, tmp_flags);
+ tmp_desc_hi = _mm_srli_epi32(tmp_desc_hi, 27);
+ tmp_flags = _mm_shuffle_epi8(rss_flags, tmp_desc_hi);
+ flags = _mm_or_si128(flags, tmp_flags);
+#ifndef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+ if (rxq->fnav_enable) {
+ __m128i tmp_fnav_flags = sxe2_rx_desc_fnav_flags_sse(descs_arr);
+ flags = _mm_or_si128(flags, tmp_fnav_flags);
+ rx_pkts[0]->hash.fdir.hi = desc[0].wb.fd_filter_id;
+ rx_pkts[1]->hash.fdir.hi = desc[1].wb.fd_filter_id;
+ rx_pkts[2]->hash.fdir.hi = desc[2].wb.fd_filter_id;
+ rx_pkts[3]->hash.fdir.hi = desc[3].wb.fd_filter_id;
+ }
+#endif
+ rearm_arr[0] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x30);
+ rearm_arr[1] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x30);
+ rearm_arr[2] = _mm_blend_epi16(mbuf_init, flags, 0x30);
+ rearm_arr[3] = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x30);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[0]->rearm_data), rearm_arr[0]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[1]->rearm_data), rearm_arr[1]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[2]->rearm_data), rearm_arr[2]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[3]->rearm_data), rearm_arr[3]);
+}
+
+static inline u16
+sxe2_rx_pkts_common_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts, u8 *split_rxe_flags,
+ u8 *umbcast_flags)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ __m128i descs_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i mbuf_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
+ __m128i pmbuf0;
+ __m128i ptype_all;
+#ifdef RTE_ARCH_X86_64
+ __m128i pmbuf1;
+#endif
+ u32 i;
+ u32 bit_num;
+ u16 done_num = 0;
+ const u32 *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ const __m128i crc_adjust =
+ _mm_set_epi16(0, 0, 0,
+ -rxq->crc_len,
+ 0, -rxq->crc_len,
+ 0, 0);
+ const __m128i rvp_shuf_mask =
+ _mm_set_epi8(7, 6, 5, 4,
+ 3, 2,
+ 13, 12,
+ 0XFF, 0xFF, 13, 12,
+ 0xFF, 0xFF, 0xFF, 0xFF);
+ const __m128i dd_mask = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ const __m128i eop_mask = _mm_slli_epi32(dd_mask,
+ SXE2_RX_DESC_STATUS_EOP_SHIFT);
+ const __m128i rxe_mask = _mm_set_epi64x(0x0000208000002080LL,
+ 0x0000208000002080LL);
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+ const __m128i ptype_mask = _mm_set_epi16(SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ rte_prefetch0(desc);
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, SXE2_RX_NUM_PER_LOOP_SSE);
+ if (rxq->realloc_num > SXE2_RX_REARM_THRESH_VEC)
+ sxe2_rx_queue_rearm_sse(rxq);
+ if ((rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK) == 0)
+ goto l_end;
+ buffer = &rxq->buffer_ring[rxq->processing_idx];
+ for (i = 0; i < nb_pkts; i += SXE2_RX_NUM_PER_LOOP_SSE,
+ desc += SXE2_RX_NUM_PER_LOOP_SSE) {
+ pmbuf0 = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, &buffer[i]));
+ descs_arr[3] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 3));
+ rte_compiler_barrier();
+ _mm_storeu_si128((__m128i *)&rx_pkts[i], pmbuf0);
+#ifdef RTE_ARCH_X86_64
+ pmbuf1 = _mm_loadu_si128((__m128i *)&buffer[i + 2]);
+#endif
+ descs_arr[2] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 2));
+ rte_compiler_barrier();
+ descs_arr[1] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 1));
+ rte_compiler_barrier();
+ descs_arr[0] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc));
+#ifdef RTE_ARCH_X86_64
+ _mm_storeu_si128((__m128i *)&rx_pkts[i + 2], pmbuf1);
+#endif
+ if (split_rxe_flags) {
+ rte_mbuf_prefetch_part2(rx_pkts[i]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 1]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 2]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 3]);
+ }
+ rte_compiler_barrier();
+ mbuf_arr[3] = _mm_shuffle_epi8(descs_arr[3], rvp_shuf_mask);
+ mbuf_arr[2] = _mm_shuffle_epi8(descs_arr[2], rvp_shuf_mask);
+ mbuf_arr[1] = _mm_shuffle_epi8(descs_arr[1], rvp_shuf_mask);
+ mbuf_arr[0] = _mm_shuffle_epi8(descs_arr[0], rvp_shuf_mask);
+ sterr_tmp2 = _mm_unpackhi_epi32(descs_arr[3], descs_arr[2]);
+ sterr_tmp1 = _mm_unpackhi_epi32(descs_arr[1], descs_arr[0]);
+ sxe2_rx_desc_offloads_para_fill_sse(rxq, desc, descs_arr, rx_pkts);
+ mbuf_arr[3] = _mm_add_epi16(mbuf_arr[3], crc_adjust);
+ mbuf_arr[2] = _mm_add_epi16(mbuf_arr[2], crc_adjust);
+ mbuf_arr[1] = _mm_add_epi16(mbuf_arr[1], crc_adjust);
+ mbuf_arr[0] = _mm_add_epi16(mbuf_arr[0], crc_adjust);
+ staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
+ ptype_all = _mm_and_si128(staterr, ptype_mask);
+ _mm_storeu_si128((void *)&rx_pkts[i + 3]->rx_descriptor_fields1,
+ mbuf_arr[3]);
+ _mm_storeu_si128((void *)&rx_pkts[i + 2]->rx_descriptor_fields1,
+ mbuf_arr[2]);
+ if (umbcast_flags != NULL) {
+ const __m128i umbcast_mask =
+ _mm_set_epi32(SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK);
+ const __m128i umbcast_shuf_mask =
+ _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x07, 0x0F,
+ 0x03, 0x0B);
+ __m128i umbcast_bits = _mm_and_si128(staterr, umbcast_mask);
+ umbcast_bits = _mm_shuffle_epi8(umbcast_bits, umbcast_shuf_mask);
+ *(s32 *)umbcast_flags = _mm_cvtsi128_si32(umbcast_bits);
+ umbcast_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ if (split_rxe_flags != NULL) {
+ __m128i eop_bits = _mm_andnot_si128(staterr, eop_mask);
+ __m128i rxe_bits = _mm_and_si128(staterr, rxe_mask);
+ rxe_bits = _mm_srli_epi32(rxe_bits, 7);
+ eop_bits = _mm_or_si128(eop_bits, rxe_bits);
+ eop_bits = _mm_shuffle_epi8(eop_bits, eop_shuf_mask);
+ *(s32 *)split_rxe_flags = _mm_cvtsi128_si32(eop_bits);
+ split_rxe_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ staterr = _mm_and_si128(staterr, dd_mask);
+ staterr = _mm_packs_epi32(staterr, _mm_setzero_si128());
+ _mm_storeu_si128((void *)&rx_pkts[i + 1]->rx_descriptor_fields1,
+ mbuf_arr[1]);
+ _mm_storeu_si128((void *)&rx_pkts[i]->rx_descriptor_fields1,
+ mbuf_arr[0]);
+ rx_pkts[i + 3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[i + 2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
+ rx_pkts[i + 1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[i]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ bit_num = rte_popcount64(_mm_cvtsi128_si64(staterr));
+ done_num += bit_num;
+ if (likely(bit_num != SXE2_RX_NUM_PER_LOOP_SSE))
+ break;
+ }
+ rxq->processing_idx += done_num;
+ rxq->processing_idx &= (rxq->ring_depth - 1);
+ rxq->realloc_num += done_num;
+ PMD_LOG_DEBUG(RX, "port_id=%u queue_id=%u last_id=%u recv_pkts=%d",
+ rxq->port_id, rxq->queue_id, rxq->processing_idx, done_num);
+l_end:
+ return done_num;
+}
+static __rte_always_inline u16
+sxe2_rx_pkts_scattered_batch_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ const u64 *split_rxe_flags64;
+ u8 split_rxe_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ u8 umbcast_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ u16 rx_done_num;
+ u16 rx_pkt_done_num;
+ rx_pkt_done_num = 0;
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, umbcast_flags);
+ } else {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, NULL);
+ }
+ if (rx_done_num == 0)
+ goto l_end;
+ if (!rxq->vsi->adapter->devargs.sw_stats_en) {
+ split_rxe_flags64 = (u64 *)split_rxe_flags;
+ if (rxq->pkt_first_seg == NULL &&
+ split_rxe_flags64[0] == 0 &&
+ split_rxe_flags64[1] == 0 &&
+ split_rxe_flags64[2] == 0 &&
+ split_rxe_flags64[3] == 0) {
+ rx_pkt_done_num = rx_done_num;
+ goto l_end;
+ }
+ if (rxq->pkt_first_seg == NULL) {
+ while (rx_pkt_done_num < rx_done_num &&
+ split_rxe_flags[rx_pkt_done_num] == 0)
+ rx_pkt_done_num++;
+ if (rx_pkt_done_num == rx_done_num)
+ goto l_end;
+ rxq->pkt_first_seg = rx_pkts[rx_pkt_done_num];
+ }
+ }
+ rx_pkt_done_num += sxe2_rx_pkts_refactor(rxq, &rx_pkts[rx_pkt_done_num],
+ rx_done_num - rx_pkt_done_num, &split_rxe_flags[rx_pkt_done_num],
+ &umbcast_flags[rx_pkt_done_num]);
+l_end:
+ return rx_pkt_done_num;
+}
+
+u16 sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, u16 nb_pkts)
+{
+ u16 done_num = 0;
+ u16 once_num;
+ while (nb_pkts > SXE2_RX_PKTS_BURST_BATCH_NUM_VEC) {
+ once_num =
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num,
+ SXE2_RX_PKTS_BURST_BATCH_NUM_VEC);
+ done_num += once_num;
+ nb_pkts -= once_num;
+ if (once_num < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC)
+ goto l_end;
+ }
+ done_num +=
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num, nb_pkts);
+l_end:
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox