* Re: [PATCH v4 16/20] net/txgbe: fix SFP module identification
From: Stephen Hemminger @ 2026-05-17 23:52 UTC (permalink / raw)
To: Zaiyu Wang; +Cc: dev, stable, Jiawen Wu
In-Reply-To: <20260511103604.19724-17-zaiyuwang@trustnetic.com>
On Mon, 11 May 2026 18:35:58 +0800
Zaiyu Wang <zaiyuwang@trustnetic.com> wrote:
> Some optical modules were not correctly recognized due to ambiguous
> classification in the original detection flow. Rework the module
> identification logic to cover all module types. Also narrow the
> I2C lock scope to avoid potential race conditions during module
> access.
>
> Fixes: ab191e6d9189 ("net/txgbe: support new SFP/QSFP modules")
> Cc: stable@dpdk.org
>
> Signed-off-by: Zaiyu Wang <zaiyuwang@trustnetic.com>
> ---
Lots of AI feedback here.
Note: not all this is required; like the request for extra release notes is AI being picky.
# Review of Patch 16/20: net/txgbe: fix SFP module identification
## Summary
This patch refactors SFP/QSFP module identification logic and adjusts I2C locking scope. The changes address module type detection and claim to fix race conditions.
---
## Errors
### 1. Resource leak in `txgbe_identify_sfp_module()` - multiple error paths missing unlock
**Error paths that acquire lock but fail to release on error:**
At line 836, `hw->mac.acquire_swfw_sync()` acquires `TXGBE_MNGSEM_SWPHY`. Several error paths after this point fail to release the lock:
```c
err = hw->phy.read_i2c_eeprom(hw, TXGBE_SFF_IDENTIFIER, &identifier);
if (err != 0) {
/* ... */
hw->mac.release_swfw_sync(hw, TXGBE_MNGSEM_SWPHY); // OK - releases
return TXGBE_ERR_SFP_NOT_PRESENT;
}
if (identifier != TXGBE_SFF_IDENTIFIER_SFP) {
hw->phy.type = txgbe_phy_sfp_unsupported;
hw->mac.release_swfw_sync(hw, TXGBE_MNGSEM_SWPHY); // OK - releases
return TXGBE_ERR_SFP_NOT_SUPPORTED;
}
err = hw->phy.read_i2c_eeprom(hw, ...);
/* Multiple read_i2c_eeprom calls follow */
```
**After line 894**, there are at least **4 read_i2c_eeprom() calls** (lines ~895-930 in the context) that can fail with `goto out;` - but `goto out;` is removed in this patch. If any of these reads fail now, the function returns immediately **without releasing the lock acquired at line 836**.
Example at line 1000:
```c
err = hw->phy.read_i2c_eeprom(hw, TXGBE_SFF_CABLE_SPEC_COMP, &cable_spec);
if (err != 0)
goto out; /* This label is REMOVED - now falls through or returns */
```
The old code had `out:` label that might have handled cleanup. The new code removes this pattern but doesn't add `hw->mac.release_swfw_sync()` calls before each early return.
**Impact:** Lock held indefinitely on I2C read failure, blocking other threads.
**Fix:** Add lock release before every early return after line 836:
```c
err = hw->phy.read_i2c_eeprom(hw, TXGBE_SFF_CABLE_SPEC_COMP, &cable_spec);
if (err != 0) {
hw->mac.release_swfw_sync(hw, TXGBE_MNGSEM_SWPHY);
goto out;
}
```
---
### 2. `txgbe_write_i2c_byte()` removes STOP condition - probable hardware protocol violation
**Line 1299 (old) vs 1299 (new):**
Old code:
```c
wr32(hw, TXGBE_I2CDATA, byte_offset | TXGBE_I2CDATA_STOP);
```
New code:
```c
wr32(hw, TXGBE_I2CDATA, byte_offset);
```
The `TXGBE_I2CDATA_STOP` flag is removed from the byte_offset write but remains only on the data write. This changes the I2C transaction sequence:
- Old: Write byte_offset with STOP, write data with STOP
- New: Write byte_offset (no STOP), write data with WRITE+STOP (via `txgbe_i2c_stop()`)
**Analysis:** I2C writes to EEPROMs typically follow: START → device_addr+W → ACK → byte_offset → ACK → data → ACK → STOP. Removing STOP from the byte_offset phase may be correct if the hardware requires a combined write transaction, **but this is a functional change to I2C timing that is not explained in the commit message** and could break module writes.
**Impact:** Could cause I2C write failures or data corruption on SFP/QSFP EEPROMs.
**Fix:** Either:
1. Verify via hardware datasheet that single STOP at end is correct, document in commit message
2. Restore `TXGBE_I2CDATA_STOP` on byte_offset write
---
### 3. Removed validation logic without replacement - SFP vendor checking deleted
**Lines 1025-1070 (old code, removed):** The old `txgbe_identify_qsfp_module()` read vendor OUI bytes, checked for Intel vendor, and enforced "allow_any_sfp" policy. This entire block is deleted with no equivalent in the new code.
**Impact:**
- Unsupported vendor modules are now silently accepted (security/quality risk)
- `hw->allow_unsupported_sfp` flag is no longer checked (user configuration ignored)
- Warning messages for untested modules removed (users not informed of risks)
**This is a functional regression unless the validation moved elsewhere** (not evident in this patch).
**Fix:** Restore vendor OUI check or document why it's safe to remove.
---
## Warnings
### 1. `txgbe_identify_qsfp_module()` GPIO config runs unconditionally
**Lines 1098-1099:**
```c
/* config GPIO before read i2c */
wr32(hw, TXGBE_GPIODATA, TXGBE_GPIOBIT_1);
```
This GPIO write now runs **even when media_type is not fiber_qsfp** (the media check is after the GPIO write at lines 1108-1111). The old code had this GPIO write inside the `if (hw->mac.type == txgbe_mac_aml40)` block, after the media check.
**Impact:** Unnecessary hardware access, potential side effects on non-QSFP ports.
**Fix:** Move GPIO config inside the `if (hw->mac.type == txgbe_mac_aml40)` block or add media_type check first.
---
### 2. Missing error propagation - `txgbe_read_i2c_sff8636()` page select failure ignored
**Line 1215:**
```c
hw->phy.write_i2c_byte(hw, TXGBE_SFF_QSFP_PAGE_SELECT,
TXGBE_I2C_EEPROM_DEV_ADDR,
page);
```
The return value from `write_i2c_byte()` is discarded. If page select fails, the subsequent read at line 1218 will read from the **wrong page**, producing incorrect module data.
**Fix:**
```c
s32 err = hw->phy.write_i2c_byte(hw, TXGBE_SFF_QSFP_PAGE_SELECT,
TXGBE_I2C_EEPROM_DEV_ADDR, page);
if (err != 0)
return err;
```
---
### 3. Implicit comparison on pointer
**Line 1123 (in `txgbe_identify_qsfp_module`):**
```c
if (err)
return -EBUSY;
```
Should be explicit:
```c
if (err != 0)
return -EBUSY;
```
---
### 4. Release notes likely required
This patch changes module detection logic and affects supported module types. API behavior changes (vendor validation removed, new module type support) typically require release notes under "Fixed Issues" or "Known Issues" if compatibility changes exist.
---
### 5. Function pointer assignments removed but functions still defined
**Lines 2912-2913 (txgbe_hw.c):**
```c
phy->read_i2c_byte_unlocked = txgbe_read_i2c_byte_unlocked;
phy->write_i2c_byte_unlocked = txgbe_write_i2c_byte_unlocked;
```
These assignments are removed, but the commit message doesn't indicate whether the `*_unlocked` functions are deleted or just unused. If the functions remain defined but unreferenced, they are dead code. If other drivers reference `phy->read_i2c_byte_unlocked` as a function pointer, this breaks them.
---
## Info
### 1. Simplified I2C locking approach is clearer
The removal of separate `*_unlocked` variants and the move to caller-managed locking (acquire once, multiple reads, release) is a good pattern that reduces lock churn.
---
### 2. New SFP type handling for 25G/40G modules
The patch adds proper detection for:
- 25G AOC modules (`txgbe_sfp_type_25g_aoc_core0/1`)
- 40G QSFP types with Ethernet compliance byte checks
This appears to address the "ambiguous classification" mentioned in the commit message.
---
### 3. Consistent use of `read_i2c_sff8636()` helper
The new `txgbe_read_i2c_sff8636()` helper centralizes page-select logic for QSFP modules, which is good for maintainability (despite the error handling issue noted above).
---
## Final Check Summary
**Errors (must fix):**
1. Lock leaks in `txgbe_identify_sfp_module()` on I2C read errors after line 836
2. `TXGBE_I2CDATA_STOP` removal in `txgbe_write_i2c_byte()` changes I2C protocol, not explained/verified
3. Vendor OUI validation removed without replacement - security/quality regression
**Warnings (should fix):**
1. GPIO config runs before media_type check
2. Page select error ignored in `txgbe_read_i2c_sff8636()`
3. Implicit comparison `if (err)`
4. Release notes likely needed for behavior changes
5. Unused function pointer assignments removed - check for dead code
^ permalink raw reply
* Re: [PATCH v4 20/20] net/txgbe: fix to enable Tx desc check
From: Stephen Hemminger @ 2026-05-17 23:55 UTC (permalink / raw)
To: Zaiyu Wang; +Cc: dev, stable, Jiawen Wu
In-Reply-To: <20260511103604.19724-21-zaiyuwang@trustnetic.com>
On Mon, 11 May 2026 18:36:02 +0800
Zaiyu Wang <zaiyuwang@trustnetic.com> wrote:
> Now lib security is enabled by default, and cannot be disabled if the
> driver is intended to be used. So Tdm_desc_chk is always unable to enable.
> Remove this restriction, and just enable the corresponding queue check.
>
> Fixes: 0eabdfcd4af4 ("net/txgbe: enable Tx descriptor error interrupt")
> Cc: stable@dpdk.org
>
> Signed-off-by: Zaiyu Wang <zaiyuwang@trustnetic.com>
> ---
Minor AI stuff here:
# Review of DPDK Patch: net/txgbe: fix to enable Tx desc check
## Summary
This patch modifies the Tx descriptor check logic in the txgbe driver to work correctly when security features are enabled. The change moves descriptor check setup from a global per-device operation to a per-queue operation.
## ERRORS
None found.
## WARNINGS
### 1. Bit manipulation should use RTE_BIT32() macro
**File:** drivers/net/txgbe/txgbe_rxtx.c
**Lines:** 4774, 4775
The code uses `BIT()` macro for bitmask generation. DPDK coding standards prefer `RTE_BIT32()` or `RTE_BIT64()` for clarity and portability.
**Current code:**
```c
wr32m(hw, TXGBE_TDM_DESC_CHK(txq->reg_idx / 32),
BIT(txq->reg_idx % 32), BIT(txq->reg_idx % 32));
```
**Suggested fix:**
```c
wr32m(hw, TXGBE_TDM_DESC_CHK(txq->reg_idx / 32),
RTE_BIT32(txq->reg_idx % 32), RTE_BIT32(txq->reg_idx % 32));
```
Verify that `BIT()` is defined in the driver's headers. If it's a legacy macro, consider using the DPDK standard `RTE_BIT32()` for consistency.
### 2. Conditional compilation without corresponding documentation update
**Context:** The patch changes the logic for when descriptor checks are enabled but there's no indication of release notes or documentation updates.
The original code disabled all descriptor checks when `RTE_LIB_SECURITY` was not defined. The new code enables descriptor checks per-queue unless the queue is using IPsec. This is a functional behavior change that should be documented.
**Suggested action:**
- Update release notes to describe the change in Tx descriptor check behavior
- Document that descriptor checks are now enabled per-queue based on IPsec usage rather than globally based on library compilation
## INFO
### 1. Potential logic optimization
The same bitmask value `BIT(txq->reg_idx % 32)` is used twice in the `wr32m()` call. Consider extracting it to a local variable for clarity:
```c
uint32_t queue_bit = BIT(txq->reg_idx % 32);
wr32m(hw, TXGBE_TDM_DESC_CHK(txq->reg_idx / 32), queue_bit, queue_bit);
```
This makes the intent clearer (setting the bit to 1) and slightly improves readability.
### 2. Parentheses around `using_ipsec` unnecessary
**Line:** 4772
```c
if (!(txq->using_ipsec))
```
Could be written as:
```c
if (!txq->using_ipsec)
```
The extra parentheses around `using_ipsec` are unnecessary since `!` has higher precedence than most operators.
---
## OVERALL ASSESSMENT
The patch correctly addresses the issue where descriptor checks could not be enabled when the security library was compiled in. The logic change from global enable/disable to per-queue configuration based on IPsec usage is sound.
**Key improvements made:**
- Moves descriptor check setup into the per-queue loop
- Only disables checks for queues actually using IPsec
- Removes the blanket disable when security library is present
**Correctness:** No bugs detected. The resource management and control flow are correct.
**Style compliance:** Minor issues with macro usage; otherwise compliant.
^ permalink raw reply
* Re: [PATCH v2 0/6] net/gve: add hardware timestamping support
From: Stephen Hemminger @ 2026-05-17 23:15 UTC (permalink / raw)
To: Mark Blasko; +Cc: dev
In-Reply-To: <20260515231936.3296603-1-blasko@google.com>
On Fri, 15 May 2026 23:19:29 +0000
Mark Blasko <blasko@google.com> wrote:
> This patch series introduces support for GVE hardware timestamping on DQO
> queues. To support concurrent access, a mutex lock is introduced to protect
> admin queue operations. A mechanism is then added to periodically synchronize
> the NIC clock via AdminQ, and support is introduced for the read_clock ethdev
> operation. Finally, the RX datapath is updated to reconstruct full 64-bit
> timestamps from the 32-bit values in DQO descriptors.
>
> ---
> v2:
> - Patch 1: Dropped ROBUST mutex attribute.
> - Patch 3: Added adminq timestamp counter reset to gve_adminq_alloc.
> - Patch 4:
> - Removed redundant void* casts.
> - Handled alarm reschedule failures by marking timestamp stale.
> - Added transient error logging on memzone allocation failure.
> - Patch 5: Scoped read_clock ethdev operation strictly to DQO queues.
> - Patch 6:
> - Scoped timestamp offload capability advertisement strictly to
> DQO queues.
> - Predicated capability advertisement directly on memzone
> allocation.
> - Initialized mbuf_timestamp_offset to -1.
> - Added blank line separating release notes.
> ---
>
> Mark Blasko (6):
> net/gve: add thread safety to admin queue
> net/gve: add device option support for HW timestamps
> net/gve: add AdminQ command for NIC timestamps
> net/gve: add periodic NIC clock synchronization
> net/gve: support read clock ethdev op
> net/gve: reconstruct HW timestamps from DQO
>
> .mailmap | 1 +
> doc/guides/nics/features/gve.ini | 1 +
> doc/guides/nics/gve.rst | 20 ++++
> doc/guides/rel_notes/release_26_07.rst | 4 +
> drivers/net/gve/base/gve_adminq.c | 128 +++++++++++++++++----
> drivers/net/gve/base/gve_adminq.h | 29 +++++
> drivers/net/gve/base/gve_desc_dqo.h | 8 +-
> drivers/net/gve/gve_ethdev.c | 149 ++++++++++++++++++++++++-
> drivers/net/gve/gve_ethdev.h | 39 +++++++
> drivers/net/gve/gve_rx_dqo.c | 26 +++++
> 10 files changed, 383 insertions(+), 22 deletions(-)
>
Looks good, once again AI followup still found some things you need to address.
Patch 5: net/gve: support read clock ethdev op
Error: gve_read_clock is defined as static but never assigned to
any eth_dev_ops table. In v1 the patch added
".read_clock = gve_read_clock" to both gve_eth_dev_ops and
gve_eth_dev_ops_dqo. The v2 changelog notes "Scoped read_clock
ethdev operation strictly to DQO queues," which should have left
the assignment in gve_eth_dev_ops_dqo and removed it from
gve_eth_dev_ops. Instead both assignments were dropped, leaving
the function unreferenced. DPDK CI builds with -Dwerror=true, so
-Wunused-function will fail the build, and the read_clock feature
is unreachable at runtime in any case.
Restore the assignment in gve_eth_dev_ops_dqo:
static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
...
.reta_query = gve_rss_reta_query,
.read_clock = gve_read_clock,
};
Warning: (unchanged from v1) gve_read_clock and the periodic
gve_read_nic_clock alarm callback both issue
GVE_ADMINQ_REPORT_NIC_TIMESTAMP into the single shared DMA buffer
priv->nic_ts_report, then read it after gve_adminq_execute_cmd
has released adminq_lock. If gve_read_clock is preempted between
gve_adminq_report_nic_timestamp returning and the be64_to_cpu
read, the alarm callback can memset() and reissue its own
command, so the user thread will read either zero or another
command's response. The simplest fix is for gve_read_clock to
return the cached priv->last_read_nic_timestamp instead of
issuing a fresh adminq command - the 250ms periodic sync keeps
it fresh enough for .read_clock semantics. Once the dev_op
registration is restored this race becomes reachable.
^ permalink raw reply
* Re: [PATCH] devtools: fix SPDX tag check
From: Thomas Monjalon @ 2026-05-17 21:34 UTC (permalink / raw)
To: David Marchand; +Cc: dev, stable, Richardson, Bruce, Hemant Agrawal
In-Reply-To: <CAJFAV8yAP34iRGiGKx0+Ls6_FpSa0EubP3KiDrUEU0fY-tp0FQ@mail.gmail.com>
>> If a file has no SPDX tag and is not filtered out by no_license_list,
>> there will be an error when using its path containing a slash
>> in the sed command delimited with slashes.
>>
>> It is fixed by using the pipe character as sed command delimiter.
>>
>> Fixes: b99a3b8aa989 ("license: standardize SPDX tag")
>> Cc: stable@dpdk.org
>>
>> Reported-by: David Marchand <david.marchand@redhat.com>
>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>
> Acked-by: David Marchand <david.marchand@redhat.com>
Applied
^ permalink raw reply
* Re: [PATCH v2] devtools: fix SPDX tag check
From: Thomas Monjalon @ 2026-05-17 20:26 UTC (permalink / raw)
To: Marat Khalili
Cc: dev@dpdk.org, stable@dpdk.org, David Marchand, Bruce Richardson,
Hemant Agrawal
In-Reply-To: <67049e0367f2462bade5c2fe837e8173@huawei.com>
01/05/2026 11:55, Marat Khalili:
> > Sorry I don't understand the need for a sentinel.
> > The script is working fine with an empty files_without_spdx.
>
> It may filter out everything instead of nothing in that case.
OK you're right.
So it becomes complex.
The first solution with sed looks better.
^ permalink raw reply
* [PATCH v2 2/2] dma/dpaa2: fix dpaa2_qdma_remove always returning success
From: Md Shofiqul Islam @ 2026-05-16 11:08 UTC (permalink / raw)
To: dev; +Cc: hemant.agrawal, stable
In-Reply-To: <20260516110828.35701-1-shofiqtest@gmail.com>
dpaa2_qdma_remove() checked the return value of rte_dma_pmd_release()
and logged an error on failure, but then unconditionally returned 0,
hiding the failure from fslmc_bus_unplug() and any higher-level
caller. A device-cleanup failure was silently swallowed.
Return the actual error code so that callers can detect and handle
removal failures correctly.
Fixes: 8caf8427f85a ("dma/dpaa2: introduce driver skeleton")
Bugzilla ID: 1914
Cc: hemant.agrawal@nxp.com
Cc: stable@dpdk.org
Signed-off-by: Md Shofiqul Islam <shofiqtest@gmail.com>
---
drivers/dma/dpaa2/dpaa2_qdma.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.c b/drivers/dma/dpaa2/dpaa2_qdma.c
index beca464c72..f7d94bb799 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.c
+++ b/drivers/dma/dpaa2/dpaa2_qdma.c
@@ -1739,7 +1739,7 @@ dpaa2_qdma_remove(struct rte_dpaa2_device *dpaa2_dev)
if (ret)
DPAA2_QDMA_ERR("Device cleanup failed");
- return 0;
+ return ret;
}
static struct rte_dpaa2_driver rte_dpaa2_qdma_pmd;
--
2.51.1
^ permalink raw reply related
* [PATCH v2 1/2] bus/fslmc: fix ignored return value in fslmc_bus_unplug
From: Md Shofiqul Islam @ 2026-05-16 11:08 UTC (permalink / raw)
To: dev; +Cc: hemant.agrawal, stable
In-Reply-To: <20260516110828.35701-1-shofiqtest@gmail.com>
fslmc_bus_unplug() called drv->remove() and discarded the return value,
unconditionally clearing driver references and reporting success even
when the remove callback signalled failure. As a result, callers had
no way to detect or react to removal errors.
Capture the return value and propagate it to the caller. Only clear
the driver references and log successful unplug when the callback
returns zero.
Fixes: b5721f271cbf ("bus/fslmc: support DPNI hotplug")
Bugzilla ID: 1914
Cc: hemant.agrawal@nxp.com
Cc: stable@dpdk.org
Signed-off-by: Md Shofiqul Islam <shofiqtest@gmail.com>
---
drivers/bus/fslmc/fslmc_bus.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c
index cf881b3eec..57bb61dc35 100644
--- a/drivers/bus/fslmc/fslmc_bus.c
+++ b/drivers/bus/fslmc/fslmc_bus.c
@@ -620,7 +620,9 @@ fslmc_bus_unplug(struct rte_device *rte_dev)
struct rte_dpaa2_driver *drv = dev->driver;
if (drv && drv->remove) {
- drv->remove(dev);
+ int ret = drv->remove(dev);
+ if (ret != 0)
+ return ret;
dev->driver = NULL;
dev->device.driver = NULL;
DPAA2_BUS_INFO("%s Un-Plugged", dev->device.name);
--
2.51.1
^ permalink raw reply related
* [PATCH v2 0/2] dpaa2: fix error propagation in remove path
From: Md Shofiqul Islam @ 2026-05-16 11:08 UTC (permalink / raw)
To: dev; +Cc: hemant.agrawal, stable
This series fixes two bugs in the dpaa2/fslmc device removal path where
error codes were silently discarded, making it impossible for callers to
detect removal failures.
Patch 1 fixes fslmc_bus_unplug() to capture and propagate the return
value of drv->remove() instead of ignoring it.
Patch 2 fixes dpaa2_qdma_remove() to return the actual error code from
rte_dma_pmd_release() instead of unconditionally returning 0.
v2:
- bus/fslmc: use 'if (ret != 0)' style per DPDK coding standard
Md Shofiqul Islam (2):
bus/fslmc: fix ignored return value in fslmc_bus_unplug
dma/dpaa2: fix dpaa2_qdma_remove always returning success
drivers/bus/fslmc/fslmc_bus.c | 4 +++-
drivers/dma/dpaa2/dpaa2_qdma.c | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
--
2.51.1
^ permalink raw reply
* Re: [PATCH 1/1] crypto/zsda: Update product name and maintainer
From: Thomas Monjalon @ 2026-05-17 10:01 UTC (permalink / raw)
To: Chengfei Han; +Cc: dev, ran.ming
In-Reply-To: <20260515081212.3565572-2-han.chengfei@zte.com.cn>
15/05/2026 10:12, Chengfei Han:
> Correct product name from '1cf2' to 'Neo X510/X512' in PMD documentation.
> Update maintainer for ZTE Storage Data Accelerator (ZSDA)
>
> Signed-off-by: Chengfei Han <han.chengfei@zte.com.cn>
> ---
> ZTE Storage Data Accelerator (ZSDA)
> -M: Hanxiao Li <li.hanxiao@zte.com.cn>
> +M: Chengfei Han <han.chengfei@zte.com.cn>
> +M: Ming Ran <ran.ming@zte.com.cn>
> F: drivers/common/zsda/
> F: drivers/compress/zsda/
I suppose you wanted to apply this change to the crypto driver as well,
not only for the compress driver.
Applied with assumed change.
^ permalink raw reply
* Re: [PATCH v1 1/2] net/zxdh: add supported zxdh support nics
From: Thomas Monjalon @ 2026-05-17 9:39 UTC (permalink / raw)
To: Junlong Wang; +Cc: stephen, ran.ming, dev
In-Reply-To: <20260514015650.3500578-1-wang.junlong1@zte.com.cn>
14/05/2026 03:56, Junlong Wang:
> update zxdh nics doc.rst, add zxdh support nics.
>
> Signed-off-by: Junlong Wang <wang.junlong1@zte.com.cn>
Series applies with fixes mentioned in the mail thread.
^ permalink raw reply
* Re: [PATCH v1 1/2] net/zxdh: add supported zxdh support nics
From: Thomas Monjalon @ 2026-05-17 9:38 UTC (permalink / raw)
To: Junlong Wang; +Cc: stephen, dev, ran.ming
In-Reply-To: <20260514015650.3500578-1-wang.junlong1@zte.com.cn>
14/05/2026 03:56, Junlong Wang:
> update zxdh nics doc.rst, add zxdh support nics.
There is no such file doc.rst
[...]
> +Supported NICs
> +---------------------------
Underlining is too long
> +
> +- ZXDH E310 25 Gigabit Ethernet Controller
> +- ZXDH E312 100 Gigabit Ethernet Controller
> +- ZXDH E312S 100 Gigabit Ethernet Controller
> +- ZXDH E316 200 Gigabit Ethernet Controller
^ permalink raw reply
* Re: [PATCH v1 2/2] net/zxdh: update zxdh Maintainers eamil
From: Thomas Monjalon @ 2026-05-17 9:36 UTC (permalink / raw)
To: Junlong Wang; +Cc: stephen, dev, ran.ming, dev
In-Reply-To: <20260514015650.3500578-2-wang.junlong1@zte.com.cn>
14/05/2026 03:56, Junlong Wang:
> update zxdh Maintainers eamil.
>
> Signed-off-by: Junlong Wang <wang.junlong1@zte.com.cn>
> ---
> ZTE zxdh
> M: Junlong Wang <wang.junlong1@zte.com.cn>
> -M: Lijie Shan <shan.lijie@zte.com.cn>
> +M: Ming Ran <ran.ming@zte.com.cn>
He must be added in .mailmap.
^ permalink raw reply
* [PATCH] net/ice: improve log messages for DDP loading
From: David Marchand @ 2026-05-16 10:19 UTC (permalink / raw)
To: dev; +Cc: patrick.mahan, Bruce Richardson, Anatoly Burakov
Some nics may not provide a serial number (PCI capability
RTE_PCI_EXT_CAP_ID_DSN).
This results in a confusing ERROR log:
ICE_INIT: ice_dev_init(): Failed to read device serial number
This is confusing as DDP loading does *not* require the serial number to
be present for the port to be functional afterwards.
Besides, after trying various path, if the default DDP is not present on
the runtime system, the port initialisation ends up with a vague error:
ICE_INIT: ice_load_pkg(): failed to search file path
Improve the situation with adjusting the log level when reading the
SN fails, then add more debug context to DDP file loading and end up
with a ERROR log mentioning the expected file.
ICE_INIT: ice_firmware_read(): Cannot read DDP file
/lib/firmware/updates/intel/ice/ddp/ice-b49691ffffe6e69c.pkg
ICE_INIT: ice_firmware_read(): Cannot read DDP file
/lib/firmware/intel/ice/ddp/ice-b49691ffffe6e69c.pkg
ICE_INIT: ice_firmware_read(): Cannot read DDP file
/lib/firmware/updates/intel/ice/ddp/ice.pkg
ICE_INIT: ice_firmware_read(): Cannot read DDP file
/lib/firmware/intel/ice/ddp/ice.pkg
ICE_INIT: ice_load_pkg(): Failed to load default DDP package
/lib/firmware/intel/ice/ddp/ice.pkg
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
drivers/net/intel/ice/ice_ethdev.c | 31 ++++++++++++++++++++----------
1 file changed, 21 insertions(+), 10 deletions(-)
diff --git a/drivers/net/intel/ice/ice_ethdev.c b/drivers/net/intel/ice/ice_ethdev.c
index 0f2e7aee14..e065581ccf 100644
--- a/drivers/net/intel/ice/ice_ethdev.c
+++ b/drivers/net/intel/ice/ice_ethdev.c
@@ -2003,6 +2003,17 @@ static int ice_read_customized_path(char *pkg_file, uint16_t buff_len)
return n;
}
+static int
+ice_firmware_read(const char *file, void *buf, size_t *bufsz)
+{
+ int ret = rte_firmware_read(file, buf, bufsz);
+
+ if (ret < 0)
+ PMD_INIT_LOG(DEBUG, "Cannot read DDP file %s", file);
+
+ return ret;
+}
+
int ice_load_pkg(struct ice_adapter *adapter, bool use_dsn, uint64_t dsn)
{
struct ice_hw *hw = &adapter->hw;
@@ -2016,7 +2027,7 @@ int ice_load_pkg(struct ice_adapter *adapter, bool use_dsn, uint64_t dsn)
/* first read any explicitly referenced DDP file*/
if (adapter->devargs.ddp_filename != NULL) {
strlcpy(pkg_file, adapter->devargs.ddp_filename, sizeof(pkg_file));
- if (rte_firmware_read(pkg_file, &buf, &bufsz) == 0) {
+ if (ice_firmware_read(pkg_file, &buf, &bufsz) == 0) {
goto load_fw;
} else {
PMD_INIT_LOG(ERR, "Cannot load DDP file: %s", pkg_file);
@@ -2032,11 +2043,11 @@ int ice_load_pkg(struct ice_adapter *adapter, bool use_dsn, uint64_t dsn)
if (use_dsn) {
snprintf(pkg_file, RTE_DIM(pkg_file), "%s/%s",
customized_path, opt_ddp_filename);
- if (rte_firmware_read(pkg_file, &buf, &bufsz) == 0)
+ if (ice_firmware_read(pkg_file, &buf, &bufsz) == 0)
goto load_fw;
}
snprintf(pkg_file, RTE_DIM(pkg_file), "%s/%s", customized_path, "ice.pkg");
- if (rte_firmware_read(pkg_file, &buf, &bufsz) == 0)
+ if (ice_firmware_read(pkg_file, &buf, &bufsz) == 0)
goto load_fw;
}
@@ -2046,23 +2057,23 @@ int ice_load_pkg(struct ice_adapter *adapter, bool use_dsn, uint64_t dsn)
strncpy(pkg_file, ICE_PKG_FILE_SEARCH_PATH_UPDATES,
ICE_MAX_PKG_FILENAME_SIZE);
strcat(pkg_file, opt_ddp_filename);
- if (rte_firmware_read(pkg_file, &buf, &bufsz) == 0)
+ if (ice_firmware_read(pkg_file, &buf, &bufsz) == 0)
goto load_fw;
strncpy(pkg_file, ICE_PKG_FILE_SEARCH_PATH_DEFAULT,
ICE_MAX_PKG_FILENAME_SIZE);
strcat(pkg_file, opt_ddp_filename);
- if (rte_firmware_read(pkg_file, &buf, &bufsz) == 0)
+ if (ice_firmware_read(pkg_file, &buf, &bufsz) == 0)
goto load_fw;
no_dsn:
strncpy(pkg_file, ICE_PKG_FILE_UPDATES, ICE_MAX_PKG_FILENAME_SIZE);
- if (rte_firmware_read(pkg_file, &buf, &bufsz) == 0)
+ if (ice_firmware_read(pkg_file, &buf, &bufsz) == 0)
goto load_fw;
strncpy(pkg_file, ICE_PKG_FILE_DEFAULT, ICE_MAX_PKG_FILENAME_SIZE);
- if (rte_firmware_read(pkg_file, &buf, &bufsz) < 0) {
- PMD_INIT_LOG(ERR, "failed to search file path");
+ if (ice_firmware_read(pkg_file, &buf, &bufsz) < 0) {
+ PMD_INIT_LOG(ERR, "Failed to load default DDP package " ICE_PKG_FILE_DEFAULT);
return -1;
}
@@ -2658,13 +2669,13 @@ ice_dev_init(struct rte_eth_dev *dev)
if (pos) {
if (rte_pci_read_config(pci_dev, &dsn_low, 4, pos + 4) < 0 ||
rte_pci_read_config(pci_dev, &dsn_high, 4, pos + 8) < 0) {
- PMD_INIT_LOG(ERR, "Failed to read pci config space");
+ PMD_INIT_LOG(WARNING, "Failed to read pci config space");
} else {
use_dsn = true;
dsn = (uint64_t)dsn_high << 32 | dsn_low;
}
} else {
- PMD_INIT_LOG(ERR, "Failed to read device serial number");
+ PMD_INIT_LOG(INFO, "Failed to read device serial number");
}
ret = ice_load_pkg(pf->adapter, use_dsn, dsn);
--
2.53.0
^ permalink raw reply related
* [PATCH 1/1] crypto/zsda: Update product name and maintainer
From: Chengfei Han @ 2026-05-15 7:27 UTC (permalink / raw)
To: dev; +Cc: ran.ming, Chengfei Han
In-Reply-To: <20260515072725.3563941-1-han.chengfei@zte.com.cn>
[-- Attachment #1.1.1: Type: text/plain, Size: 1826 bytes --]
Correct product name from '1cf2' to 'Neo X510/X512' in PMD documentation.
Update maintainer for ZTE Storage Data Accelerator (ZSDA)
Signed-off-by: Chengfei Han <han.chengfei@zte.com.cn>
---
MAINTAINERS | 3 ++-
doc/guides/compressdevs/zsda.rst | 3 +--
doc/guides/cryptodevs/zsda.rst | 3 +--
3 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 0f5539f851..7f30205b24 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1352,7 +1352,8 @@ F: doc/guides/compressdevs/zlib.rst
F: doc/guides/compressdevs/features/zlib.ini
ZTE Storage Data Accelerator (ZSDA)
-M: Hanxiao Li <li.hanxiao@zte.com.cn>
+M: Chengfei Han <han.chengfei@zte.com.cn>
+M: Ming Ran <ran.ming@zte.com.cn>
F: drivers/common/zsda/
F: drivers/compress/zsda/
F: doc/guides/compressdevs/zsda.rst
diff --git a/doc/guides/compressdevs/zsda.rst b/doc/guides/compressdevs/zsda.rst
index 25b7884535..5b8d4846a0 100644
--- a/doc/guides/compressdevs/zsda.rst
+++ b/doc/guides/compressdevs/zsda.rst
@@ -7,8 +7,7 @@ ZTE Storage Data Accelerator (ZSDA) Poll Mode Driver
The ZSDA compression PMD provides poll mode compression & decompression driver
support for the following hardware accelerator devices:
-* ZTE Processing accelerators 1cf2
-
+- Neo X510/X512
Features
--------
diff --git a/doc/guides/cryptodevs/zsda.rst b/doc/guides/cryptodevs/zsda.rst
index b024f537c1..8885663e9e 100644
--- a/doc/guides/cryptodevs/zsda.rst
+++ b/doc/guides/cryptodevs/zsda.rst
@@ -7,8 +7,7 @@ ZTE Storage Data Accelerator (ZSDA) Poll Mode Driver
The ZSDA crypto PMD provides poll mode Cipher and Hash driver
support for the following hardware accelerator devices:
-* ZTE Processing accelerators 1cf2
-
+- Neo X510/X512
Features
--------
--
2.27.0
[-- Attachment #1.1.2: Type: text/html , Size: 3054 bytes --]
^ permalink raw reply related
* [PATCH 0/1] crypto/zsda: update product name and maintainer
From: Chengfei Han @ 2026-05-15 7:27 UTC (permalink / raw)
To: dev; +Cc: ran.ming, Chengfei Han
[-- Attachment #1.1.1: Type: text/plain, Size: 463 bytes --]
*** BLURB HERE ***
This patch updates the ZSDA PMD driver:
- Correct product name from '1cf2' to 'Neo X510/X512' in documentation
- Update maintainer for ZTE Storage Data Accelerator (ZSDA)
Chengfei Han (1):
crypto/zsda: Update product name and maintainer
MAINTAINERS | 3 ++-
doc/guides/compressdevs/zsda.rst | 3 +--
doc/guides/cryptodevs/zsda.rst | 3 +--
3 files changed, 4 insertions(+), 5 deletions(-)
--
2.27.0
[-- Attachment #1.1.2: Type: text/html , Size: 949 bytes --]
^ permalink raw reply
* [PATCH 2/2] dma/dpaa2: fix dpaa2_qdma_remove always returning success
From: Md Shofiqul Islam @ 2026-05-13 20:37 UTC (permalink / raw)
To: dev; +Cc: hemant.agrawal, sachin.saxena, g.singh, stable
In-Reply-To: <20260513203725.1905-1-shofiqtest@gmail.com>
dpaa2_qdma_remove() checked the return value of rte_dma_pmd_release()
and logged an error on failure, but then unconditionally returned 0,
hiding the failure from fslmc_bus_unplug() and any higher-level
caller. A device-cleanup failure was silently swallowed.
Return the actual error code so that callers can detect and handle
removal failures correctly.
Fixes: 8caf8427f85a ("dma/dpaa2: introduce driver skeleton")
Bugzilla ID: 1914
Cc: hemant.agrawal@nxp.com
Cc: stable@dpdk.org
Signed-off-by: Md Shofiqul Islam <shofiqtest@gmail.com>
---
drivers/dma/dpaa2/dpaa2_qdma.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/dma/dpaa2/dpaa2_qdma.c b/drivers/dma/dpaa2/dpaa2_qdma.c
index beca464c72..f7d94bb799 100644
--- a/drivers/dma/dpaa2/dpaa2_qdma.c
+++ b/drivers/dma/dpaa2/dpaa2_qdma.c
@@ -1739,7 +1739,7 @@ dpaa2_qdma_remove(struct rte_dpaa2_device *dpaa2_dev)
if (ret)
DPAA2_QDMA_ERR("Device cleanup failed");
- return 0;
+ return ret;
}
static struct rte_dpaa2_driver rte_dpaa2_qdma_pmd;
--
2.54.0.windows.1
^ permalink raw reply related
* [PATCH 1/2] bus/fslmc: fix ignored return value in fslmc_bus_unplug
From: Md Shofiqul Islam @ 2026-05-13 20:37 UTC (permalink / raw)
To: dev; +Cc: hemant.agrawal, sachin.saxena, g.singh, stable
In-Reply-To: <20260513203725.1905-1-shofiqtest@gmail.com>
fslmc_bus_unplug() called drv->remove() and discarded the return value,
unconditionally clearing driver references and reporting success even
when the remove callback signalled failure. As a result, callers had
no way to detect or react to removal errors.
Capture the return value and propagate it to the caller. Only clear
the driver references and log successful unplug when the callback
returns zero.
Fixes: b5721f271cbf ("bus/fslmc: support DPNI hotplug")
Bugzilla ID: 1914
Cc: hemant.agrawal@nxp.com
Cc: stable@dpdk.org
Signed-off-by: Md Shofiqul Islam <shofiqtest@gmail.com>
---
drivers/bus/fslmc/fslmc_bus.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c
index cf881b3eec..9cfd8b10ba 100644
--- a/drivers/bus/fslmc/fslmc_bus.c
+++ b/drivers/bus/fslmc/fslmc_bus.c
@@ -620,7 +620,9 @@ fslmc_bus_unplug(struct rte_device *rte_dev)
struct rte_dpaa2_driver *drv = dev->driver;
if (drv && drv->remove) {
- drv->remove(dev);
+ int ret = drv->remove(dev);
+ if (ret)
+ return ret;
dev->driver = NULL;
dev->device.driver = NULL;
DPAA2_BUS_INFO("%s Un-Plugged", dev->device.name);
--
2.54.0.windows.1
^ permalink raw reply related
* [PATCH 0/2] bus/fslmc, dma/dpaa2: fix remove callback error propagation
From: Md Shofiqul Islam @ 2026-05-13 20:37 UTC (permalink / raw)
To: dev; +Cc: hemant.agrawal, sachin.saxena, g.singh, stable
Two dpaa2 drivers silently discard errors from their remove callbacks,
making it impossible for callers to detect cleanup failures:
- fslmc_bus_unplug() (bus/fslmc) calls drv->remove() and throws away
the return value, reporting success regardless of outcome.
- dpaa2_qdma_remove() (dma/dpaa2) logs a cleanup error but always
returns 0, hiding the failure from its caller.
This series fixes both by propagating the actual return values.
Bugzilla ID: 1914
Md Shofiqul Islam (2):
bus/fslmc: fix ignored return value in fslmc_bus_unplug
dma/dpaa2: fix dpaa2_qdma_remove always returning success
drivers/bus/fslmc/fslmc_bus.c | 4 +++-
drivers/dma/dpaa2/dpaa2_qdma.c | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
--
2.54.0.windows.1
^ permalink raw reply
* [PATCH v15 00/11] net/sxe2: fix logic errors and address feedback
From: liujie5 @ 2026-05-16 7:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260516025540.2092621-12-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch set addresses the feedback received on the v10 submission
for the sxe2 PMD. The primary focus is on fixing vector path selection,
ensuring memory safety during mbuf initialization, and cleaning up
redundant logic in the configuration functions.
v15 Changes:
- Fixed vector Rx burst function being overwritten by scalar selection.
- Refactored Rx/Tx mode set functions to seed flags from caps first,
eliminating tautological checks.
- Added memset for mbuf_def in vector init to avoid uninitialized reads.
- Converted pci_map_addr_info to designated initializers.
- Removed dead Windows-only code in meson.build.
- Added NULL checks for mbuf free for driver-wide consistency.
- Updated burst_mode_get to accurately report AVX paths.
- Adjusted SXE2_ETH_OVERHEAD to match actual VLAN capabilities.
Jie Liu (11):
mailmap: add Jie Liu
doc: add sxe2 guide and release notes
common/sxe2: add sxe2 basic structures
drivers: add base driver skeleton
drivers: add base driver probe skeleton
drivers: support PCI BAR mapping
common/sxe2: add ioctl interface for DMA map and unmap
net/sxe2: support queue setup and control
drivers: add data path for Rx and Tx
net/sxe2: add vectorized Rx and Tx
net/sxe2: implement Tx done cleanup
.mailmap | 1 +
doc/guides/nics/features/sxe2.ini | 29 +
doc/guides/nics/index.rst | 1 +
doc/guides/nics/sxe2.rst | 34 +
doc/guides/rel_notes/release_26_07.rst | 4 +
drivers/common/sxe2/meson.build | 15 +
drivers/common/sxe2/sxe2_common.c | 683 +++++++++++++
drivers/common/sxe2/sxe2_common.h | 85 ++
drivers/common/sxe2/sxe2_common_log.h | 81 ++
drivers/common/sxe2/sxe2_host_regs.h | 707 +++++++++++++
drivers/common/sxe2/sxe2_internal_ver.h | 33 +
drivers/common/sxe2/sxe2_ioctl_chnl.c | 325 ++++++
drivers/common/sxe2/sxe2_ioctl_chnl.h | 130 +++
drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 62 ++
drivers/common/sxe2/sxe2_osal.h | 73 ++
drivers/meson.build | 1 +
drivers/net/meson.build | 1 +
drivers/net/sxe2/meson.build | 32 +
drivers/net/sxe2/sxe2_cmd_chnl.c | 323 ++++++
drivers/net/sxe2/sxe2_cmd_chnl.h | 37 +
drivers/net/sxe2/sxe2_drv_cmd.h | 388 ++++++++
drivers/net/sxe2/sxe2_ethdev.c | 968 ++++++++++++++++++
drivers/net/sxe2/sxe2_ethdev.h | 318 ++++++
drivers/net/sxe2/sxe2_irq.h | 48 +
drivers/net/sxe2/sxe2_queue.c | 66 ++
drivers/net/sxe2/sxe2_queue.h | 195 ++++
drivers/net/sxe2/sxe2_rx.c | 559 +++++++++++
drivers/net/sxe2/sxe2_rx.h | 34 +
drivers/net/sxe2/sxe2_tx.c | 420 ++++++++
drivers/net/sxe2/sxe2_tx.h | 32 +
drivers/net/sxe2/sxe2_txrx.c | 357 +++++++
drivers/net/sxe2/sxe2_txrx.h | 23 +
drivers/net/sxe2/sxe2_txrx_common.h | 540 ++++++++++
drivers/net/sxe2/sxe2_txrx_poll.c | 1045 ++++++++++++++++++++
drivers/net/sxe2/sxe2_txrx_poll.h | 20 +
drivers/net/sxe2/sxe2_txrx_vec.c | 200 ++++
drivers/net/sxe2/sxe2_txrx_vec.h | 73 ++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 235 +++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 549 ++++++++++
drivers/net/sxe2/sxe2_vsi.c | 214 ++++
drivers/net/sxe2/sxe2_vsi.h | 204 ++++
41 files changed, 9145 insertions(+)
create mode 100644 doc/guides/nics/features/sxe2.ini
create mode 100644 doc/guides/nics/sxe2.rst
create mode 100644 drivers/common/sxe2/meson.build
create mode 100644 drivers/common/sxe2/sxe2_common.c
create mode 100644 drivers/common/sxe2/sxe2_common.h
create mode 100644 drivers/common/sxe2/sxe2_common_log.h
create mode 100644 drivers/common/sxe2/sxe2_host_regs.h
create mode 100644 drivers/common/sxe2/sxe2_internal_ver.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.c
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl.h
create mode 100644 drivers/common/sxe2/sxe2_ioctl_chnl_func.h
create mode 100644 drivers/common/sxe2/sxe2_osal.h
create mode 100644 drivers/net/sxe2/meson.build
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.c
create mode 100644 drivers/net/sxe2/sxe2_cmd_chnl.h
create mode 100644 drivers/net/sxe2/sxe2_drv_cmd.h
create mode 100644 drivers/net/sxe2/sxe2_ethdev.c
create mode 100644 drivers/net/sxe2/sxe2_ethdev.h
create mode 100644 drivers/net/sxe2/sxe2_irq.h
create mode 100644 drivers/net/sxe2/sxe2_queue.c
create mode 100644 drivers/net/sxe2/sxe2_queue.h
create mode 100644 drivers/net/sxe2/sxe2_rx.c
create mode 100644 drivers/net/sxe2/sxe2_rx.h
create mode 100644 drivers/net/sxe2/sxe2_tx.c
create mode 100644 drivers/net/sxe2/sxe2_tx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx.c
create mode 100644 drivers/net/sxe2/sxe2_txrx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_sse.c
create mode 100644 drivers/net/sxe2/sxe2_vsi.c
create mode 100644 drivers/net/sxe2/sxe2_vsi.h
--
2.47.3
^ permalink raw reply
* [PATCH v15 11/11] net/sxe2: implement Tx done cleanup
From: liujie5 @ 2026-05-16 7:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260516074618.2343883-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch implements the 'tx_done_cleanup' ethdev ops in the sxe2
PMD. This interface allows applications to explicitly request the
driver to release mbufs that have been transmitted and are no longer
needed by the hardware.
The implementation iterates through the Tx ring, checking the status
of the descriptors starting from the last cleaned tail. It releases
the corresponding mbufs back to the mempool until either the requested
number of packets are freed or no more completed descriptors are
found.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/sxe2_ethdev.c | 1 +
drivers/net/sxe2/sxe2_txrx.h | 1 +
drivers/net/sxe2/sxe2_txrx_poll.c | 101 +++++++++++++++++++++++++++++-
3 files changed, 102 insertions(+), 1 deletion(-)
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index d1bdc22bd0..8d66e5d8c5 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -290,6 +290,7 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.txq_info_get = sxe2_tx_queue_info_get,
.rx_burst_mode_get = sxe2_rx_burst_mode_get,
.tx_burst_mode_get = sxe2_tx_burst_mode_get,
+ .tx_done_cleanup = sxe2_tx_done_cleanup,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
index 61c6641e49..6d3d7455c2 100644
--- a/drivers/net/sxe2/sxe2_txrx.h
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -12,6 +12,7 @@ int32_t __rte_cold sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
uint32_t *batch_flags);
uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+int32_t sxe2_tx_done_cleanup(void *txq, uint32_t free_cnt);
void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.c b/drivers/net/sxe2/sxe2_txrx_poll.c
index dc6a83e380..7302ceb6f5 100644
--- a/drivers/net/sxe2/sxe2_txrx_poll.c
+++ b/drivers/net/sxe2/sxe2_txrx_poll.c
@@ -8,12 +8,13 @@
#include <rte_malloc.h>
#include <rte_memzone.h>
#include <ethdev_driver.h>
-#include <unistd.h>
#include "sxe2_osal.h"
#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_vec_common.h"
#include "sxe2_txrx_poll.h"
#include "sxe2_txrx.h"
+#include "sxe2_txrx_vec.h"
#include "sxe2_queue.h"
#include "sxe2_ethdev.h"
#include "sxe2_common_log.h"
@@ -118,6 +119,104 @@ static inline int32_t sxe2_tx_cleanup(struct sxe2_tx_queue *txq)
return ret;
}
+static int32_t sxe2_tx_done_cleanup_simple(struct sxe2_tx_queue *txq, uint32_t free_cnt)
+{
+ uint32_t free_cnt_align;
+ uint32_t free_cnt_once;
+ uint32_t i;
+
+ if (free_cnt == 0 || free_cnt > txq->ring_depth)
+ free_cnt = txq->ring_depth;
+
+ free_cnt_align = free_cnt - (free_cnt % txq->rs_thresh);
+ for (i = 0; i < free_cnt_align; i += free_cnt_once) {
+ if ((txq->ring_depth - txq->desc_free_num) < txq->rs_thresh)
+ break;
+
+ free_cnt_once = sxe2_tx_bufs_free(txq);
+ if (free_cnt_once == 0)
+ break;
+ }
+
+ return i;
+}
+
+static int32_t sxe2_tx_done_cleanup_normal(struct sxe2_tx_queue *txq, uint32_t free_cnt)
+{
+ struct sxe2_tx_buffer *buffer_ring = txq->buffer_ring;
+ int32_t ret;
+ uint16_t clean_last_idx, clean_idx;
+ uint16_t clean_last, clean_once;
+ uint16_t pkt_cnt, i;
+
+ if (txq->desc_free_num == 0 && sxe2_tx_cleanup(txq) != 0) {
+ ret = 0;
+ goto l_end;
+ }
+
+ if (free_cnt == 0)
+ free_cnt = txq->ring_depth;
+
+ clean_last_idx = txq->next_use;
+ clean_idx = buffer_ring[clean_last_idx].next_id;
+ clean_once = txq->desc_free_num;
+ clean_last = txq->desc_free_num;
+
+ for (pkt_cnt = 0; pkt_cnt < free_cnt;) {
+ for (i = 0; ((i < clean_once) &&
+ (pkt_cnt < free_cnt) &&
+ clean_idx != clean_last_idx); ++i) {
+ if (buffer_ring[clean_idx].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer_ring[clean_idx].mbuf);
+ buffer_ring[clean_idx].mbuf = NULL;
+ if (buffer_ring[clean_idx].last_id == clean_idx)
+ pkt_cnt++;
+ }
+ clean_idx = buffer_ring[clean_idx].next_id;
+ }
+
+ if ((txq->rs_thresh > (txq->ring_depth - txq->desc_free_num)) ||
+ clean_idx == clean_last_idx)
+ break;
+
+ if (pkt_cnt < free_cnt) {
+ if (sxe2_tx_cleanup(txq) != 0)
+ break;
+
+ clean_once = txq->desc_free_num - clean_last;
+ clean_last = txq->desc_free_num;
+ }
+ }
+
+ ret = pkt_cnt;
+l_end:
+ return ret;
+}
+
+int32_t sxe2_tx_done_cleanup(void *tx_queue, uint32_t free_cnt)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ struct sxe2_adapter *adapter = txq->vsi->adapter;
+ int32_t ret;
+
+ if (txq == NULL) {
+ ret = 0;
+ goto l_end;
+ }
+ if (adapter->q_ctxt.tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK)
+ ret = -ENOTSUP;
+ else if (adapter->q_ctxt.tx_mode_flags & SXE2_TX_MODE_SIMPLE_BATCH)
+ ret = sxe2_tx_done_cleanup_simple(txq, free_cnt);
+ else
+ ret = sxe2_tx_done_cleanup_normal(txq, free_cnt);
+
+ PMD_LOG_DEBUG(TX, "TX cleanup done desc queue_id=%u free_cnt=%d.",
+ txq->queue_id, ret);
+
+l_end:
+ return ret;
+}
+
static __rte_always_inline uint16_t
sxe2_tx_pkt_data_desc_count(struct rte_mbuf *tx_pkt)
{
--
2.47.3
^ permalink raw reply related
* [PATCH v15 02/11] doc: add sxe2 guide and release notes
From: liujie5 @ 2026-05-16 7:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260516074618.2343883-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Add a new guide for SXE2 PMD in the nics directory.
The guide contains driver capabilities, prerequisites,
and compilation/usage instructions.
Update the release notes to announce the addition of the
sxe2 network driver.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
doc/guides/nics/features/sxe2.ini | 29 ++++++++++++++++++++++
doc/guides/nics/index.rst | 1 +
doc/guides/nics/sxe2.rst | 34 ++++++++++++++++++++++++++
doc/guides/rel_notes/release_26_07.rst | 4 +++
4 files changed, 68 insertions(+)
create mode 100644 doc/guides/nics/features/sxe2.ini
create mode 100644 doc/guides/nics/sxe2.rst
diff --git a/doc/guides/nics/features/sxe2.ini b/doc/guides/nics/features/sxe2.ini
new file mode 100644
index 0000000000..7e350bab54
--- /dev/null
+++ b/doc/guides/nics/features/sxe2.ini
@@ -0,0 +1,29 @@
+;
+; Supported features of the 'sxe2' network poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; A feature with "P" indicates only be supported when non-vector path
+; is selected.
+;
+[Features]
+Fast mbuf free = P
+Free Tx mbuf on demand = Y
+Burst mode info = Y
+Queue start/stop = Y
+Buffer split on Rx = P
+Scattered Rx = Y
+CRC offload = Y
+VLAN offload = Y
+QinQ offload = P
+L3 checksum offload = Y
+L4 checksum offload = Y
+Timestamp offload = P
+Inner L3 checksum = P
+Inner L4 checksum = P
+Rx descriptor status = Y
+Tx descriptor status = Y
+FreeBSD = Y
+Linux = Y
+x86-32 = Y
+x86-64 = Y
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index cb818284fe..e20be478f8 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -68,6 +68,7 @@ Network Interface Controller Drivers
rnp
sfc_efx
softnic
+ sxe2
tap
thunderx
txgbe
diff --git a/doc/guides/nics/sxe2.rst b/doc/guides/nics/sxe2.rst
new file mode 100644
index 0000000000..7fcf9c085b
--- /dev/null
+++ b/doc/guides/nics/sxe2.rst
@@ -0,0 +1,34 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+
+SXE2 Poll Mode Driver
+======================
+
+The sxe2 PMD (**librte_net_sxe2**) provides poll mode driver support for
+10/25/50/100 Gbps Network Adapters.
+The embedded switch, Physical Functions (PF),
+and SR-IOV Virtual Functions (VF) are supported.
+
+Implementation details
+----------------------
+
+The sxe2 PMD is designed to operate alongside the sxe2 kernel network driver.
+For management and control operations, the PMD communicates with the kernel
+driver via ioctl interfaces. These commands are processed by the kernel
+driver and subsequently dispatched to the hardware firmware for execution.
+
+For security and robustness, the driver's data path is optimized to operate
+using virtual addresses (IOVA as VA mode). However, to ensure full
+compatibility in system environments where an IOMMU is absent or disabled,
+the driver also provides an explicit path to support physical addressing
+(IOVA as PA mode).
+
+The hardware is capable of handling the corresponding IOVA addresses (either
+VA or PA) directly, as provided by the DPDK memory subsystem. This ensures
+that DPDK applications can only access memory segments explicitly allocated
+to the current process, preventing unauthorized access to random physical
+memory.
+
+This capability allows the PMD to coexist with kernel network interfaces
+which remain functional, although they stop receiving unicast packets as
+long as they share the same MAC address.
diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
index f012d47a4b..fa0f0f5cca 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -64,6 +64,10 @@ New Features
* ``--auto-probing`` enables the initial bus probing, which is the current default behavior.
+* **Added Linkdata sxe2 ethernet driver.**
+
+ Added network driver for the Linkdata Network Adapters.
+
Removed Items
-------------
--
2.47.3
^ permalink raw reply related
* [PATCH v15 10/11] net/sxe2: add vectorized Rx and Tx
From: liujie5 @ 2026-05-16 7:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260516074618.2343883-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
This patch implements the vectorized data path for the sxe2 PMD.
It utilizes SIMD instructions (e.g., SSE) to process multiple
packets simultaneously, significantly improving throughput for
small packet processing.
The implementation includes:
* Vectorized Rx burst function for bulk descriptor processing.
* Vectorized Tx burst function with optimized resource cleanup.
* Capability flags update to reflect vectorized path support.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/meson.build | 5 +
drivers/net/sxe2/sxe2_ethdev.c | 31 +-
drivers/net/sxe2/sxe2_queue.c | 28 ++
drivers/net/sxe2/sxe2_queue.h | 4 +
drivers/net/sxe2/sxe2_txrx.c | 199 +++++++--
drivers/net/sxe2/sxe2_txrx.h | 11 +-
drivers/net/sxe2/sxe2_txrx_poll.c | 31 +-
drivers/net/sxe2/sxe2_txrx_poll.h | 4 +
drivers/net/sxe2/sxe2_txrx_vec.c | 200 +++++++++
drivers/net/sxe2/sxe2_txrx_vec.h | 73 ++++
drivers/net/sxe2/sxe2_txrx_vec_common.h | 235 ++++++++++
drivers/net/sxe2/sxe2_txrx_vec_sse.c | 549 ++++++++++++++++++++++++
12 files changed, 1296 insertions(+), 74 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.c
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_common.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_sse.c
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 5645e3ad61..3df57aee8c 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -13,6 +13,10 @@ deps += ['common_sxe2', 'hash','cryptodev','security']
includes += include_directories('../../common/sxe2')
+if arch_subdir == 'x86'
+ sources += files('sxe2_txrx_vec_sse.c')
+endif
+
sources += files(
'sxe2_ethdev.c',
'sxe2_cmd_chnl.c',
@@ -22,6 +26,7 @@ sources += files(
'sxe2_rx.c',
'sxe2_txrx_poll.c',
'sxe2_txrx.c',
+ 'sxe2_txrx_vec.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 8b76231057..d1bdc22bd0 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -107,25 +107,6 @@ static int32_t sxe2_dev_stop(struct rte_eth_dev *dev)
return ret;
}
-static int32_t sxe2_queues_start(struct rte_eth_dev *dev)
-{
- int32_t ret = 0;
- ret = sxe2_txqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start tx queue.");
- goto l_end;
- }
-
- ret = sxe2_rxqs_all_start(dev);
- if (ret) {
- PMD_LOG_ERR(INIT, "Failed to start rx queue.");
- sxe2_txqs_all_stop(dev);
- }
-
-l_end:
- return ret;
-}
-
static int32_t sxe2_dev_start(struct rte_eth_dev *dev)
{
int32_t ret = 0;
@@ -158,7 +139,7 @@ static int32_t sxe2_dev_start(struct rte_eth_dev *dev)
static int32_t sxe2_dev_close(struct rte_eth_dev *dev)
{
(void)sxe2_dev_stop(dev);
-
+ (void)sxe2_queues_release(dev);
sxe2_vsi_uninit(dev);
sxe2_dev_pci_map_uinit(dev);
@@ -296,13 +277,19 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_close = sxe2_dev_close,
.dev_infos_get = sxe2_dev_infos_get,
+ .rx_queue_start = sxe2_rx_queue_start,
+ .rx_queue_stop = sxe2_rx_queue_stop,
+ .tx_queue_start = sxe2_tx_queue_start,
+ .tx_queue_stop = sxe2_tx_queue_stop,
.rx_queue_setup = sxe2_rx_queue_setup,
- .tx_queue_setup = sxe2_tx_queue_setup,
.rx_queue_release = sxe2_rx_queue_release,
+ .tx_queue_setup = sxe2_tx_queue_setup,
.tx_queue_release = sxe2_tx_queue_release,
.rxq_info_get = sxe2_rx_queue_info_get,
.txq_info_get = sxe2_tx_queue_info_get,
+ .rx_burst_mode_get = sxe2_rx_burst_mode_get,
+ .tx_burst_mode_get = sxe2_tx_burst_mode_get,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
@@ -774,8 +761,6 @@ static int32_t sxe2_dev_init(struct rte_eth_dev *dev,
if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
sxe2_rx_mode_func_set(dev);
sxe2_tx_mode_func_set(dev);
- if (ret != 0)
- PMD_LOG_ERR(INIT, "Failed to mp init (secondary), ret=%d", ret);
goto l_end;
}
diff --git a/drivers/net/sxe2/sxe2_queue.c b/drivers/net/sxe2/sxe2_queue.c
index 93f8236381..1786d6ea4f 100644
--- a/drivers/net/sxe2/sxe2_queue.c
+++ b/drivers/net/sxe2/sxe2_queue.c
@@ -5,6 +5,8 @@
#include "sxe2_ethdev.h"
#include "sxe2_queue.h"
#include "sxe2_common_log.h"
+#include "sxe2_tx.h"
+#include "sxe2_rx.h"
void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
struct sxe2_drv_queue_caps *q_caps)
@@ -36,3 +38,29 @@ int32_t sxe2_queues_init(struct rte_eth_dev *dev)
return ret;
}
+
+int32_t sxe2_queues_start(struct rte_eth_dev *dev)
+{
+ int32_t ret = 0;
+
+ ret = sxe2_txqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start tx queue.");
+ goto l_end;
+ }
+
+ ret = sxe2_rxqs_all_start(dev);
+ if (ret) {
+ PMD_LOG_ERR(INIT, "Failed to start rx queue.");
+ sxe2_txqs_all_stop(dev);
+ }
+l_end:
+ return ret;
+}
+
+void sxe2_queues_release(struct rte_eth_dev *dev)
+{
+ sxe2_all_rxqs_release(dev);
+
+ sxe2_all_txqs_release(dev);
+}
diff --git a/drivers/net/sxe2/sxe2_queue.h b/drivers/net/sxe2/sxe2_queue.h
index e587e582fa..5195e2dd16 100644
--- a/drivers/net/sxe2/sxe2_queue.h
+++ b/drivers/net/sxe2/sxe2_queue.h
@@ -188,4 +188,8 @@ void sxe2_sw_queue_ctx_hw_cap_set(struct sxe2_adapter *adapter,
int32_t sxe2_queues_init(struct rte_eth_dev *dev);
+int32_t sxe2_queues_start(struct rte_eth_dev *dev);
+
+void sxe2_queues_release(struct rte_eth_dev *dev);
+
#endif /* __SXE2_QUEUE_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
index 2531b49a52..058bf86931 100644
--- a/drivers/net/sxe2/sxe2_txrx.c
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -9,12 +9,11 @@
#include <rte_memzone.h>
#include <ethdev_driver.h>
#include <unistd.h>
-
#include "sxe2_txrx.h"
#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_vec.h"
#include "sxe2_txrx_poll.h"
#include "sxe2_ethdev.h"
-
#include "sxe2_common_log.h"
#include "sxe2_osal.h"
#include "sxe2_cmd_chnl.h"
@@ -22,6 +21,30 @@
#include <rte_cpuflags.h>
#endif
+int32_t __rte_cold
+sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ uint32_t *batch_flags)
+{
+ struct sxe2_tx_queue *txq;
+ int32_t ret = 0;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+ if (txq->offloads != (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) ||
+ txq->rs_thresh < SXE2_TX_PKTS_BURST_BATCH_NUM) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ }
+ *batch_flags = SXE2_TX_MODE_SIMPLE_BATCH;
+l_end:
+ return ret;
+}
static int32_t sxe2_tx_desciptor_status(void *tx_queue, uint16_t offset)
{
struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
@@ -32,27 +55,23 @@ static int32_t sxe2_tx_desciptor_status(void *tx_queue, uint16_t offset)
ret = -EINVAL;
goto l_end;
}
-
desc_idx = txq->next_use + offset;
- desc_idx = DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
+ desc_idx = SXE2_DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
if (desc_idx >= txq->ring_depth) {
desc_idx -= txq->ring_depth;
if (desc_idx >= txq->ring_depth)
desc_idx -= txq->ring_depth;
}
-
if (desc_idx == 0)
desc_idx = txq->rs_thresh - 1;
else
desc_idx -= 1;
-
if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
(txq->desc_ring[desc_idx].wb.dd &
rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
ret = RTE_ETH_TX_DESC_DONE;
else
ret = RTE_ETH_TX_DESC_FULL;
-
l_end:
return ret;
}
@@ -60,7 +79,6 @@ static int32_t sxe2_tx_desciptor_status(void *tx_queue, uint16_t offset)
static inline int32_t sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
{
struct rte_mbuf *m_seg = mbuf;
-
while (m_seg != NULL) {
if (m_seg->data_len == 0)
return -EINVAL;
@@ -68,6 +86,7 @@ static inline int32_t sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
}
return 0;
+
}
uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
@@ -97,12 +116,10 @@ uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -EINVAL;
goto l_end;
}
-
if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
rte_errno = -EINVAL;
goto l_end;
}
-
#ifdef RTE_ETHDEV_DEBUG_TX
ret = rte_validate_tx_offload(mbuf);
if (ret != 0) {
@@ -115,14 +132,12 @@ uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
rte_errno = -ret;
goto l_end;
}
-
ret = sxe2_tx_mbuf_empty_check(mbuf);
if (ret != 0) {
rte_errno = -ret;
goto l_end;
}
}
-
l_end:
return i;
}
@@ -131,16 +146,85 @@ void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
uint32_t tx_mode_flags = 0;
+ int32_t ret;
+ uint32_t vec_flags;
+ uint32_t batch_flags;
PMD_INIT_FUNC_TRACE();
-
- dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
- dev->tx_pkt_burst = sxe2_tx_pkts;
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = sxe2_tx_vec_support_check(dev, &vec_flags);
+ if (ret == 0 &&
+ (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128)) {
+ tx_mode_flags = vec_flags;
+#ifdef RTE_ARCH_X86
+ if (((tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) == 0))
+ tx_mode_flags |= SXE2_TX_MODE_VEC_SSE;
+#endif
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ ret = sxe2_tx_queues_vec_prepare(dev);
+ if (ret != 0)
+ tx_mode_flags &= (~SXE2_TX_MODE_VEC_SET_MASK);
+ }
+ }
+ ret = sxe2_tx_simple_batch_support_check(dev, &batch_flags);
+ if (ret == 0 && batch_flags == SXE2_TX_MODE_SIMPLE_BATCH)
+ tx_mode_flags |= SXE2_TX_MODE_SIMPLE_BATCH;
+ }
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK) {
+ dev->tx_pkt_prepare = NULL;
+#ifdef RTE_ARCH_X86
+ if (tx_mode_flags & SXE2_TX_MODE_VEC_OFFLOAD) {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse;
+ } else {
+ dev->tx_pkt_burst = sxe2_tx_pkts_vec_sse_simple;
+ }
+#endif
+ } else {
+ if (tx_mode_flags & SXE2_TX_MODE_SIMPLE_BATCH) {
+ dev->tx_pkt_prepare = NULL;
+ dev->tx_pkt_burst = sxe2_tx_pkts_simple;
+ } else {
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ }
+ }
adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
tx_mode_flags, dev->data->port_id);
}
+static const struct {
+ eth_tx_burst_t tx_burst;
+ const char *info;
+} sxe2_tx_burst_infos[] = {
+ { sxe2_tx_pkts, "Scalar" },
+#ifdef RTE_ARCH_X86
+ { sxe2_tx_pkts_vec_sse, "Vector SSE" },
+ { sxe2_tx_pkts_vec_sse_simple, "Vector SSE Simple" },
+#endif
+};
+
+int32_t sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_tx_burst_t pkt_burst = dev->tx_pkt_burst;
+ int32_t ret = -EINVAL;
+ uint32_t i;
+ uint32_t size;
+
+ size = RTE_DIM(sxe2_tx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_tx_burst_infos[i].tx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_tx_burst_infos[i].info);
+ ret = 0;
+ break;
+ }
+ }
+ return ret;
+}
+
static int32_t sxe2_rx_desciptor_status(void *rx_queue, uint16_t offset)
{
struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
@@ -151,22 +235,18 @@ static int32_t sxe2_rx_desciptor_status(void *rx_queue, uint16_t offset)
ret = -EINVAL;
goto l_end;
}
-
if (offset >= rxq->ring_depth - rxq->hold_num) {
ret = RTE_ETH_RX_DESC_UNAVAIL;
goto l_end;
}
-
if (rxq->processing_idx + offset >= rxq->ring_depth)
desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
else
desc = &rxq->desc_ring[rxq->processing_idx + offset];
-
if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
ret = RTE_ETH_RX_DESC_DONE;
else
ret = RTE_ETH_RX_DESC_AVAIL;
-
l_end:
PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
offset, ret, rxq->queue_id, rxq->port_id);
@@ -189,55 +269,86 @@ static int32_t sxe2_rx_queue_count(void *rx_queue)
else
desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
}
-
PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
done_num, rxq->queue_id, rxq->port_id);
-
return done_num;
}
-static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload)
-{
- struct sxe2_rx_queue *rxq;
- bool en = false;
- uint16_t i;
-
- for (i = 0; i < dev->data->nb_rx_queues; ++i) {
- rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
- if (rxq == NULL)
- continue;
-
- if (0 != (rxq->offloads & offload)) {
- en = true;
- goto l_end;
- }
- }
-
-l_end:
- return en;
-}
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
{
struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
uint32_t rx_mode_flags = 0;
+ int32_t ret;
+ uint32_t vec_flags;
PMD_INIT_FUNC_TRACE();
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = sxe2_rx_vec_support_check(dev, &vec_flags);
+ if (ret == 0 &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128) {
+ rx_mode_flags = vec_flags;
+#ifdef RTE_ARCH_X86
+ if (((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) == 0) &&
+ rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_128)
+ rx_mode_flags |= SXE2_RX_MODE_VEC_SSE;
+#endif
+ if ((rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) != 0) {
+ ret = sxe2_rx_queues_vec_prepare(dev);
+ if (ret != 0)
+ rx_mode_flags &= (~SXE2_RX_MODE_VEC_SET_MASK);
+ }
+ }
+ }
+#ifdef RTE_ARCH_X86
+ if (rx_mode_flags & SXE2_RX_MODE_VEC_SET_MASK) {
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_vec_sse_offload;
+ goto l_end;
+ }
+#endif
if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
else
dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
-
+ goto l_end;
+l_end:
PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
rx_mode_flags, dev->data->port_id);
adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
}
+static const struct {
+ eth_rx_burst_t rx_burst;
+ const char *info;
+} sxe2_rx_burst_infos[] = {
+ { sxe2_rx_pkts_scattered, "Scalar Scattered" },
+ { sxe2_rx_pkts_scattered_split, "Scalar Scattered split" },
+#ifdef RTE_ARCH_X86
+ { sxe2_rx_pkts_scattered_vec_sse_offload, "Vector SSE Scattered" },
+#endif
+};
+
+int32_t sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode)
+{
+ eth_rx_burst_t pkt_burst = dev->rx_pkt_burst;
+ int32_t ret = -EINVAL;
+ uint32_t i, size;
+ size = RTE_DIM(sxe2_rx_burst_infos);
+ for (i = 0; i < size; ++i) {
+ if (pkt_burst == sxe2_rx_burst_infos[i].rx_burst) {
+ snprintf(mode->info, sizeof(mode->info), "%s",
+ sxe2_rx_burst_infos[i].info);
+ ret = 0;
+ break;
+ }
+ }
+ return ret;
+}
+
void sxe2_set_common_function(struct rte_eth_dev *dev)
{
PMD_INIT_FUNC_TRACE();
-
dev->rx_queue_count = sxe2_rx_queue_count;
dev->rx_descriptor_status = sxe2_rx_desciptor_status;
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
index f6558e2189..61c6641e49 100644
--- a/drivers/net/sxe2/sxe2_txrx.h
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -6,16 +6,17 @@
#define SXE2_TXRX_H
#include <ethdev_driver.h>
#include "sxe2_queue.h"
-
void sxe2_set_common_function(struct rte_eth_dev *dev);
+int32_t __rte_cold sxe2_tx_simple_batch_support_check(struct rte_eth_dev *dev,
+ uint32_t *batch_flags);
uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
-
void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
-
void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
-
void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
-
+int32_t sxe2_tx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
+int32_t sxe2_rx_burst_mode_get(struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, struct rte_eth_burst_mode *mode);
#endif /* __SXE2_TXRX_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.c b/drivers/net/sxe2/sxe2_txrx_poll.c
index 6d37fdef36..dc6a83e380 100644
--- a/drivers/net/sxe2/sxe2_txrx_poll.c
+++ b/drivers/net/sxe2/sxe2_txrx_poll.c
@@ -125,7 +125,7 @@ sxe2_tx_pkt_data_desc_count(struct rte_mbuf *tx_pkt)
uint16_t count = 0;
while (m_seg != NULL) {
- count += DIV_ROUND_UP(m_seg->data_len,
+ count += SXE2_DIV_ROUND_UP(m_seg->data_len,
SXE2_TX_MAX_DATA_NUM_PER_DESC);
m_seg = m_seg->next;
}
@@ -369,11 +369,12 @@ uint16_t sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkt
desc->read.type_cmd_off_bsz_l2t |=
rte_cpu_to_le_64(((uint64_t)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT);
}
+ goto l_end_of_tx;
l_exit_logic:
if (tx_num == 0)
goto l_end;
- goto l_end_of_tx;
+
l_end_of_tx:
SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
@@ -483,6 +484,32 @@ static inline uint16_t sxe2_tx_pkts_batch(void *tx_queue,
return nb_pkts;
}
+uint16_t sxe2_tx_pkts_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ uint16_t tx_done_num;
+ uint16_t tx_once_num;
+ uint16_t tx_need_num;
+ if (likely(nb_pkts <= SXE2_TX_PKTS_BURST_BATCH_NUM)) {
+ tx_done_num = sxe2_tx_pkts_batch(tx_queue,
+ tx_pkts, nb_pkts);
+ goto l_end;
+ }
+ tx_done_num = 0;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, SXE2_TX_PKTS_BURST_BATCH_NUM);
+ tx_once_num = sxe2_tx_pkts_batch(tx_queue,
+ &tx_pkts[tx_done_num],
+ tx_need_num);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+l_end:
+ return tx_done_num;
+}
+
static inline void
sxe2_update_rx_tail(struct sxe2_rx_queue *rxq, uint16_t hold_num, uint16_t rx_id)
{
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.h b/drivers/net/sxe2/sxe2_txrx_poll.h
index f45e33f9b7..6bb2238a2f 100644
--- a/drivers/net/sxe2/sxe2_txrx_poll.h
+++ b/drivers/net/sxe2/sxe2_txrx_poll.h
@@ -9,6 +9,10 @@
uint16_t sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+uint16_t sxe2_tx_pkts_simple(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+
+uint16_t sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
+
uint16_t sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
uint16_t sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.c b/drivers/net/sxe2/sxe2_txrx_vec.c
new file mode 100644
index 0000000000..1e03b53d67
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.c
@@ -0,0 +1,200 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+
+int32_t __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags)
+{
+ struct sxe2_rx_queue *rxq;
+ int32_t ret = 0;
+ uint16_t i;
+
+ *vec_flags = SXE2_RX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+ if (!rte_is_power_of_2(rxq->ring_depth)) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if (rxq->rx_free_thresh < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC &&
+ (rxq->ring_depth % rxq->rx_free_thresh) != 0) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((rxq->offloads & SXE2_RX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_RX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+ if ((rxq->offloads & offload) != 0) {
+ en = true;
+ goto l_end;
+ }
+ }
+l_end:
+ return en;
+}
+
+static inline void sxe2_rx_queue_mbufs_release_vec(struct sxe2_rx_queue *rxq)
+{
+ const uint16_t mask = rxq->ring_depth - 1;
+ uint16_t i;
+
+ if (unlikely(!rxq->buffer_ring)) {
+ PMD_LOG_DEBUG(RX, "Rx queue release mbufs vec, buffer_ring if NULL."
+ "port_id:%u queue_id:%u", rxq->port_id, rxq->queue_id);
+ return;
+ }
+ if (rxq->realloc_num >= rxq->ring_depth)
+ return;
+ if (rxq->realloc_num == 0) {
+ for (i = 0; i < rxq->ring_depth; ++i) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ } else {
+ for (i = rxq->processing_idx;
+ i != rxq->realloc_start;
+ i = (i + 1) & mask) {
+ if (rxq->buffer_ring[i]) {
+ rte_pktmbuf_free_seg(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ }
+ rxq->realloc_num = rxq->ring_depth;
+ memset(rxq->buffer_ring, 0, rxq->ring_depth * sizeof(rxq->buffer_ring[0]));
+}
+
+static inline void sxe2_rx_queue_vec_init(struct sxe2_rx_queue *rxq)
+{
+ uintptr_t data;
+ struct rte_mbuf mbuf_def;
+
+ memset(&mbuf_def, 0, sizeof(mbuf_def));
+ mbuf_def.buf_addr = 0;
+ mbuf_def.nb_segs = 1;
+ mbuf_def.data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf_def.port = rxq->port_id;
+ rte_mbuf_refcnt_set(&mbuf_def, 1);
+ rte_compiler_barrier();
+ data = (uintptr_t)&mbuf_def.rearm_data;
+ rxq->mbuf_init_value = *(uint64_t *)data;
+}
+
+int32_t __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_rx_queue *rxq = NULL;
+ int32_t ret = 0;
+ uint16_t i;
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL) {
+ PMD_LOG_INFO(RX, "Failed to prepare rx queue, rxq[%d] is NULL", i);
+ continue;
+ }
+ rxq->ops.mbufs_release = sxe2_rx_queue_mbufs_release_vec;
+ sxe2_rx_queue_vec_init(rxq);
+ }
+ return ret;
+}
+
+int32_t __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags)
+{
+ struct sxe2_tx_queue *txq;
+ int32_t ret = 0;
+ uint32_t i;
+ *vec_flags = SXE2_TX_MODE_VEC_SIMPLE;
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = (struct sxe2_tx_queue *)dev->data->tx_queues[i];
+ if (txq == NULL) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+ if (txq->rs_thresh < SXE2_TX_RS_THRESH_MIN_VEC ||
+ txq->rs_thresh > SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_NO_SUPPORT_OFFLOAD) != 0) {
+ ret = -ENOTSUP;
+ goto l_end;
+ }
+ if ((txq->offloads & SXE2_TX_VEC_SUPPORT_OFFLOAD) != 0)
+ *vec_flags = SXE2_TX_MODE_VEC_OFFLOAD;
+ }
+l_end:
+ return ret;
+}
+
+static void sxe2_tx_queue_mbufs_release_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ uint16_t i;
+
+ if (unlikely(txq == NULL || txq->buffer_ring == NULL)) {
+ PMD_LOG_ERR(TX, "Tx release mbufs vec, invalid params.");
+ return;
+ }
+ i = txq->next_dd - (txq->rs_thresh - 1);
+ buffer = txq->buffer_ring;
+ if (txq->next_use < i) {
+ for ( ; i < txq->ring_depth; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+ i = 0;
+ }
+ for (; i < txq->next_use; ++i) {
+ if (buffer[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(buffer[i].mbuf);
+ buffer[i].mbuf = NULL;
+ }
+ }
+}
+
+int32_t __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev)
+{
+ struct sxe2_tx_queue *txq = NULL;
+ int32_t ret = 0;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ txq = dev->data->tx_queues[i];
+ if (txq == NULL) {
+ PMD_LOG_INFO(TX, "Failed to prepare tx queue, txq[%d] is NULL", i);
+ continue;
+ }
+ txq->ops.mbufs_release = sxe2_tx_queue_mbufs_release_vec;
+ }
+ return ret;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx_vec.h b/drivers/net/sxe2/sxe2_txrx_vec.h
new file mode 100644
index 0000000000..ff2f2f1d5c
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef _SXE2_TXRX_VEC_H_
+#define _SXE2_TXRX_VEC_H_
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+
+#define SXE2_RX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_RX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_RX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_RX_MODE_VEC_AVX2 RTE_BIT32(3)
+#define SXE2_RX_MODE_VEC_AVX512 RTE_BIT32(4)
+#define SXE2_RX_MODE_VEC_NEON RTE_BIT32(5)
+#define SXE2_RX_MODE_BATCH_ALLOC RTE_BIT32(10)
+#define SXE2_RX_MODE_VEC_SET_MASK (SXE2_RX_MODE_VEC_SIMPLE | \
+ SXE2_RX_MODE_VEC_OFFLOAD | SXE2_RX_MODE_VEC_SSE | \
+ SXE2_RX_MODE_VEC_AVX2 | SXE2_RX_MODE_VEC_AVX512 | \
+ SXE2_RX_MODE_VEC_NEON)
+#define SXE2_TX_MODE_VEC_SIMPLE RTE_BIT32(0)
+#define SXE2_TX_MODE_VEC_OFFLOAD RTE_BIT32(1)
+#define SXE2_TX_MODE_VEC_SSE RTE_BIT32(2)
+#define SXE2_TX_MODE_VEC_AVX2 RTE_BIT32(3)
+#define SXE2_TX_MODE_VEC_AVX512 RTE_BIT32(4)
+#define SXE2_TX_MODE_VEC_NEON RTE_BIT32(5)
+#define SXE2_TX_MODE_SIMPLE_BATCH RTE_BIT32(10)
+#define SXE2_TX_MODE_VEC_SET_MASK (SXE2_TX_MODE_VEC_SIMPLE | \
+ SXE2_TX_MODE_VEC_OFFLOAD | SXE2_TX_MODE_VEC_SSE | \
+ SXE2_TX_MODE_VEC_AVX2 | SXE2_TX_MODE_VEC_AVX512 | \
+ SXE2_TX_MODE_VEC_NEON)
+#define SXE2_TX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_MULTI_SEGS | \
+ RTE_ETH_TX_OFFLOAD_QINQ_INSERT | \
+ RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_TSO | \
+ RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO | \
+ RTE_ETH_TX_OFFLOAD_SECURITY | \
+ RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM)
+#define SXE2_TX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_TX_OFFLOAD_VLAN_INSERT | \
+ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM)
+#define SXE2_RX_VEC_NO_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_TIMESTAMP | \
+ RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT | \
+ RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_SECURITY | \
+ RTE_ETH_RX_OFFLOAD_QINQ_STRIP)
+#define SXE2_RX_VEC_SUPPORT_OFFLOAD ( \
+ RTE_ETH_RX_OFFLOAD_CHECKSUM | \
+ RTE_ETH_RX_OFFLOAD_SCTP_CKSUM | \
+ RTE_ETH_RX_OFFLOAD_VLAN_STRIP | \
+ RTE_ETH_RX_OFFLOAD_VLAN_FILTER | \
+ RTE_ETH_RX_OFFLOAD_RSS_HASH)
+#ifdef RTE_ARCH_X86
+uint16_t sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+uint16_t sxe2_tx_pkts_vec_sse_simple(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+uint16_t sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts);
+#endif
+int32_t __rte_cold sxe2_tx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags);
+int32_t __rte_cold sxe2_tx_queues_vec_prepare(struct rte_eth_dev *dev);
+int32_t __rte_cold sxe2_rx_vec_support_check(struct rte_eth_dev *dev, uint32_t *vec_flags);
+bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload);
+int32_t __rte_cold sxe2_rx_queues_vec_prepare(struct rte_eth_dev *dev);
+
+#endif /* __SXE2_TXRX_VEC_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_common.h b/drivers/net/sxe2/sxe2_txrx_vec_common.h
new file mode 100644
index 0000000000..1ce687e09f
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_common.h
@@ -0,0 +1,235 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TXRX_VEC_COMMON_H__
+#define __SXE2_TXRX_VEC_COMMON_H__
+#include <rte_atomic.h>
+#ifdef PCLINT
+#include "avx_stub.h"
+#endif
+#include "sxe2_rx.h"
+#include "sxe2_queue.h"
+#include "sxe2_tx.h"
+#include "sxe2_vsi.h"
+#include "sxe2_ethdev.h"
+#define SXE2_RX_NUM_PER_LOOP_SSE 4
+#define SXE2_RX_NUM_PER_LOOP_AVX 8
+#define SXE2_RX_NUM_PER_LOOP_NEON 4
+#define SXE2_RX_REARM_THRESH_VEC 64
+#define SXE2_RX_PKTS_BURST_BATCH_NUM_VEC 32
+#define SXE2_TX_RS_THRESH_MIN_VEC 32
+#define SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC 64
+
+static __rte_always_inline void
+sxe2_tx_pkts_mbuf_fill(struct sxe2_tx_buffer *buffer,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ uint16_t i;
+ for (i = 0; i < nb_pkts; ++i)
+ buffer[i].mbuf = tx_pkts[i];
+}
+
+static __rte_always_inline int32_t
+sxe2_tx_bufs_free_vec(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX_VEC];
+ int32_t ret;
+ uint32_t i;
+ uint16_t rs_thresh;
+ uint16_t free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - (rs_thresh - 1)];
+ mbuf = rte_pktmbuf_prefree_seg(buffer[0].mbuf);
+ if (likely(mbuf)) {
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (likely(mbuf)) {
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline void
+sxe2_tx_desc_fill_offloads(struct rte_mbuf *mbuf, uint64_t *desc_qw1)
+{
+ uint64_t offloads = mbuf->ol_flags;
+ uint32_t desc_cmd = 0;
+ uint32_t desc_offset = 0;
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(mbuf->l3_len);
+ }
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(mbuf->l4_len);
+ break;
+ default:
+ break;
+ }
+ *desc_qw1 |= ((uint64_t)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ *desc_qw1 |= ((uint64_t)mbuf->vlan_tci) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT;
+ }
+ *desc_qw1 |= ((uint64_t)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT;
+}
+#define SXE2_RX_UMBCAST_FLAGS_VAL_GET(_flags) \
+ (((_flags) & 0x30) >> 4)
+
+static inline void sxe2_vf_rx_vec_sw_stats_cnt(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf *mbuf, uint8_t umbcast_flag)
+{
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN, rte_memory_order_relaxed);
+ switch (SXE2_RX_UMBCAST_FLAGS_VAL_GET(umbcast_flag)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static inline uint16_t
+sxe2_rx_pkts_refactor(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **mbuf_bufs, uint16_t mbuf_num,
+ uint8_t *split_rxe_flags, uint8_t *umbcast_flags)
+{
+ struct rte_mbuf *done_pkts[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ struct rte_mbuf *first_seg = rxq->pkt_first_seg;
+ struct rte_mbuf *last_seg = rxq->pkt_last_seg;
+ struct rte_mbuf *tmp_seg;
+ uint16_t done_num, buf_idx;
+ done_num = 0;
+ for (buf_idx = 0; buf_idx < mbuf_num; buf_idx++) {
+ if (last_seg) {
+ last_seg->next = mbuf_bufs[buf_idx];
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ first_seg->nb_segs++;
+ first_seg->pkt_len += mbuf_bufs[buf_idx]->data_len;
+ last_seg = last_seg->next;
+ if (split_rxe_flags[buf_idx] == 0) {
+ first_seg->hash = last_seg->hash;
+ first_seg->vlan_tci = last_seg->vlan_tci;
+ first_seg->ol_flags = last_seg->ol_flags;
+ first_seg->pkt_len -= rxq->crc_len;
+ if (last_seg->data_len > rxq->crc_len) {
+ last_seg->data_len -= rxq->crc_len;
+ } else {
+ tmp_seg = first_seg;
+ first_seg->nb_segs--;
+ while (tmp_seg->next != last_seg)
+ tmp_seg = tmp_seg->next;
+ tmp_seg->data_len -= (rxq->crc_len - last_seg->data_len);
+ tmp_seg->next = NULL;
+ rte_pktmbuf_free_seg(last_seg);
+ last_seg = NULL;
+ }
+ done_pkts[done_num++] = first_seg;
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, first_seg, umbcast_flags[buf_idx]);
+ first_seg = NULL;
+ last_seg = NULL;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ continue;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ last_seg = NULL;
+ continue;
+ }
+ } else {
+ if (split_rxe_flags[buf_idx] == 0) {
+ done_pkts[done_num++] = mbuf_bufs[buf_idx];
+ sxe2_vf_rx_vec_sw_stats_cnt(rxq, mbuf_bufs[buf_idx],
+ umbcast_flags[buf_idx]);
+ continue;
+ } else if (split_rxe_flags[buf_idx] & SXE2_RX_DESC_STATUS_EOP_MASK) {
+ first_seg = mbuf_bufs[buf_idx];
+ last_seg = first_seg;
+ mbuf_bufs[buf_idx]->data_len += rxq->crc_len;
+ mbuf_bufs[buf_idx]->pkt_len += rxq->crc_len;
+ } else {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ mbuf_bufs[buf_idx]->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free_seg(mbuf_bufs[buf_idx]);
+ continue;
+ }
+ }
+ }
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+ rte_memcpy(mbuf_bufs, done_pkts, done_num * (sizeof(struct rte_mbuf *)));
+ return done_num;
+}
+#endif /* __SXE2_TXRX_VEC_COMMON_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_vec_sse.c b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
new file mode 100644
index 0000000000..f6e3f45937
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_vec_sse.c
@@ -0,0 +1,549 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <ethdev_driver.h>
+#include <rte_bitops.h>
+#include <rte_malloc.h>
+#include <rte_mempool.h>
+#include <rte_vect.h>
+#include "rte_common.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_queue.h"
+#include "sxe2_txrx_vec.h"
+#include "sxe2_txrx_vec_common.h"
+#include "sxe2_vsi.h"
+
+static __rte_always_inline void
+sxe2_tx_desc_fill_one_sse(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf *pkt,
+ uint64_t desc_cmd, bool with_offloads)
+{
+ __m128i data_desc;
+ uint64_t desc_qw1;
+ uint32_t desc_offset;
+ desc_qw1 = (SXE2_TX_DESC_DTYPE_DATA |
+ ((uint64_t)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT |
+ ((uint64_t)pkt->data_len) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL(pkt->l2_len);
+ desc_qw1 |= ((uint64_t)desc_offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT;
+ if (with_offloads)
+ sxe2_tx_desc_fill_offloads(pkt, &desc_qw1);
+ data_desc = _mm_set_epi64x(desc_qw1, rte_pktmbuf_iova(pkt));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, desc), data_desc);
+}
+
+static __rte_always_inline uint16_t
+sxe2_tx_pkts_vec_sse_batch(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts, bool with_offloads)
+{
+ volatile union sxe2_tx_data_desc *desc;
+ struct sxe2_tx_buffer *buffer;
+ uint16_t next_use;
+ uint16_t res_num;
+ uint16_t tx_num;
+ uint16_t i;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free_vec(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx pkts sse batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ tx_num = nb_pkts;
+ next_use = txq->next_use;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ txq->desc_free_num -= nb_pkts;
+ res_num = txq->ring_depth - txq->next_use;
+ if (tx_num >= res_num) {
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, res_num);
+ for (i = 0; i < res_num - 1; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts++,
+ (SXE2_TX_DATA_DESC_CMD_EOP | SXE2_TX_DATA_DESC_CMD_RS),
+ with_offloads);
+ tx_num -= res_num;
+ next_use = 0;
+ txq->next_rs = txq->rs_thresh - 1;
+ desc = &txq->desc_ring[next_use];
+ buffer = &txq->buffer_ring[next_use];
+ }
+ sxe2_tx_pkts_mbuf_fill(buffer, tx_pkts, tx_num);
+ for (i = 0; i < tx_num; ++i, ++tx_pkts, ++desc) {
+ sxe2_tx_desc_fill_one_sse(desc, *tx_pkts,
+ SXE2_TX_DATA_DESC_CMD_EOP,
+ with_offloads);
+ }
+ next_use += tx_num;
+ if (next_use > txq->next_rs) {
+ txq->desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ }
+ txq->next_use = next_use;
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, nb_pkts);
+l_end:
+ return nb_pkts;
+}
+
+static __rte_always_inline uint16_t
+sxe2_tx_pkts_vec_sse_common(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts, bool with_offloads)
+{
+ uint16_t tx_done_num = 0;
+ uint16_t tx_once_num;
+ uint16_t tx_need_num;
+ while (nb_pkts) {
+ tx_need_num = RTE_MIN(nb_pkts, txq->rs_thresh);
+ tx_once_num = sxe2_tx_pkts_vec_sse_batch(txq,
+ tx_pkts + tx_done_num,
+ tx_need_num, with_offloads);
+ nb_pkts -= tx_once_num;
+ tx_done_num += tx_once_num;
+ if (tx_once_num < tx_need_num)
+ break;
+ }
+ return tx_done_num;
+}
+
+uint16_t sxe2_tx_pkts_vec_sse_simple(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, false);
+}
+uint16_t sxe2_tx_pkts_vec_sse(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ return sxe2_tx_pkts_vec_sse_common((struct sxe2_tx_queue *)tx_queue,
+ tx_pkts, nb_pkts, true);
+}
+
+static inline void sxe2_rx_queue_rearm_sse(struct sxe2_rx_queue *rxq)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ struct rte_mbuf *mbuf0, *mbuf1;
+ __m128i dma_addr0, dma_addr1;
+ __m128i virt_addr0, virt_addr1;
+ __m128i hdr_room = _mm_set_epi64x(RTE_PKTMBUF_HEADROOM,
+ RTE_PKTMBUF_HEADROOM);
+ int32_t ret;
+ uint16_t i;
+ uint16_t new_tail;
+
+ buffer = &rxq->buffer_ring[rxq->realloc_start];
+ desc = &rxq->desc_ring[rxq->realloc_start];
+ ret = rte_mempool_get_bulk(rxq->mb_pool, (void *)buffer,
+ SXE2_RX_REARM_THRESH_VEC);
+ if (ret != 0) {
+ PMD_LOG_INFO(RX, "Rx mbuf vec alloc failed port_id=%u "
+ "queue_id=%u", rxq->port_id, rxq->queue_id);
+ if ((rxq->realloc_num + SXE2_RX_REARM_THRESH_VEC) >= rxq->ring_depth) {
+ dma_addr0 = _mm_setzero_si128();
+ for (i = 0; i < SXE2_RX_NUM_PER_LOOP_SSE; ++i) {
+ buffer[i] = &rxq->fake_mbuf;
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc[i].read),
+ dma_addr0);
+ }
+ }
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed +=
+ SXE2_RX_REARM_THRESH_VEC;
+ goto l_end;
+ }
+ for (i = 0; i < SXE2_RX_REARM_THRESH_VEC; i += 2, buffer += 2) {
+ mbuf0 = buffer[0];
+ mbuf1 = buffer[1];
+#if RTE_IOVA_IN_MBUF
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) !=
+ offsetof(struct rte_mbuf, buf_addr) + 8);
+#endif
+ virt_addr0 = _mm_loadu_si128((__m128i *)&mbuf0->buf_addr);
+ virt_addr1 = _mm_loadu_si128((__m128i *)&mbuf1->buf_addr);
+#if RTE_IOVA_IN_MBUF
+ dma_addr0 = _mm_unpackhi_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpackhi_epi64(virt_addr1, virt_addr1);
+#else
+ dma_addr0 = _mm_unpacklo_epi64(virt_addr0, virt_addr0);
+ dma_addr1 = _mm_unpacklo_epi64(virt_addr1, virt_addr1);
+#endif
+ dma_addr0 = _mm_add_epi64(dma_addr0, hdr_room);
+ dma_addr1 = _mm_add_epi64(dma_addr1, hdr_room);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr0);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &desc++->read), dma_addr1);
+ }
+ rxq->realloc_start += SXE2_RX_REARM_THRESH_VEC;
+ if (rxq->realloc_start >= rxq->ring_depth)
+ rxq->realloc_start = 0;
+ rxq->realloc_num -= SXE2_RX_REARM_THRESH_VEC;
+ new_tail = (rxq->realloc_start == 0) ?
+ (rxq->ring_depth - 1) : (rxq->realloc_start - 1);
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, new_tail);
+l_end:
+ return;
+}
+
+static __rte_always_inline __m128i
+sxe2_rx_desc_fnav_flags_sse(__m128i descs_arr[4])
+{
+ __m128i descs_tmp1, descs_tmp2;
+ __m128i descs_fnav_vld;
+ __m128i v_zeros, v_ffff, v_u32_one;
+ __m128i m_flags;
+ const __m128i fdir_flags = _mm_set1_epi32(RTE_MBUF_F_RX_FDIR | RTE_MBUF_F_RX_FDIR_ID);
+ descs_tmp1 = _mm_unpacklo_epi32(descs_arr[0], descs_arr[1]);
+ descs_tmp2 = _mm_unpacklo_epi32(descs_arr[2], descs_arr[3]);
+ descs_fnav_vld = _mm_unpacklo_epi64(descs_tmp1, descs_tmp2);
+ descs_fnav_vld = _mm_slli_epi32(descs_fnav_vld, 26);
+ descs_fnav_vld = _mm_srli_epi32(descs_fnav_vld, 31);
+ v_zeros = _mm_setzero_si128();
+ v_ffff = _mm_cmpeq_epi32(v_zeros, v_zeros);
+ v_u32_one = _mm_srli_epi32(v_ffff, 31);
+ m_flags = _mm_cmpeq_epi32(descs_fnav_vld, v_u32_one);
+ m_flags = _mm_and_si128(m_flags, fdir_flags);
+ return m_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_desc_offloads_para_fill_sse(struct sxe2_rx_queue *rxq,
+ volatile union sxe2_rx_desc *desc __rte_unused,
+ __m128i descs_arr[4],
+ struct rte_mbuf **rx_pkts)
+{
+ const __m128i mbuf_init = _mm_set_epi64x(0, rxq->mbuf_init_value);
+ __m128i rearm_arr[4];
+ __m128i tmp_desc_lo, tmp_desc_hi, flags, tmp_flags;
+ const __m128i desc_flags_mask = _mm_set_epi32(0x00001C04, 0x00001C04,
+ 0x00001C04, 0x00001C04);
+ const __m128i desc_flags_rss_mask = _mm_set_epi32(0x20000000, 0x20000000,
+ 0x20000000, 0x20000000);
+ const __m128i vlan_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0,
+ 0, 0, 0, RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ 0, 0, 0, 0);
+ const __m128i rss_flags = _mm_set_epi8(0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, RTE_MBUF_F_RX_RSS_HASH,
+ 0, 0, 0, 0);
+ const __m128i cksum_flags =
+ _mm_set_epi8(0, 0, 0, 0, 0, 0, 0, 0,
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_BAD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_BAD) >> 1),
+ ((RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_IP_CKSUM_GOOD) >> 1));
+ const __m128i cksum_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD,
+ RTE_MBUF_F_RX_IP_CKSUM_MASK |
+ RTE_MBUF_F_RX_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK |
+ RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD);
+ const __m128i vlan_mask =
+ _mm_set_epi32(RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN |
+ RTE_MBUF_F_RX_VLAN_STRIPPED,
+ RTE_MBUF_F_RX_VLAN | RTE_MBUF_F_RX_VLAN_STRIPPED);
+ flags = _mm_unpackhi_epi32(descs_arr[0], descs_arr[1]);
+ tmp_flags = _mm_unpackhi_epi32(descs_arr[2], descs_arr[3]);
+ tmp_desc_lo = _mm_unpacklo_epi64(flags, tmp_flags);
+ tmp_desc_hi = _mm_unpackhi_epi64(flags, tmp_flags);
+ tmp_desc_lo = _mm_and_si128(tmp_desc_lo, desc_flags_mask);
+ tmp_desc_hi = _mm_and_si128(tmp_desc_hi, desc_flags_rss_mask);
+ tmp_flags = _mm_shuffle_epi8(vlan_flags, tmp_desc_lo);
+ flags = _mm_and_si128(tmp_flags, vlan_mask);
+ tmp_desc_lo = _mm_srli_epi32(tmp_desc_lo, 10);
+ tmp_flags = _mm_shuffle_epi8(cksum_flags, tmp_desc_lo);
+ tmp_flags = _mm_slli_epi32(tmp_flags, 1);
+ tmp_flags = _mm_and_si128(tmp_flags, cksum_mask);
+ flags = _mm_or_si128(flags, tmp_flags);
+ tmp_desc_hi = _mm_srli_epi32(tmp_desc_hi, 27);
+ tmp_flags = _mm_shuffle_epi8(rss_flags, tmp_desc_hi);
+ flags = _mm_or_si128(flags, tmp_flags);
+#ifndef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+ if (rxq->fnav_enable) {
+ __m128i tmp_fnav_flags = sxe2_rx_desc_fnav_flags_sse(descs_arr);
+ flags = _mm_or_si128(flags, tmp_fnav_flags);
+ rx_pkts[0]->hash.fdir.hi = desc[0].wb.fd_filter_id;
+ rx_pkts[1]->hash.fdir.hi = desc[1].wb.fd_filter_id;
+ rx_pkts[2]->hash.fdir.hi = desc[2].wb.fd_filter_id;
+ rx_pkts[3]->hash.fdir.hi = desc[3].wb.fd_filter_id;
+ }
+#endif
+ rearm_arr[0] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 8), 0x30);
+ rearm_arr[1] = _mm_blend_epi16(mbuf_init, _mm_slli_si128(flags, 4), 0x30);
+ rearm_arr[2] = _mm_blend_epi16(mbuf_init, flags, 0x30);
+ rearm_arr[3] = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x30);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) !=
+ offsetof(struct rte_mbuf, rearm_data) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) !=
+ RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16));
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[0]->rearm_data), rearm_arr[0]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[1]->rearm_data), rearm_arr[1]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[2]->rearm_data), rearm_arr[2]);
+ _mm_store_si128(RTE_CAST_PTR(__m128i *, &rx_pkts[3]->rearm_data), rearm_arr[3]);
+}
+
+static inline uint16_t
+sxe2_rx_pkts_common_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts, uint8_t *split_rxe_flags,
+ uint8_t *umbcast_flags)
+{
+ volatile union sxe2_rx_desc *desc;
+ struct rte_mbuf **buffer;
+ __m128i descs_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i mbuf_arr[SXE2_RX_NUM_PER_LOOP_SSE];
+ __m128i staterr, sterr_tmp1, sterr_tmp2;
+ __m128i pmbuf0;
+ __m128i ptype_all;
+#ifdef RTE_ARCH_X86_64
+ __m128i pmbuf1;
+#endif
+ uint32_t i;
+ uint32_t bit_num;
+ uint16_t done_num = 0;
+ const uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ const __m128i crc_adjust =
+ _mm_set_epi16(0, 0, 0,
+ -rxq->crc_len,
+ 0, -rxq->crc_len,
+ 0, 0);
+ const __m128i rvp_shuf_mask =
+ _mm_set_epi8(7, 6, 5, 4,
+ 3, 2,
+ 13, 12,
+ 0XFF, 0xFF, 13, 12,
+ 0xFF, 0xFF, 0xFF, 0xFF);
+ const __m128i dd_mask = _mm_set_epi64x(0x0000000100000001LL,
+ 0x0000000100000001LL);
+ const __m128i eop_mask = _mm_slli_epi32(dd_mask,
+ SXE2_RX_DESC_STATUS_EOP_SHIFT);
+ const __m128i rxe_mask = _mm_set_epi64x(0x0000208000002080LL,
+ 0x0000208000002080LL);
+ const __m128i eop_shuf_mask = _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x04, 0x0C,
+ 0x00, 0x08);
+ const __m128i ptype_mask = _mm_set_epi16(SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0,
+ SXE2_RX_DESC_PTYPE_MASK_NO_SHIFT, 0);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10);
+ RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) !=
+ offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12);
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ rte_prefetch0(desc);
+ nb_pkts = RTE_ALIGN_FLOOR(nb_pkts, SXE2_RX_NUM_PER_LOOP_SSE);
+ if (rxq->realloc_num > SXE2_RX_REARM_THRESH_VEC)
+ sxe2_rx_queue_rearm_sse(rxq);
+ if ((rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK) == 0)
+ goto l_end;
+ buffer = &rxq->buffer_ring[rxq->processing_idx];
+ for (i = 0; i < nb_pkts; i += SXE2_RX_NUM_PER_LOOP_SSE,
+ desc += SXE2_RX_NUM_PER_LOOP_SSE) {
+ pmbuf0 = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, &buffer[i]));
+ descs_arr[3] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 3));
+ rte_compiler_barrier();
+ _mm_storeu_si128((__m128i *)&rx_pkts[i], pmbuf0);
+#ifdef RTE_ARCH_X86_64
+ pmbuf1 = _mm_loadu_si128((__m128i *)&buffer[i + 2]);
+#endif
+ descs_arr[2] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 2));
+ rte_compiler_barrier();
+ descs_arr[1] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc + 1));
+ rte_compiler_barrier();
+ descs_arr[0] = _mm_loadu_si128(RTE_CAST_PTR(__m128i *, desc));
+#ifdef RTE_ARCH_X86_64
+ _mm_storeu_si128((__m128i *)&rx_pkts[i + 2], pmbuf1);
+#endif
+ if (split_rxe_flags) {
+ rte_mbuf_prefetch_part2(rx_pkts[i]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 1]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 2]);
+ rte_mbuf_prefetch_part2(rx_pkts[i + 3]);
+ }
+ rte_compiler_barrier();
+ mbuf_arr[3] = _mm_shuffle_epi8(descs_arr[3], rvp_shuf_mask);
+ mbuf_arr[2] = _mm_shuffle_epi8(descs_arr[2], rvp_shuf_mask);
+ mbuf_arr[1] = _mm_shuffle_epi8(descs_arr[1], rvp_shuf_mask);
+ mbuf_arr[0] = _mm_shuffle_epi8(descs_arr[0], rvp_shuf_mask);
+ sterr_tmp2 = _mm_unpackhi_epi32(descs_arr[3], descs_arr[2]);
+ sterr_tmp1 = _mm_unpackhi_epi32(descs_arr[1], descs_arr[0]);
+ sxe2_rx_desc_offloads_para_fill_sse(rxq, desc, descs_arr, rx_pkts);
+ mbuf_arr[3] = _mm_add_epi16(mbuf_arr[3], crc_adjust);
+ mbuf_arr[2] = _mm_add_epi16(mbuf_arr[2], crc_adjust);
+ mbuf_arr[1] = _mm_add_epi16(mbuf_arr[1], crc_adjust);
+ mbuf_arr[0] = _mm_add_epi16(mbuf_arr[0], crc_adjust);
+ staterr = _mm_unpacklo_epi32(sterr_tmp1, sterr_tmp2);
+ ptype_all = _mm_and_si128(staterr, ptype_mask);
+ _mm_storeu_si128((void *)&rx_pkts[i + 3]->rx_descriptor_fields1,
+ mbuf_arr[3]);
+ _mm_storeu_si128((void *)&rx_pkts[i + 2]->rx_descriptor_fields1,
+ mbuf_arr[2]);
+ if (umbcast_flags != NULL) {
+ const __m128i umbcast_mask =
+ _mm_set_epi32(SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK,
+ SXE2_RX_DESC_STATUS_UMBCAST_MASK);
+ const __m128i umbcast_shuf_mask =
+ _mm_set_epi8(0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0xFF, 0xFF,
+ 0x07, 0x0F,
+ 0x03, 0x0B);
+ __m128i umbcast_bits = _mm_and_si128(staterr, umbcast_mask);
+ umbcast_bits = _mm_shuffle_epi8(umbcast_bits, umbcast_shuf_mask);
+ *(int32_t *)umbcast_flags = _mm_cvtsi128_si32(umbcast_bits);
+ umbcast_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ if (split_rxe_flags != NULL) {
+ __m128i eop_bits = _mm_andnot_si128(staterr, eop_mask);
+ __m128i rxe_bits = _mm_and_si128(staterr, rxe_mask);
+ rxe_bits = _mm_srli_epi32(rxe_bits, 7);
+ eop_bits = _mm_or_si128(eop_bits, rxe_bits);
+ eop_bits = _mm_shuffle_epi8(eop_bits, eop_shuf_mask);
+ *(int32_t *)split_rxe_flags = _mm_cvtsi128_si32(eop_bits);
+ split_rxe_flags += SXE2_RX_NUM_PER_LOOP_SSE;
+ }
+ staterr = _mm_and_si128(staterr, dd_mask);
+ staterr = _mm_packs_epi32(staterr, _mm_setzero_si128());
+ _mm_storeu_si128((void *)&rx_pkts[i + 1]->rx_descriptor_fields1,
+ mbuf_arr[1]);
+ _mm_storeu_si128((void *)&rx_pkts[i]->rx_descriptor_fields1,
+ mbuf_arr[0]);
+ rx_pkts[i + 3]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 3)];
+ rx_pkts[i + 2]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 7)];
+ rx_pkts[i + 1]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 1)];
+ rx_pkts[i]->packet_type = ptype_tbl[_mm_extract_epi16(ptype_all, 5)];
+ bit_num = rte_popcount64(_mm_cvtsi128_si64(staterr));
+ done_num += bit_num;
+ if (likely(bit_num != SXE2_RX_NUM_PER_LOOP_SSE))
+ break;
+ }
+ rxq->processing_idx += done_num;
+ rxq->processing_idx &= (rxq->ring_depth - 1);
+ rxq->realloc_num += done_num;
+ PMD_LOG_DEBUG(RX, "port_id=%u queue_id=%u last_id=%u recv_pkts=%d",
+ rxq->port_id, rxq->queue_id, rxq->processing_idx, done_num);
+l_end:
+ return done_num;
+}
+
+static __rte_always_inline uint16_t
+sxe2_rx_pkts_scattered_batch_vec_sse(struct sxe2_rx_queue *rxq,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ const uint64_t *split_rxe_flags64;
+ uint8_t split_rxe_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ uint8_t umbcast_flags[SXE2_RX_PKTS_BURST_BATCH_NUM_VEC] = {0};
+ uint16_t rx_done_num;
+ uint16_t rx_pkt_done_num;
+ rx_pkt_done_num = 0;
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en) {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, umbcast_flags);
+ } else {
+ rx_done_num = sxe2_rx_pkts_common_vec_sse(rxq, rx_pkts,
+ nb_pkts, split_rxe_flags, NULL);
+ }
+ if (rx_done_num == 0)
+ goto l_end;
+ if (!rxq->vsi->adapter->devargs.sw_stats_en) {
+ split_rxe_flags64 = (uint64_t *)split_rxe_flags;
+ if (rxq->pkt_first_seg == NULL &&
+ split_rxe_flags64[0] == 0 &&
+ split_rxe_flags64[1] == 0 &&
+ split_rxe_flags64[2] == 0 &&
+ split_rxe_flags64[3] == 0) {
+ rx_pkt_done_num = rx_done_num;
+ goto l_end;
+ }
+ if (rxq->pkt_first_seg == NULL) {
+ while (rx_pkt_done_num < rx_done_num &&
+ split_rxe_flags[rx_pkt_done_num] == 0)
+ rx_pkt_done_num++;
+ if (rx_pkt_done_num == rx_done_num)
+ goto l_end;
+ rxq->pkt_first_seg = rx_pkts[rx_pkt_done_num];
+ }
+ }
+ rx_pkt_done_num += sxe2_rx_pkts_refactor(rxq, &rx_pkts[rx_pkt_done_num],
+ rx_done_num - rx_pkt_done_num, &split_rxe_flags[rx_pkt_done_num],
+ &umbcast_flags[rx_pkt_done_num]);
+l_end:
+ return rx_pkt_done_num;
+}
+
+uint16_t sxe2_rx_pkts_scattered_vec_sse_offload(void *rx_queue,
+ struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ uint16_t done_num = 0;
+ uint16_t once_num;
+
+ while (nb_pkts > SXE2_RX_PKTS_BURST_BATCH_NUM_VEC) {
+ once_num =
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num,
+ SXE2_RX_PKTS_BURST_BATCH_NUM_VEC);
+ done_num += once_num;
+ nb_pkts -= once_num;
+ if (once_num < SXE2_RX_PKTS_BURST_BATCH_NUM_VEC)
+ goto l_end;
+ }
+ done_num +=
+ sxe2_rx_pkts_scattered_batch_vec_sse((struct sxe2_rx_queue *)rx_queue,
+ rx_pkts + done_num, nb_pkts);
+l_end:
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
* [PATCH v15 01/11] mailmap: add Jie Liu
From: liujie5 @ 2026-05-16 7:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260516074618.2343883-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
.mailmap | 1 +
1 file changed, 1 insertion(+)
diff --git a/.mailmap b/.mailmap
index 895412e568..d2c4485636 100644
--- a/.mailmap
+++ b/.mailmap
@@ -739,6 +739,7 @@ Jiawen Wu <jiawenwu@trustnetic.com>
Jiayu Hu <hujiayu.hu@foxmail.com> <jiayu.hu@intel.com>
Jie Hai <haijie1@huawei.com>
Jie Liu <jie2.liu@hxt-semitech.com>
+Jie Liu <liujie5@linkdatatechnology.com>
Jie Pan <panjie5@jd.com>
Jie Wang <jie1x.wang@intel.com>
Jie Zhou <jizh@linux.microsoft.com> <jizh@microsoft.com>
--
2.47.3
^ permalink raw reply related
* [PATCH v15 09/11] drivers: add data path for Rx and Tx
From: liujie5 @ 2026-05-16 7:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260516074618.2343883-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Implement receive and transmit burst functions for sxe2 PMD.
Add sxe2_recv_pkts and sxe2_xmit_pkts as the primary data path
interfaces.
The implementation includes:
- Efficient descriptor fetching and mbuf allocation for Rx.
- Descriptor setup and checksum offload handling for Tx.
- Buffer recycling and hardware tail pointer updates.
- Performance-oriented loop unrolling and prefetching where applicable.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/common/sxe2/sxe2_ioctl_chnl.c | 8 +-
drivers/net/sxe2/meson.build | 2 +
drivers/net/sxe2/sxe2_ethdev.c | 13 +-
drivers/net/sxe2/sxe2_txrx.c | 246 +++++++
drivers/net/sxe2/sxe2_txrx.h | 21 +
drivers/net/sxe2/sxe2_txrx_poll.c | 919 ++++++++++++++++++++++++++
6 files changed, 1204 insertions(+), 5 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_txrx.c
create mode 100644 drivers/net/sxe2/sxe2_txrx.h
create mode 100644 drivers/net/sxe2/sxe2_txrx_poll.c
diff --git a/drivers/common/sxe2/sxe2_ioctl_chnl.c b/drivers/common/sxe2/sxe2_ioctl_chnl.c
index 08df9373d7..0522dc68b5 100644
--- a/drivers/common/sxe2/sxe2_ioctl_chnl.c
+++ b/drivers/common/sxe2/sxe2_ioctl_chnl.c
@@ -177,13 +177,13 @@ void
goto l_err;
}
- PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=0x%zx, src=0x%"PRIx64", offset=0x%"PRIx64"",
+ PMD_LOG_DEBUG(COM, "fd=%d, bar idx=%d, len=%"PRIu64", src=0x%"PRIx64", offset=0x%"PRIx64"",
bar_idx, cmd_fd, len, offset, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
virt = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_SHARED, cmd_fd, SXE2_COM_PCI_OFFSET_GEN(bar_idx, offset));
if (virt == MAP_FAILED) {
- PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=0x%zx, offset=0x%"PRIx64", err:%s",
+ PMD_LOG_ERR(COM, "Failed mmap, cmd_fd=%d, len=%"PRIu64", offset=0x%"PRIx64", err:%s",
cmd_fd, len, offset, strerror(errno));
goto l_err;
}
@@ -205,12 +205,12 @@ sxe2_drv_dev_munmap(struct sxe2_common_device *cdev, void *virt, uint64_t len)
goto l_end;
}
- PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%zx",
+ PMD_LOG_DEBUG(COM, "Munmap virt=%p, len=0x%"PRIx64"",
virt, len);
ret = munmap(virt, len);
if (ret < 0) {
- PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=0x%zx, err:%s",
+ PMD_LOG_ERR(COM, "Failed to munmap, virt=%p, len=%"PRIu64", err:%s",
virt, len, strerror(errno));
ret = -errno;
goto l_end;
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 3dfe54903a..5645e3ad61 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -20,6 +20,8 @@ sources += files(
'sxe2_queue.c',
'sxe2_tx.c',
'sxe2_rx.c',
+ 'sxe2_txrx_poll.c',
+ 'sxe2_txrx.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 6abb4672f6..8b76231057 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -26,6 +26,7 @@
#include "sxe2_cmd_chnl.h"
#include "sxe2_tx.h"
#include "sxe2_rx.h"
+#include "sxe2_txrx.h"
#include "sxe2_common.h"
#include "sxe2_common_log.h"
#include "sxe2_host_regs.h"
@@ -137,6 +138,9 @@ static int32_t sxe2_dev_start(struct rte_eth_dev *dev)
goto l_end;
}
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
+
ret = sxe2_queues_start(dev);
if (ret) {
PMD_LOG_ERR(INIT, "enable queues failed");
@@ -763,10 +767,17 @@ static int32_t sxe2_dev_init(struct rte_eth_dev *dev,
PMD_INIT_FUNC_TRACE();
+ sxe2_set_common_function(dev);
+
dev->dev_ops = &sxe2_eth_dev_ops;
- if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ sxe2_rx_mode_func_set(dev);
+ sxe2_tx_mode_func_set(dev);
+ if (ret != 0)
+ PMD_LOG_ERR(INIT, "Failed to mp init (secondary), ret=%d", ret);
goto l_end;
+ }
ret = sxe2_hw_init(dev);
if (ret) {
diff --git a/drivers/net/sxe2/sxe2_txrx.c b/drivers/net/sxe2/sxe2_txrx.c
new file mode 100644
index 0000000000..2531b49a52
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.c
@@ -0,0 +1,246 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include <unistd.h>
+
+#include "sxe2_txrx.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_ethdev.h"
+
+#include "sxe2_common_log.h"
+#include "sxe2_osal.h"
+#include "sxe2_cmd_chnl.h"
+#if defined(RTE_ARCH_ARM64)
+#include <rte_cpuflags.h>
+#endif
+
+static int32_t sxe2_tx_desciptor_status(void *tx_queue, uint16_t offset)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ int32_t ret;
+ uint16_t desc_idx;
+
+ if (unlikely(offset >= txq->ring_depth)) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ desc_idx = txq->next_use + offset;
+ desc_idx = DIV_ROUND_UP(desc_idx, txq->rs_thresh) * (txq->rs_thresh);
+ if (desc_idx >= txq->ring_depth) {
+ desc_idx -= txq->ring_depth;
+ if (desc_idx >= txq->ring_depth)
+ desc_idx -= txq->ring_depth;
+ }
+
+ if (desc_idx == 0)
+ desc_idx = txq->rs_thresh - 1;
+ else
+ desc_idx -= 1;
+
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) ==
+ (txq->desc_ring[desc_idx].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_MASK)))
+ ret = RTE_ETH_TX_DESC_DONE;
+ else
+ ret = RTE_ETH_TX_DESC_FULL;
+
+l_end:
+ return ret;
+}
+
+static inline int32_t sxe2_tx_mbuf_empty_check(struct rte_mbuf *mbuf)
+{
+ struct rte_mbuf *m_seg = mbuf;
+
+ while (m_seg != NULL) {
+ if (m_seg->data_len == 0)
+ return -EINVAL;
+ m_seg = m_seg->next;
+ }
+
+ return 0;
+}
+
+uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct rte_mbuf *mbuf;
+ uint64_t ol_flags = 0;
+ int32_t ret = 0;
+ int32_t i = 0;
+
+ for (i = 0; i < nb_pkts; i++) {
+ mbuf = tx_pkts[i];
+ if (!mbuf)
+ continue;
+ ol_flags = mbuf->ol_flags;
+ if (!(ol_flags & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))) {
+ if (mbuf->nb_segs > SXE2_TX_MTU_SEG_MAX ||
+ mbuf->pkt_len > SXE2_FRAME_SIZE_MAX) {
+ rte_errno = -EINVAL;
+ goto l_end;
+ }
+ } else if ((mbuf->tso_segsz < SXE2_MIN_TSO_MSS) ||
+ (mbuf->tso_segsz > SXE2_MAX_TSO_MSS) ||
+ (mbuf->nb_segs > txq->ring_depth) ||
+ (mbuf->pkt_len > SXE2_TX_TSO_PKTLEN_MAX)) {
+ rte_errno = -EINVAL;
+ goto l_end;
+ }
+
+ if (mbuf->pkt_len < SXE2_TX_MIN_PKT_LEN) {
+ rte_errno = -EINVAL;
+ goto l_end;
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_TX
+ ret = rte_validate_tx_offload(mbuf);
+ if (ret != 0) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+#endif
+ ret = rte_net_intel_cksum_prepare(mbuf);
+ if (ret != 0) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+
+ ret = sxe2_tx_mbuf_empty_check(mbuf);
+ if (ret != 0) {
+ rte_errno = -ret;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return i;
+}
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ uint32_t tx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+ dev->tx_pkt_burst = sxe2_tx_pkts;
+ adapter->q_ctxt.tx_mode_flags = tx_mode_flags;
+ PMD_LOG_DEBUG(TX, "Tx mode flags:0x%016x port_id:%u.",
+ tx_mode_flags, dev->data->port_id);
+}
+
+static int32_t sxe2_rx_desciptor_status(void *rx_queue, uint16_t offset)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ int32_t ret;
+
+ if (unlikely(offset >= rxq->ring_depth)) {
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (offset >= rxq->ring_depth - rxq->hold_num) {
+ ret = RTE_ETH_RX_DESC_UNAVAIL;
+ goto l_end;
+ }
+
+ if (rxq->processing_idx + offset >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + offset - rxq->ring_depth];
+ else
+ desc = &rxq->desc_ring[rxq->processing_idx + offset];
+
+ if (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) & SXE2_RX_DESC_STATUS_DD_MASK)
+ ret = RTE_ETH_RX_DESC_DONE;
+ else
+ ret = RTE_ETH_RX_DESC_AVAIL;
+
+l_end:
+ PMD_LOG_DEBUG(RX, "Rx queue desc[%u] status:%d queue_id:%u port_id:%u",
+ offset, ret, rxq->queue_id, rxq->port_id);
+ return ret;
+}
+
+static int32_t sxe2_rx_queue_count(void *rx_queue)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc;
+ uint16_t done_num = 0;
+
+ desc = &rxq->desc_ring[rxq->processing_idx];
+ while ((done_num < rxq->ring_depth) &&
+ (rte_le_to_cpu_64(desc->wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_DD_MASK)) {
+ done_num += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ if (rxq->processing_idx + done_num >= rxq->ring_depth)
+ desc = &rxq->desc_ring[rxq->processing_idx + done_num - rxq->ring_depth];
+ else
+ desc += SXE2_RX_QUEUE_CHECK_INTERVAL_NUM;
+ }
+
+ PMD_LOG_DEBUG(RX, "Rx queue done desc count:%u queue_id:%u port_id:%u",
+ done_num, rxq->queue_id, rxq->port_id);
+
+ return done_num;
+}
+
+static bool __rte_cold sxe2_rx_offload_en_check(struct rte_eth_dev *dev, uint64_t offload)
+{
+ struct sxe2_rx_queue *rxq;
+ bool en = false;
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ rxq = (struct sxe2_rx_queue *)dev->data->rx_queues[i];
+ if (rxq == NULL)
+ continue;
+
+ if (0 != (rxq->offloads & offload)) {
+ en = true;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return en;
+}
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ uint32_t rx_mode_flags = 0;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (sxe2_rx_offload_en_check(dev, RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT))
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered_split;
+ else
+ dev->rx_pkt_burst = sxe2_rx_pkts_scattered;
+
+ PMD_LOG_DEBUG(RX, "Rx mode flags:0x%016x port_id:%u.",
+ rx_mode_flags, dev->data->port_id);
+ adapter->q_ctxt.rx_mode_flags = rx_mode_flags;
+}
+
+void sxe2_set_common_function(struct rte_eth_dev *dev)
+{
+ PMD_INIT_FUNC_TRACE();
+
+ dev->rx_queue_count = sxe2_rx_queue_count;
+ dev->rx_descriptor_status = sxe2_rx_desciptor_status;
+
+ dev->tx_descriptor_status = sxe2_tx_desciptor_status;
+ dev->tx_pkt_prepare = sxe2_tx_pkts_prepare;
+}
diff --git a/drivers/net/sxe2/sxe2_txrx.h b/drivers/net/sxe2/sxe2_txrx.h
new file mode 100644
index 0000000000..f6558e2189
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef SXE2_TXRX_H
+#define SXE2_TXRX_H
+#include <ethdev_driver.h>
+#include "sxe2_queue.h"
+
+void sxe2_set_common_function(struct rte_eth_dev *dev);
+
+uint16_t sxe2_tx_pkts_prepare(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts);
+
+void sxe2_tx_mode_func_set(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq);
+
+void sxe2_rx_mode_func_set(struct rte_eth_dev *dev);
+
+#endif /* __SXE2_TXRX_H__ */
diff --git a/drivers/net/sxe2/sxe2_txrx_poll.c b/drivers/net/sxe2/sxe2_txrx_poll.c
new file mode 100644
index 0000000000..6d37fdef36
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_txrx_poll.c
@@ -0,0 +1,919 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include <unistd.h>
+
+#include "sxe2_osal.h"
+#include "sxe2_txrx_common.h"
+#include "sxe2_txrx_poll.h"
+#include "sxe2_txrx.h"
+#include "sxe2_queue.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+
+static __rte_always_inline int32_t
+sxe2_tx_bufs_free(struct sxe2_tx_queue *txq)
+{
+ struct sxe2_tx_buffer *buffer;
+ struct rte_mbuf *mbuf;
+ struct rte_mbuf *mbuf_free_arr[SXE2_TX_FREE_BUFFER_SIZE_MAX];
+ int32_t ret;
+ uint32_t i;
+ uint16_t rs_thresh;
+ uint16_t free_num;
+ if ((txq->desc_ring[txq->next_dd].wb.dd &
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK)) !=
+ rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE)) {
+ ret = 0;
+ goto l_end;
+ }
+ rs_thresh = txq->rs_thresh;
+ buffer = &txq->buffer_ring[txq->next_dd - rs_thresh + 1];
+ if (txq->offloads & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) {
+ if (likely(rs_thresh <= SXE2_TX_FREE_BUFFER_SIZE_MAX)) {
+ mbuf = buffer[0].mbuf;
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ for (i = 1; i < rs_thresh; ++i) {
+ mbuf = buffer[i].mbuf;
+ if (likely(mbuf->pool == mbuf_free_arr[0]->pool)) {
+ mbuf_free_arr[free_num] = mbuf;
+ free_num++;
+ } else {
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ mbuf_free_arr[0] = mbuf;
+ free_num = 1;
+ }
+ }
+ rte_mempool_put_bulk(mbuf_free_arr[0]->pool,
+ (void *)mbuf_free_arr, free_num);
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ rte_mempool_put(buffer->mbuf->pool, buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ } else {
+ for (i = 0; i < rs_thresh; ++i, ++buffer) {
+ mbuf = rte_pktmbuf_prefree_seg(buffer[i].mbuf);
+ if (mbuf != NULL)
+ rte_mempool_put(mbuf->pool, mbuf);
+ buffer->mbuf = NULL;
+ }
+ }
+ txq->desc_free_num += rs_thresh;
+ txq->next_dd += rs_thresh;
+ if (txq->next_dd >= txq->ring_depth)
+ txq->next_dd = rs_thresh - 1;
+ ret = rs_thresh;
+l_end:
+ return ret;
+}
+
+static inline int32_t sxe2_tx_cleanup(struct sxe2_tx_queue *txq)
+{
+ int32_t ret = 0;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ struct sxe2_tx_buffer *buffer_ring = txq->buffer_ring;
+ uint16_t ring_depth = txq->ring_depth;
+ uint16_t next_clean = txq->next_clean;
+ uint16_t clean_last;
+ uint16_t clean_num;
+
+ clean_last = next_clean + txq->rs_thresh;
+ if (clean_last >= ring_depth)
+ clean_last = clean_last - ring_depth;
+
+ clean_last = buffer_ring[clean_last].last_id;
+ if (rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE) !=
+ (txq->desc_ring[clean_last].wb.dd & rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_MASK))) {
+ PMD_LOG_DEBUG(TX, "desc[%u] is not done.port_id=%u queue_id=%u val=0x%" PRIx64,
+ clean_last, txq->port_id,
+ txq->queue_id, txq->desc_ring[clean_last].wb.dd);
+ ret = -1;
+ goto l_end;
+ }
+
+ if (clean_last > next_clean)
+ clean_num = clean_last - next_clean;
+ else
+ clean_num = ring_depth - next_clean + clean_last;
+
+ desc_ring[clean_last].wb.dd = 0;
+
+ txq->next_clean = clean_last;
+ txq->desc_free_num += clean_num;
+
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+static __rte_always_inline uint16_t
+sxe2_tx_pkt_data_desc_count(struct rte_mbuf *tx_pkt)
+{
+ struct rte_mbuf *m_seg = tx_pkt;
+ uint16_t count = 0;
+
+ while (m_seg != NULL) {
+ count += DIV_ROUND_UP(m_seg->data_len,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC);
+ m_seg = m_seg->next;
+ }
+
+ return count;
+}
+
+static __rte_always_inline void
+sxe2_tx_desc_checksum_fill(uint64_t offloads, uint32_t *desc_cmd, uint32_t *desc_offset,
+ union sxe2_tx_offload_info ol_info)
+{
+ if (offloads & RTE_MBUF_F_TX_IP_CKSUM) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4_CSUM;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV4) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV4;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ } else if (offloads & RTE_MBUF_F_TX_IPV6) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_IIPT_IPV6;
+ *desc_offset |= SXE2_TX_DATA_DESC_IPLEN_VAL(ol_info.l3_len);
+ }
+
+ if (offloads & RTE_MBUF_F_TX_TCP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_UDP_SEG) {
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ goto l_end;
+ }
+
+ switch (offloads & RTE_MBUF_F_TX_L4_MASK) {
+ case RTE_MBUF_F_TX_TCP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_TCP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_SCTP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_SCTP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ case RTE_MBUF_F_TX_UDP_CKSUM:
+ *desc_cmd |= SXE2_TX_DATA_DESC_CMD_L4T_EOFT_UDP;
+ *desc_offset |= SXE2_TX_DATA_DESC_L4LEN_VAL(ol_info.l4_len);
+ break;
+ default:
+
+ break;
+ }
+
+l_end:
+ return;
+}
+
+static __rte_always_inline uint64_t
+sxe2_tx_data_desc_build_cobt(uint32_t cmd, uint32_t offset, uint16_t buf_size, uint16_t l2tag)
+{
+ return rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DATA |
+ (((uint64_t)cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT) |
+ (((uint64_t)offset) << SXE2_TX_DATA_DESC_OFFSET_SHIFT) |
+ (((uint64_t)buf_size) << SXE2_TX_DATA_DESC_BUF_SZ_SHIFT) |
+ (((uint64_t)l2tag) << SXE2_TX_DATA_DESC_L2TAG1_SHIFT));
+}
+
+uint16_t sxe2_tx_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_queue *txq = tx_queue;
+ struct sxe2_tx_buffer *buffer_ring;
+ struct sxe2_tx_buffer *buffer;
+ struct sxe2_tx_buffer *next_buffer;
+ struct rte_mbuf *tx_pkt;
+ struct rte_mbuf *m_seg;
+ volatile union sxe2_tx_data_desc *desc_ring;
+ volatile union sxe2_tx_data_desc *desc;
+ volatile struct sxe2_tx_context_desc *ctxt_desc;
+ union sxe2_tx_offload_info ol_info;
+ struct sxe2_vsi *vsi = txq->vsi;
+ rte_iova_t buf_dma_addr;
+ uint64_t offloads;
+ uint64_t desc_type_cmd_tso_mss;
+ uint32_t desc_cmd;
+ uint32_t desc_offset;
+ uint32_t desc_tag;
+ uint32_t desc_tunneling_params;
+ uint16_t ipsec_offset;
+ uint16_t ctxt_desc_num;
+ uint16_t desc_sum_num;
+ uint16_t tx_num;
+ uint16_t seg_len;
+ uint16_t next_use;
+ uint16_t last_use;
+ uint16_t desc_l2tag2;
+
+ buffer_ring = txq->buffer_ring;
+ desc_ring = txq->desc_ring;
+ next_use = txq->next_use;
+ buffer = &buffer_ring[next_use];
+
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_cleanup(txq);
+
+ for (tx_num = 0; tx_num < nb_pkts; tx_num++) {
+ tx_pkt = *tx_pkts++;
+ desc_cmd = 0;
+ desc_offset = 0;
+ desc_tag = 0;
+ desc_tunneling_params = 0;
+ ipsec_offset = 0;
+ offloads = tx_pkt->ol_flags;
+ ol_info.l2_len = tx_pkt->l2_len;
+ ol_info.l3_len = tx_pkt->l3_len;
+ ol_info.l4_len = tx_pkt->l4_len;
+ ol_info.tso_segsz = tx_pkt->tso_segsz;
+ ol_info.outer_l2_len = tx_pkt->outer_l2_len;
+ ol_info.outer_l3_len = tx_pkt->outer_l3_len;
+
+ ctxt_desc_num = (offloads &
+ SXE2_TX_OFFLOAD_CTXT_NEEDCK_MASK) ? 1 : 0;
+ if (unlikely(vsi->vsi_type == SXE2_VSI_T_DPDK_ESW))
+ ctxt_desc_num = 1;
+
+ if (offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG))
+ desc_sum_num = sxe2_tx_pkt_data_desc_count(tx_pkt) + ctxt_desc_num;
+ else
+ desc_sum_num = tx_pkt->nb_segs + ctxt_desc_num;
+
+ last_use = next_use + desc_sum_num - 1;
+ if (last_use >= txq->ring_depth)
+ last_use = last_use - txq->ring_depth;
+
+ if (desc_sum_num > txq->desc_free_num) {
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_exit_logic;
+
+ if (unlikely(desc_sum_num > txq->rs_thresh)) {
+ while (desc_sum_num > txq->desc_free_num)
+ if (unlikely(sxe2_tx_cleanup(txq) != 0))
+ goto l_exit_logic;
+ }
+ }
+
+ desc_offset |= SXE2_TX_DATA_DESC_MACLEN_VAL(ol_info.l2_len);
+
+ if (offloads & SXE2_TX_OFFLOAD_CKSUM_MASK) {
+ sxe2_tx_desc_checksum_fill(offloads, &desc_cmd,
+ &desc_offset, ol_info);
+ }
+
+ if (offloads & (RTE_MBUF_F_TX_VLAN | RTE_MBUF_F_TX_QINQ)) {
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_IL2TAG1;
+ desc_tag = tx_pkt->vlan_tci;
+ }
+
+ if (ctxt_desc_num) {
+ ctxt_desc = (volatile struct sxe2_tx_context_desc *)
+ &desc_ring[next_use];
+ desc_l2tag2 = 0;
+ desc_type_cmd_tso_mss = SXE2_TX_DESC_DTYPE_CTXT;
+
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ if (offloads & RTE_MBUF_F_TX_QINQ) {
+ desc_l2tag2 = tx_pkt->vlan_tci_outer;
+ desc_type_cmd_tso_mss |= SXE2_TX_CTXT_DESC_CMD_IL2TAG2_MASK;
+ }
+
+ ctxt_desc->tunneling_params =
+ rte_cpu_to_le_32(desc_tunneling_params);
+ ctxt_desc->l2tag2 = rte_cpu_to_le_16(desc_l2tag2);
+ ctxt_desc->type_cmd_tso_mss = rte_cpu_to_le_64(desc_type_cmd_tso_mss);
+ ctxt_desc->ipsec_offset = rte_cpu_to_le_64(ipsec_offset);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ }
+
+ m_seg = tx_pkt;
+
+ do {
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ if (buffer->mbuf) {
+ rte_pktmbuf_free_seg(buffer->mbuf);
+ buffer->mbuf = NULL;
+ }
+
+ buffer->mbuf = m_seg;
+ seg_len = m_seg->data_len;
+ buf_dma_addr = rte_mbuf_data_iova(m_seg);
+ while ((offloads & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG)) &&
+ unlikely(seg_len > SXE2_TX_MAX_DATA_NUM_PER_DESC)) {
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd, desc_offset,
+ SXE2_TX_MAX_DATA_NUM_PER_DESC,
+ desc_tag);
+ buf_dma_addr += SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ seg_len -= SXE2_TX_MAX_DATA_NUM_PER_DESC;
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+ desc = &desc_ring[next_use];
+ next_buffer = &buffer_ring[buffer->next_id];
+ RTE_MBUF_PREFETCH_TO_FREE(next_buffer->mbuf);
+ }
+
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(desc_cmd,
+ desc_offset, seg_len, desc_tag);
+
+ buffer->last_id = last_use;
+ next_use = buffer->next_id;
+ buffer = next_buffer;
+
+ m_seg = m_seg->next;
+ } while (m_seg);
+
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_EOP;
+ txq->desc_used_num += desc_sum_num;
+ txq->desc_free_num -= desc_sum_num;
+
+ if (txq->desc_used_num >= txq->rs_thresh) {
+ PMD_LOG_DEBUG(TX, "Tx pkts set RS bit."
+ "last_use=%u port_id=%u, queue_id=%u",
+ last_use, txq->port_id, txq->queue_id);
+ desc_cmd |= SXE2_TX_DATA_DESC_CMD_RS;
+ txq->desc_used_num = 0;
+ }
+
+ desc->read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(((uint64_t)desc_cmd) << SXE2_TX_DATA_DESC_CMD_SHIFT);
+ }
+
+l_exit_logic:
+ if (tx_num == 0)
+ goto l_end;
+ goto l_end_of_tx;
+l_end_of_tx:
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, next_use);
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, next_use, tx_num);
+
+ txq->next_use = next_use;
+
+l_end:
+ return tx_num;
+}
+
+static __rte_always_inline void
+sxe2_tx_data_desc_fill(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ uint32_t desc_offset;
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset, (*tx_pkts)->data_len, 0);
+}
+static __rte_always_inline void
+sxe2_tx_data_desc_fill_batch(volatile union sxe2_tx_data_desc *desc,
+ struct rte_mbuf **tx_pkts)
+{
+ rte_iova_t buf_dma_addr;
+ uint32_t i;
+ uint32_t desc_offset;
+ for (i = 0; i < SXE2_TX_FILL_PER_LOOP; ++i, ++desc, ++tx_pkts) {
+ buf_dma_addr = rte_mbuf_data_iova(*tx_pkts);
+ desc->read.buf_addr = rte_cpu_to_le_64(buf_dma_addr);
+ desc_offset = SXE2_TX_DATA_DESC_MACLEN_VAL((*tx_pkts)->l2_len);
+ desc->read.type_cmd_off_bsz_l2t =
+ sxe2_tx_data_desc_build_cobt(SXE2_TX_DATA_DESC_CMD_EOP,
+ desc_offset,
+ (*tx_pkts)->data_len,
+ 0);
+ }
+}
+
+static inline void sxe2_tx_ring_fill(struct sxe2_tx_queue *txq,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_buffer *buffer = &txq->buffer_ring[txq->next_use];
+ volatile union sxe2_tx_data_desc *desc = &txq->desc_ring[txq->next_use];
+ uint32_t i, j;
+ uint32_t mainpart;
+ uint32_t leftover;
+ mainpart = nb_pkts & ((uint32_t)~SXE2_TX_FILL_PER_LOOP_MASK);
+ leftover = nb_pkts & ((uint32_t)SXE2_TX_FILL_PER_LOOP_MASK);
+ for (i = 0; i < mainpart; i += SXE2_TX_FILL_PER_LOOP) {
+ for (j = 0; j < SXE2_TX_FILL_PER_LOOP; ++j)
+ (buffer + i + j)->mbuf = *(tx_pkts + i + j);
+ sxe2_tx_data_desc_fill_batch(desc + i, tx_pkts + i);
+ }
+ if (unlikely(leftover > 0)) {
+ for (i = 0; i < leftover; ++i) {
+ (buffer + mainpart + i)->mbuf = *(tx_pkts + mainpart + i);
+ sxe2_tx_data_desc_fill(desc + mainpart + i,
+ tx_pkts + mainpart + i);
+ }
+ }
+}
+
+static inline uint16_t sxe2_tx_pkts_batch(void *tx_queue,
+ struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_tx_queue *txq = (struct sxe2_tx_queue *)tx_queue;
+ volatile union sxe2_tx_data_desc *desc_ring = txq->desc_ring;
+ uint16_t res_num = 0;
+ if (txq->desc_free_num < txq->free_thresh)
+ (void)sxe2_tx_bufs_free(txq);
+ nb_pkts = RTE_MIN(txq->desc_free_num, nb_pkts);
+ if (unlikely(nb_pkts == 0)) {
+ PMD_LOG_DEBUG(TX, "Tx batch: may not enough free desc, "
+ "free_desc=%u, need_tx_pkts=%u",
+ txq->desc_free_num, nb_pkts);
+ goto l_end;
+ }
+ txq->desc_free_num -= nb_pkts;
+ if ((txq->next_use + nb_pkts) > txq->ring_depth) {
+ res_num = txq->ring_depth - txq->next_use;
+ sxe2_tx_ring_fill(txq, tx_pkts, res_num);
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs = txq->rs_thresh - 1;
+ txq->next_use = 0;
+ }
+ sxe2_tx_ring_fill(txq, tx_pkts + res_num, nb_pkts - res_num);
+ txq->next_use = txq->next_use + (nb_pkts - res_num);
+ if (txq->next_use > txq->next_rs) {
+ desc_ring[txq->next_rs].read.type_cmd_off_bsz_l2t |=
+ rte_cpu_to_le_64(SXE2_TX_DATA_DESC_CMD_RS_MASK);
+ txq->next_rs += txq->rs_thresh;
+ if (txq->next_rs >= txq->ring_depth)
+ txq->next_rs = txq->rs_thresh - 1;
+ }
+ if (txq->next_use >= txq->ring_depth)
+ txq->next_use = 0;
+ PMD_LOG_DEBUG(TX, "port_id=%u queue_id=%u next_use=%u send_pkts=%u",
+ txq->port_id, txq->queue_id, txq->next_use, nb_pkts);
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, txq->next_use);
+l_end:
+ return nb_pkts;
+}
+
+static inline void
+sxe2_update_rx_tail(struct sxe2_rx_queue *rxq, uint16_t hold_num, uint16_t rx_id)
+{
+ hold_num += rxq->hold_num;
+
+ if (hold_num > rxq->rx_free_thresh) {
+ rx_id = (uint16_t)((rx_id == 0) ? (rxq->ring_depth - 1) : (rx_id - 1));
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, rx_id);
+ hold_num = 0;
+ }
+ rxq->hold_num = hold_num;
+}
+
+static inline uint64_t
+sxe2_rx_desc_error_para(__rte_unused struct sxe2_rx_queue *rxq,
+ union sxe2_rx_desc *desc)
+{
+ uint64_t flags = 0;
+ uint64_t desc_qw1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (unlikely(0 == (desc_qw1 & SXE2_RX_DESC_STATUS_L3L4_P_MASK)))
+ goto l_end;
+
+ if (likely(0 == (desc->wb.rxdid_src & SXE2_RX_DESC_EUDPE_MASK)))
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD;
+ else
+ flags = RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_QW1_ERRORS_MASK))) {
+ flags |= (RTE_MBUF_F_RX_IP_CKSUM_GOOD |
+ RTE_MBUF_F_RX_L4_CKSUM_GOOD |
+ RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD);
+ goto l_end;
+ }
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_IPE_MASK)))
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+
+ if (likely(0 == (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_L4_MASK)))
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_GOOD;
+ else
+ flags |= RTE_MBUF_F_RX_L4_CKSUM_BAD;
+
+ if (unlikely(0 != (desc_qw1 & SXE2_RX_DESC_ERROR_CSUM_EIP_MASK)))
+ flags |= RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD;
+
+l_end:
+ return flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_mbuf_common_fields_fill(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
+ uint64_t qword1;
+ uint64_t pkt_flags;
+ qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+
+ mbuf->ol_flags = 0;
+ mbuf->packet_type = ptype_tbl[SXE2_RX_DESC_PTYPE_VAL_GET(qword1)];
+
+ pkt_flags = sxe2_rx_desc_error_para(rxq, rxd);
+
+ mbuf->ol_flags |= pkt_flags;
+}
+
+static __rte_always_inline void
+sxe2_rx_sw_stats_update(struct sxe2_rx_queue *rxq, struct rte_mbuf *mbuf,
+ union sxe2_rx_desc *rxd)
+{
+ uint64_t qword1 = rte_le_to_cpu_64(rxd->wb.status_err_ptype_len);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.bytes,
+ mbuf->pkt_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ switch (SXE2_RX_DESC_STATUS_UMBCAST_VAL_GET(qword1)) {
+ case SXE2_RX_DESC_STATUS_UNICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.unicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_MUTICAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.multicast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ case SXE2_RX_DESC_STATUS_BOARDCAST:
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.broadcast_pkts, 1,
+ rte_memory_order_relaxed);
+ break;
+ default:
+ break;
+ }
+}
+
+uint16_t sxe2_rx_pkts_scattered(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ uint64_t qword1;
+ uint16_t done_num;
+ uint16_t hold_num;
+ uint16_t cur_idx;
+ uint16_t pkt_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ PMD_LOG_INFO(RX, "Rx new_mbuf alloc failed port_id:%u "
+ "queue_id:%u", rxq->port_id, rxq->queue_id);
+ break;
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+
+ rte_prefetch0(buffer_ring[cur_idx]);
+
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+
+ cur_mbuf = *cur_buffer;
+
+ *cur_buffer = new_mbuf;
+
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+
+ if (0 == (qword1 & SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len + pkt_len -
+ RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+ first_seg->port = rxq->port_id;
+
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
+
+uint16_t sxe2_rx_pkts_scattered_split(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
+{
+ struct sxe2_rx_queue *rxq = (struct sxe2_rx_queue *)rx_queue;
+ volatile union sxe2_rx_desc *desc_ring;
+ volatile union sxe2_rx_desc *desc;
+ union sxe2_rx_desc desc_tmp;
+ struct rte_mbuf **buffer_ring;
+ struct rte_mbuf **cur_buffer;
+ struct rte_mbuf *cur_mbuf;
+ struct rte_mbuf *cur_mbuf_pay;
+ struct rte_mbuf *new_mbuf;
+ struct rte_mbuf *new_mbuf_pay = NULL;
+ struct rte_mbuf *first_seg;
+ struct rte_mbuf *last_seg;
+ uint64_t qword1;
+ uint16_t done_num;
+ uint16_t hold_num;
+ uint16_t cur_idx;
+ uint16_t pkt_len;
+ uint16_t hdr_len;
+
+ desc_ring = rxq->desc_ring;
+ buffer_ring = rxq->buffer_ring;
+ cur_idx = rxq->processing_idx;
+ first_seg = rxq->pkt_first_seg;
+ last_seg = rxq->pkt_last_seg;
+ done_num = 0;
+ hold_num = 0;
+ new_mbuf = NULL;
+
+ while (done_num < nb_pkts) {
+ desc = &desc_ring[cur_idx];
+ qword1 = rte_le_to_cpu_64(desc->wb.status_err_ptype_len);
+
+ if (0 == (SXE2_RX_DESC_STATUS_DD_MASK & qword1))
+ break;
+
+ if ((rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) == 0 ||
+ first_seg == NULL) {
+ new_mbuf = rte_mbuf_raw_alloc(rxq->mb_pool);
+ if (unlikely(new_mbuf == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ break;
+ }
+ }
+
+ if (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) {
+ new_mbuf_pay = rte_mbuf_raw_alloc(rxq->rx_seg[1].mp);
+ if (unlikely(new_mbuf_pay == NULL)) {
+ rxq->vsi->adapter->dev_info.dev_data->rx_mbuf_alloc_failed++;
+ if (new_mbuf != NULL)
+ rte_pktmbuf_free(new_mbuf);
+ new_mbuf = NULL;
+ break;
+ }
+ }
+
+ hold_num++;
+ desc_tmp = *desc;
+ cur_buffer = &buffer_ring[cur_idx];
+ cur_idx++;
+ if (unlikely(cur_idx == rxq->ring_depth))
+ cur_idx = 0;
+ rte_prefetch0(buffer_ring[cur_idx]);
+ if (0 == (cur_idx & 0x3)) {
+ rte_prefetch0(&desc_ring[cur_idx]);
+ rte_prefetch0(&buffer_ring[cur_idx]);
+ }
+ cur_mbuf = *cur_buffer;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ *cur_buffer = new_mbuf;
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ } else {
+ if (first_seg == NULL) {
+ *cur_buffer = new_mbuf;
+ new_mbuf->next = new_mbuf_pay;
+ new_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ } else {
+ cur_mbuf_pay = cur_mbuf->next;
+ new_mbuf_pay->next = NULL;
+ new_mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ cur_mbuf->next = new_mbuf_pay;
+ desc->read.hdr_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(cur_mbuf));
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(new_mbuf_pay));
+ cur_mbuf = cur_mbuf_pay;
+ }
+ }
+ new_mbuf = NULL;
+ if (0 == (rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+ cur_mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ if (first_seg == NULL) {
+ first_seg = cur_mbuf;
+ first_seg->nb_segs = 1;
+ first_seg->pkt_len = pkt_len;
+ } else {
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ } else {
+ if (first_seg == NULL) {
+ cur_mbuf->nb_segs = 2;
+ cur_mbuf->next->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ hdr_len = SXE2_RX_DESC_HDR_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = hdr_len;
+ cur_mbuf->pkt_len = hdr_len + pkt_len;
+ cur_mbuf->next->data_len = pkt_len;
+ first_seg = cur_mbuf;
+ cur_mbuf = cur_mbuf->next;
+ last_seg = cur_mbuf;
+ } else {
+ cur_mbuf->nb_segs = 1;
+ cur_mbuf->next = NULL;
+ pkt_len = SXE2_RX_DESC_PKT_LEN_VAL_GET(qword1);
+ cur_mbuf->data_len = pkt_len;
+
+ first_seg->pkt_len += pkt_len;
+ first_seg->nb_segs++;
+ last_seg->next = cur_mbuf;
+ }
+ }
+
+#ifdef RTE_ETHDEV_DEBUG_RX
+
+ rte_pktmbuf_dump(stdout, first_seg, rte_pktmbuf_pkt_len(first_seg));
+#endif
+
+ if (0 == (rte_le_to_cpu_64(desc_tmp.wb.status_err_ptype_len) &
+ SXE2_RX_DESC_STATUS_EOP_MASK)) {
+ last_seg = cur_mbuf;
+ continue;
+ }
+
+ if (unlikely(qword1 & SXE2_RX_DESC_ERROR_RXE_MASK) ||
+ unlikely(qword1 & SXE2_RX_DESC_ERROR_OVERSIZE_MASK)) {
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_pkts, 1,
+ rte_memory_order_relaxed);
+ rte_atomic_fetch_add_explicit(&rxq->sw_stats.drop_bytes,
+ first_seg->pkt_len - rxq->crc_len + RTE_ETHER_CRC_LEN,
+ rte_memory_order_relaxed);
+ rte_pktmbuf_free(first_seg);
+ first_seg = NULL;
+ continue;
+ }
+
+ cur_mbuf->next = NULL;
+ if (unlikely(rxq->crc_len > 0)) {
+ first_seg->pkt_len -= RTE_ETHER_CRC_LEN;
+ if (pkt_len <= RTE_ETHER_CRC_LEN) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->data_len = last_seg->data_len +
+ pkt_len - RTE_ETHER_CRC_LEN;
+ last_seg->next = NULL;
+ } else {
+ cur_mbuf->data_len = pkt_len - RTE_ETHER_CRC_LEN;
+ }
+ } else if (pkt_len == 0) {
+ rte_pktmbuf_free_seg(cur_mbuf);
+ cur_mbuf = NULL;
+ first_seg->nb_segs--;
+ last_seg->next = NULL;
+ }
+
+ first_seg->port = rxq->port_id;
+ sxe2_rx_mbuf_common_fields_fill(rxq, first_seg, &desc_tmp);
+
+ if (rxq->vsi->adapter->devargs.sw_stats_en)
+ sxe2_rx_sw_stats_update(rxq, first_seg, &desc_tmp);
+
+ rte_prefetch0(RTE_PTR_ADD(first_seg->buf_addr, first_seg->data_off));
+
+ rx_pkts[done_num] = first_seg;
+ done_num++;
+
+ first_seg = NULL;
+ }
+
+ rxq->processing_idx = cur_idx;
+ rxq->pkt_first_seg = first_seg;
+ rxq->pkt_last_seg = last_seg;
+
+ sxe2_update_rx_tail(rxq, hold_num, cur_idx);
+
+ return done_num;
+}
--
2.47.3
^ permalink raw reply related
* [PATCH v15 08/11] net/sxe2: support queue setup and control
From: liujie5 @ 2026-05-16 7:46 UTC (permalink / raw)
To: stephen; +Cc: dev, Jie Liu
In-Reply-To: <20260516074618.2343883-1-liujie5@linkdatatechnology.com>
From: Jie Liu <liujie5@linkdatatechnology.com>
Add support for Rx and Tx queue setup, release, and management.
Implement eth_dev_ops callbacks for rx_queue_setup, tx_queue_setup,
rx_queue_release, and tx_queue_release.
This includes:
- Allocating memory for hardware ring descriptors.
- Initializing software ring structures and hardware head/tail pointers.
- Implementing proper resource cleanup logic to prevent memory leaks
during queue reconfiguration or device close.
Signed-off-by: Jie Liu <liujie5@linkdatatechnology.com>
---
drivers/net/sxe2/meson.build | 2 +
drivers/net/sxe2/sxe2_ethdev.c | 82 +++--
drivers/net/sxe2/sxe2_ethdev.h | 15 +-
drivers/net/sxe2/sxe2_rx.c | 559 +++++++++++++++++++++++++++++++++
drivers/net/sxe2/sxe2_rx.h | 34 ++
drivers/net/sxe2/sxe2_tx.c | 420 +++++++++++++++++++++++++
drivers/net/sxe2/sxe2_tx.h | 32 ++
7 files changed, 1118 insertions(+), 26 deletions(-)
create mode 100644 drivers/net/sxe2/sxe2_rx.c
create mode 100644 drivers/net/sxe2/sxe2_rx.h
create mode 100644 drivers/net/sxe2/sxe2_tx.c
create mode 100644 drivers/net/sxe2/sxe2_tx.h
diff --git a/drivers/net/sxe2/meson.build b/drivers/net/sxe2/meson.build
index 00c38b147c..3dfe54903a 100644
--- a/drivers/net/sxe2/meson.build
+++ b/drivers/net/sxe2/meson.build
@@ -18,6 +18,8 @@ sources += files(
'sxe2_cmd_chnl.c',
'sxe2_vsi.c',
'sxe2_queue.c',
+ 'sxe2_tx.c',
+ 'sxe2_rx.c',
)
allow_internal_get_api = true
diff --git a/drivers/net/sxe2/sxe2_ethdev.c b/drivers/net/sxe2/sxe2_ethdev.c
index 204add9c98..6abb4672f6 100644
--- a/drivers/net/sxe2/sxe2_ethdev.c
+++ b/drivers/net/sxe2/sxe2_ethdev.c
@@ -24,6 +24,8 @@
#include "sxe2_ethdev.h"
#include "sxe2_drv_cmd.h"
#include "sxe2_cmd_chnl.h"
+#include "sxe2_tx.h"
+#include "sxe2_rx.h"
#include "sxe2_common.h"
#include "sxe2_common_log.h"
#include "sxe2_host_regs.h"
@@ -86,14 +88,6 @@ static int32_t sxe2_dev_configure(struct rte_eth_dev *dev)
return ret;
}
-static void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev __rte_unused)
-{
-}
-
-static void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev __rte_unused)
-{
-}
-
static int32_t sxe2_dev_stop(struct rte_eth_dev *dev)
{
int32_t ret = 0;
@@ -112,16 +106,6 @@ static int32_t sxe2_dev_stop(struct rte_eth_dev *dev)
return ret;
}
-static int32_t __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev __rte_unused)
-{
- return 0;
-}
-
-static int32_t __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev __rte_unused)
-{
- return 0;
-}
-
static int32_t sxe2_queues_start(struct rte_eth_dev *dev)
{
int32_t ret = 0;
@@ -307,10 +291,18 @@ static const struct eth_dev_ops sxe2_eth_dev_ops = {
.dev_stop = sxe2_dev_stop,
.dev_close = sxe2_dev_close,
.dev_infos_get = sxe2_dev_infos_get,
+
+ .rx_queue_setup = sxe2_rx_queue_setup,
+ .tx_queue_setup = sxe2_tx_queue_setup,
+ .rx_queue_release = sxe2_rx_queue_release,
+ .tx_queue_release = sxe2_tx_queue_release,
+
+ .rxq_info_get = sxe2_rx_queue_info_get,
+ .txq_info_get = sxe2_tx_queue_info_get,
};
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type)
+ enum sxe2_pci_map_resource res_type)
{
struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
struct sxe2_pci_map_bar_info *bar_info = NULL;
@@ -334,6 +326,48 @@ struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter
return bar_info;
}
+void *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type,
+ uint16_t idx_in_func)
+{
+ struct sxe2_pci_map_context *map_ctxt = &adapter->map_ctxt;
+ struct sxe2_pci_map_segment_info *seg_info = NULL;
+ struct sxe2_pci_map_bar_info *bar_info = NULL;
+ void *addr = NULL;
+ uintptr_t calc_addr = 0;
+ uint8_t reg_width = 0;
+ uint8_t i = 0;
+
+ bar_info = sxe2_dev_get_bar_info(adapter, res_type);
+ if (bar_info == NULL) {
+ PMD_DEV_LOG_WARN(adapter, INIT, "Failed to get bar info, res_type=[%d]",
+ res_type);
+ goto l_end;
+ }
+ seg_info = bar_info->seg_info;
+
+ reg_width = map_ctxt->addr_info[res_type].reg_width;
+ if (reg_width == 0) {
+ PMD_DEV_LOG_WARN(adapter, INIT, "Invalid reg width with resource type %d",
+ res_type);
+ goto l_end;
+ }
+
+ for (i = 0; i < bar_info->map_cnt; i++) {
+ seg_info = &bar_info->seg_info[i];
+ if (res_type == seg_info->type) {
+ calc_addr = (uintptr_t)seg_info->addr;
+ calc_addr += (uintptr_t)seg_info->page_inner_offset;
+ calc_addr += (uintptr_t)reg_width * (uintptr_t)idx_in_func;
+ addr = (void *)calc_addr;
+ goto l_end;
+ }
+ }
+
+l_end:
+ return addr;
+}
+
static void sxe2_drv_dev_caps_set(struct sxe2_adapter *adapter,
struct sxe2_drv_dev_caps_resp *dev_caps)
{
@@ -402,7 +436,9 @@ static int32_t sxe2_dev_caps_get(struct sxe2_adapter *adapter)
}
int32_t sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type, uint64_t org_len, uint64_t org_offset)
+ enum sxe2_pci_map_resource res_type,
+ uint64_t org_len,
+ uint64_t org_offset)
{
struct sxe2_pci_map_bar_info *bar_info = NULL;
struct sxe2_pci_map_segment_info *seg_info = NULL;
@@ -478,8 +514,10 @@ static int32_t sxe2_hw_init(struct rte_eth_dev *dev)
return ret;
}
-int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter, uint32_t res_type,
- uint32_t item_cnt, uint32_t item_base)
+int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter,
+ uint32_t res_type,
+ uint32_t item_cnt,
+ uint32_t item_base)
{
struct sxe2_pci_map_addr_info *addr_info = NULL;
int32_t ret = 0;
diff --git a/drivers/net/sxe2/sxe2_ethdev.h b/drivers/net/sxe2/sxe2_ethdev.h
index 843e652616..001413e75a 100644
--- a/drivers/net/sxe2/sxe2_ethdev.h
+++ b/drivers/net/sxe2/sxe2_ethdev.h
@@ -293,14 +293,21 @@ struct sxe2_adapter {
#define SXE2_DEV_TO_PCI(eth_dev) \
RTE_DEV_TO_PCI((eth_dev)->device)
+void *sxe2_pci_map_addr_get(struct sxe2_adapter *adapter,
+ enum sxe2_pci_map_resource res_type,
+ uint16_t idx_in_func);
+
struct sxe2_pci_map_bar_info *sxe2_dev_get_bar_info(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type);
+ enum sxe2_pci_map_resource res_type);
int32_t sxe2_dev_pci_seg_map(struct sxe2_adapter *adapter,
- enum sxe2_pci_map_resource res_type, uint64_t org_len, uint64_t org_offset);
+ enum sxe2_pci_map_resource res_type,
+ uint64_t org_len, uint64_t org_offset);
-int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter, uint32_t res_type,
- uint32_t item_cnt, uint32_t item_base);
+int32_t sxe2_dev_pci_res_seg_map(struct sxe2_adapter *adapter,
+ uint32_t res_type,
+ uint32_t item_cnt,
+ uint32_t item_base);
void sxe2_dev_pci_seg_unmap(struct sxe2_adapter *adapter, uint32_t res_type);
diff --git a/drivers/net/sxe2/sxe2_rx.c b/drivers/net/sxe2/sxe2_rx.c
new file mode 100644
index 0000000000..a04c1808a6
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_rx.c
@@ -0,0 +1,559 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <ethdev_driver.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+
+#include "sxe2_ethdev.h"
+#include "sxe2_queue.h"
+#include "sxe2_rx.h"
+#include "sxe2_cmd_chnl.h"
+
+#include "sxe2_osal.h"
+#include "sxe2_common_log.h"
+
+static void *sxe2_rx_doorbell_tail_addr_get(struct sxe2_adapter *adapter, uint16_t queue_id)
+{
+ return sxe2_pci_map_addr_get(adapter, SXE2_PCI_MAP_RES_DOORBELL_RX_TAIL,
+ queue_id);
+}
+
+static void sxe2_rx_head_tail_init(struct sxe2_adapter *adapter, struct sxe2_rx_queue *rxq)
+{
+ rxq->rdt_reg_addr = sxe2_rx_doorbell_tail_addr_get(adapter, rxq->queue_id);
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, 0);
+}
+
+static void __rte_cold sxe2_rx_queue_reset(struct sxe2_rx_queue *rxq)
+{
+ uint16_t i = 0;
+ uint16_t len = 0;
+ static const union sxe2_rx_desc zeroed_desc = {{0}};
+
+ len = rxq->ring_depth + SXE2_RX_PKTS_BURST_BATCH_NUM;
+ for (i = 0; i < len; ++i)
+ rxq->desc_ring[i] = zeroed_desc;
+
+ memset(&rxq->fake_mbuf, 0, sizeof(rxq->fake_mbuf));
+ for (i = rxq->ring_depth; i < len; i++)
+ rxq->buffer_ring[i] = &rxq->fake_mbuf;
+
+ rxq->hold_num = 0;
+ rxq->next_ret_pkt = 0;
+ rxq->processing_idx = 0;
+ rxq->completed_pkts_num = 0;
+ rxq->batch_alloc_trigger = rxq->rx_free_thresh - 1;
+
+ rxq->pkt_first_seg = NULL;
+ rxq->pkt_last_seg = NULL;
+
+ rxq->realloc_num = 0;
+ rxq->realloc_start = 0;
+}
+
+void __rte_cold sxe2_rx_queue_mbufs_release(struct sxe2_rx_queue *rxq)
+{
+ uint16_t i;
+
+ if (rxq->buffer_ring != NULL) {
+ for (i = 0; i < rxq->ring_depth; i++) {
+ if (rxq->buffer_ring[i] != NULL) {
+ rte_pktmbuf_free(rxq->buffer_ring[i]);
+ rxq->buffer_ring[i] = NULL;
+ }
+ }
+ }
+
+ if (rxq->completed_pkts_num) {
+ for (i = 0; i < rxq->completed_pkts_num; ++i) {
+ if (rxq->completed_buf[rxq->next_ret_pkt + i] != NULL) {
+ rte_pktmbuf_free(rxq->completed_buf[rxq->next_ret_pkt + i]);
+ rxq->completed_buf[rxq->next_ret_pkt + i] = NULL;
+ }
+ }
+ rxq->completed_pkts_num = 0;
+ }
+}
+
+const struct sxe2_rxq_ops sxe2_default_rxq_ops = {
+ .queue_reset = sxe2_rx_queue_reset,
+ .mbufs_release = sxe2_rx_queue_mbufs_release,
+};
+
+static struct sxe2_rxq_ops sxe2_rx_default_ops_get(void)
+{
+ return sxe2_default_rxq_ops;
+}
+
+void __rte_cold sxe2_rx_queue_info_get(struct rte_eth_dev *dev,
+ uint16_t queue_id, struct rte_eth_rxq_info *qinfo)
+{
+ struct sxe2_rx_queue *rxq = NULL;
+
+ if (queue_id >= dev->data->nb_rx_queues) {
+ PMD_LOG_ERR(RX, "rx queue:%u is out of range:%u",
+ queue_id, dev->data->nb_rx_queues);
+ goto end;
+ }
+
+ rxq = dev->data->rx_queues[queue_id];
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue:%u is NULL", queue_id);
+ goto end;
+ }
+
+ qinfo->mp = rxq->mb_pool;
+ qinfo->nb_desc = rxq->ring_depth;
+ qinfo->scattered_rx = dev->data->scattered_rx;
+ qinfo->conf.rx_free_thresh = rxq->rx_free_thresh;
+ qinfo->conf.rx_drop_en = rxq->drop_en;
+ qinfo->conf.rx_deferred_start = rxq->rx_deferred_start;
+
+end:
+ return;
+}
+
+int32_t __rte_cold sxe2_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_rx_queue *rxq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (dev->data->rx_queue_state[rx_queue_id] ==
+ RTE_ETH_QUEUE_STATE_STOPPED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (rxq == NULL) {
+ ret = 0;
+ goto l_end;
+ }
+ ret = sxe2_drv_rxq_switch(adapter, rxq, false);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Failed to switch rx queue %u off, ret = %d",
+ rx_queue_id, ret);
+ if (ret == -EPERM)
+ goto l_free;
+ goto l_end;
+ }
+
+l_free:
+ rxq->ops.mbufs_release(rxq);
+ rxq->ops.queue_reset(rxq);
+ dev->data->rx_queue_state[rx_queue_id] =
+ RTE_ETH_QUEUE_STATE_STOPPED;
+l_end:
+ return ret;
+}
+
+static void __rte_cold sxe2_rx_queue_free(struct sxe2_rx_queue *rxq)
+{
+ if (rxq != NULL) {
+ rxq->ops.mbufs_release(rxq);
+ if (rxq->buffer_ring != NULL) {
+ rte_free(rxq->buffer_ring);
+ rxq->buffer_ring = NULL;
+ }
+ rte_memzone_free(rxq->mz);
+ rte_free(rxq);
+ }
+}
+
+void __rte_cold sxe2_rx_queue_release(struct rte_eth_dev *dev,
+ uint16_t queue_idx)
+{
+ (void)sxe2_rx_queue_stop(dev, queue_idx);
+ sxe2_rx_queue_free(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+}
+
+void __rte_cold sxe2_all_rxqs_release(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ uint16_t nb_rxq;
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ if (data->rx_queues[nb_rxq] == NULL)
+ continue;
+ sxe2_rx_queue_release(dev, nb_rxq);
+ data->rx_queues[nb_rxq] = NULL;
+ }
+ data->nb_rx_queues = 0;
+}
+
+static struct sxe2_rx_queue *sxe2_rx_queue_alloc(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t ring_depth, uint32_t socket_id)
+{
+ struct sxe2_rx_queue *rxq;
+ const struct rte_memzone *tz;
+ uint16_t len;
+
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ sxe2_rx_queue_release(dev, queue_idx);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
+ rxq = rte_zmalloc_socket("rx_queue", sizeof(*rxq),
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue[%d] alloc failed", queue_idx);
+ goto l_end;
+ }
+
+ rxq->ring_depth = ring_depth;
+ len = rxq->ring_depth + SXE2_RX_PKTS_BURST_BATCH_NUM;
+
+ rxq->buffer_ring = rte_zmalloc_socket("rx_buffer_ring",
+ sizeof(struct rte_mbuf *) * len,
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (!rxq->buffer_ring) {
+ PMD_LOG_ERR(RX, "Rxq malloc mbuf mem failed");
+ rte_free(rxq);
+ rxq = NULL;
+ goto l_end;
+ }
+
+ tz = rte_eth_dma_zone_reserve(dev, "rx_dma", queue_idx,
+ SXE2_RX_RING_SIZE, SXE2_DESC_ADDR_ALIGN, socket_id);
+ if (tz == NULL) {
+ PMD_LOG_ERR(RX, "Rxq malloc desc mem failed");
+ rte_free(rxq->buffer_ring);
+ rxq->buffer_ring = NULL;
+ rte_free(rxq);
+ rxq = NULL;
+ goto l_end;
+ }
+
+ rxq->mz = tz;
+ memset(tz->addr, 0, SXE2_RX_RING_SIZE);
+ rxq->base_addr = tz->iova;
+ rxq->desc_ring = (union sxe2_rx_desc *)tz->addr;
+
+l_end:
+ return rxq;
+}
+
+int32_t __rte_cold sxe2_rx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ struct sxe2_rx_queue *rxq;
+ uint64_t offloads;
+ int32_t ret;
+ uint16_t rx_nseg;
+ uint16_t i;
+
+ PMD_INIT_FUNC_TRACE();
+
+ if (nb_desc % SXE2_RX_DESC_RING_ALIGN != 0 ||
+ nb_desc > SXE2_MAX_RING_DESC ||
+ nb_desc < SXE2_MIN_RING_DESC) {
+ PMD_LOG_ERR(RX, "param desc num:%u is invalid", nb_desc);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (mp != NULL)
+ rx_nseg = 1;
+ else
+ rx_nseg = rx_conf->rx_nseg;
+
+ offloads = rx_conf->offloads | dev->data->dev_conf.rxmode.offloads;
+
+ if (rx_nseg > 1 && !(offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ PMD_LOG_ERR(RX, "Port %u queue %u Buffer split offload not configured, but rx_nseg is %u",
+ dev->data->port_id, queue_idx, rx_nseg);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) && !(rx_nseg > 1)) {
+ PMD_LOG_ERR(RX, "Port %u queue %u Buffer split offload configured, but rx_nseg is %u",
+ dev->data->port_id, queue_idx, rx_nseg);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO) &&
+ (offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)) {
+ PMD_LOG_ERR(RX, "port_id %u queue %u, LRO can't be configure with Keep crc.",
+ dev->data->port_id, queue_idx);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ rxq = sxe2_rx_queue_alloc(dev, queue_idx, nb_desc, socket_id);
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "rx queue[%d] resource alloc failed", queue_idx);
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ if (offloads & RTE_ETH_RX_OFFLOAD_TCP_LRO)
+ dev->data->lro = 1;
+
+ if (rx_nseg > 1) {
+ for (i = 0; i < rx_nseg; i++) {
+ rte_memcpy(&rxq->rx_seg[i], &rx_conf->rx_seg[i].split,
+ sizeof(struct rte_eth_rxseg_split));
+ }
+ rxq->mb_pool = rxq->rx_seg[0].mp;
+ } else {
+ rxq->mb_pool = mp;
+ }
+
+ rxq->rx_free_thresh = rx_conf->rx_free_thresh;
+ rxq->port_id = dev->data->port_id;
+ rxq->offloads = offloads;
+ if (offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC)
+ rxq->crc_len = RTE_ETHER_CRC_LEN;
+ else
+ rxq->crc_len = 0;
+
+ rxq->queue_id = queue_idx;
+ rxq->idx_in_func = vsi->rxqs.base_idx_in_func + queue_idx;
+ rxq->drop_en = rx_conf->rx_drop_en;
+ rxq->rx_deferred_start = rx_conf->rx_deferred_start;
+ rxq->vsi = vsi;
+ rxq->ops = sxe2_rx_default_ops_get();
+ rxq->ops.queue_reset(rxq);
+ dev->data->rx_queues[queue_idx] = rxq;
+
+ ret = 0;
+l_end:
+ return ret;
+}
+
+struct rte_mbuf *sxe2_mbuf_raw_alloc(struct rte_mempool *mp)
+{
+ return rte_mbuf_raw_alloc(mp);
+}
+
+static int32_t __rte_cold sxe2_rx_queue_mbufs_alloc(struct sxe2_rx_queue *rxq)
+{
+ struct rte_mbuf **buf_ring = rxq->buffer_ring;
+ struct rte_mbuf *mbuf = NULL;
+ struct rte_mbuf *mbuf_pay;
+ volatile union sxe2_rx_desc *desc;
+ uint64_t dma_addr;
+ int32_t ret;
+ uint16_t i, j;
+
+ for (i = 0; i < rxq->ring_depth; i++) {
+ mbuf = sxe2_mbuf_raw_alloc(rxq->mb_pool);
+ if (mbuf == NULL) {
+ PMD_LOG_ERR(RX, "Rx queue is not available or setup");
+ ret = -ENOMEM;
+ goto l_err_free_mbuf;
+ }
+
+ buf_ring[i] = mbuf;
+ mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf->nb_segs = 1;
+ mbuf->port = rxq->port_id;
+
+ dma_addr = rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf));
+ desc = &rxq->desc_ring[i];
+ if (!(rxq->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT)) {
+ mbuf->next = NULL;
+ desc->read.hdr_addr = 0;
+ desc->read.pkt_addr = dma_addr;
+ } else {
+ mbuf_pay = rte_mbuf_raw_alloc(rxq->rx_seg[1].mp);
+ if (unlikely(!mbuf_pay)) {
+ PMD_LOG_ERR(RX, "Failed to allocate payload mbuf for RX");
+ ret = -ENOMEM;
+ goto l_err_free_mbuf;
+ }
+
+ mbuf_pay->next = NULL;
+ mbuf_pay->data_off = RTE_PKTMBUF_HEADROOM;
+ mbuf_pay->nb_segs = 1;
+ mbuf_pay->port = rxq->port_id;
+ mbuf->next = mbuf_pay;
+
+ desc->read.hdr_addr = dma_addr;
+ desc->read.pkt_addr =
+ rte_cpu_to_le_64(rte_mbuf_data_iova_default(mbuf_pay));
+ }
+
+#ifndef RTE_LIBRTE_SXE2_16BYTE_RX_DESC
+ desc->read.rsvd1 = 0;
+ desc->read.rsvd2 = 0;
+#endif
+ }
+
+ ret = 0;
+ goto l_end;
+
+l_err_free_mbuf:
+ for (j = 0; j <= i; j++) {
+ if (buf_ring[j] != NULL && buf_ring[j]->next != NULL) {
+ rte_pktmbuf_free(buf_ring[j]->next);
+ buf_ring[j]->next = NULL;
+ }
+
+ if (buf_ring[j] != NULL) {
+ rte_pktmbuf_free(buf_ring[j]);
+ buf_ring[j] = NULL;
+ }
+ }
+
+l_end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
+{
+ struct sxe2_rx_queue *rxq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ rxq = dev->data->rx_queues[rx_queue_id];
+ if (rxq == NULL) {
+ PMD_LOG_ERR(RX, "Rx queue %u is not available or setup",
+ rx_queue_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (dev->data->rx_queue_state[rx_queue_id] ==
+ RTE_ETH_QUEUE_STATE_STARTED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ ret = sxe2_rx_queue_mbufs_alloc(rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Rx queue %u apply desc ring fail",
+ rx_queue_id);
+ ret = -ENOMEM;
+ goto l_end;
+ }
+
+ sxe2_rx_head_tail_init(adapter, rxq);
+
+ ret = sxe2_drv_rxq_ctxt_cfg(adapter, rxq, 1);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Rx queue %u config ctxt fail, ret=%d",
+ rx_queue_id, ret);
+
+ (void)sxe2_drv_rxq_switch(adapter, rxq, false);
+ rxq->ops.mbufs_release(rxq);
+ rxq->ops.queue_reset(rxq);
+ goto l_end;
+ }
+
+ SXE2_PCI_REG_WRITE_WC(rxq->rdt_reg_addr, rxq->ring_depth - 1);
+ dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+
+l_end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_rx_queue *rxq;
+ uint16_t nb_rxq;
+ uint16_t nb_started_rxq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ rxq = dev->data->rx_queues[nb_rxq];
+ if (!rxq || rxq->rx_deferred_start)
+ continue;
+
+ ret = sxe2_rx_queue_start(dev, nb_rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Fail to start rx queue %u", nb_rxq);
+ goto l_free_started_queue;
+ }
+
+ rte_atomic_store_explicit(&rxq->sw_stats.pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.bytes, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.drop_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.drop_bytes, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.unicast_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.broadcast_pkts, 0,
+ rte_memory_order_relaxed);
+ rte_atomic_store_explicit(&rxq->sw_stats.multicast_pkts, 0,
+ rte_memory_order_relaxed);
+ }
+ ret = 0;
+ goto l_end;
+
+l_free_started_queue:
+ for (nb_started_rxq = 0; nb_started_rxq <= nb_rxq; nb_started_rxq++)
+ (void)sxe2_rx_queue_stop(dev, nb_started_rxq);
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ struct sxe2_stats *sw_stats_prev = &vsi->vsi_stats.vsi_sw_stats_prev;
+ struct sxe2_rx_queue *rxq = NULL;
+ int32_t ret;
+ uint16_t nb_rxq;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_rxq = 0; nb_rxq < data->nb_rx_queues; nb_rxq++) {
+ ret = sxe2_rx_queue_stop(dev, nb_rxq);
+ if (ret) {
+ PMD_LOG_ERR(RX, "Fail to start rx queue %u", nb_rxq);
+ continue;
+ }
+
+ rxq = dev->data->rx_queues[nb_rxq];
+ if (rxq) {
+ sw_stats_prev->ipackets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->ierrors +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->ibytes +=
+ rte_atomic_load_explicit(&rxq->sw_stats.bytes,
+ rte_memory_order_relaxed);
+
+ sw_stats_prev->rx_sw_unicast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.unicast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_broadcast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.broadcast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_multicast_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.multicast_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_drop_packets +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_pkts,
+ rte_memory_order_relaxed);
+ sw_stats_prev->rx_sw_drop_bytes +=
+ rte_atomic_load_explicit(&rxq->sw_stats.drop_bytes,
+ rte_memory_order_relaxed);
+ }
+ }
+}
diff --git a/drivers/net/sxe2/sxe2_rx.h b/drivers/net/sxe2/sxe2_rx.h
new file mode 100644
index 0000000000..138a9d56c9
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_rx.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_RX_H__
+#define __SXE2_RX_H__
+
+#include "sxe2_queue.h"
+
+int32_t __rte_cold sxe2_rx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mp);
+
+int32_t __rte_cold sxe2_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+
+void __rte_cold sxe2_rx_queue_mbufs_release(struct sxe2_rx_queue *rxq);
+
+void __rte_cold sxe2_rx_queue_release(struct rte_eth_dev *dev, uint16_t queue_idx);
+
+void __rte_cold sxe2_all_rxqs_release(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rx_queue_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_rxq_info *qinfo);
+
+int32_t __rte_cold sxe2_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id);
+
+int32_t __rte_cold sxe2_rxqs_all_start(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_rxqs_all_stop(struct rte_eth_dev *dev);
+
+struct rte_mbuf *sxe2_mbuf_raw_alloc(struct rte_mempool *mp);
+
+#endif /* __SXE2_RX_H__ */
diff --git a/drivers/net/sxe2/sxe2_tx.c b/drivers/net/sxe2/sxe2_tx.c
new file mode 100644
index 0000000000..a05beb8c7a
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_tx.c
@@ -0,0 +1,420 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#include <rte_common.h>
+#include <rte_net.h>
+#include <rte_vect.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
+#include <ethdev_driver.h>
+#include "sxe2_tx.h"
+#include "sxe2_ethdev.h"
+#include "sxe2_common_log.h"
+#include "sxe2_cmd_chnl.h"
+
+static void *sxe2_tx_doorbell_addr_get(struct sxe2_adapter *adapter, uint16_t queue_id)
+{
+ return sxe2_pci_map_addr_get(adapter, SXE2_PCI_MAP_RES_DOORBELL_TX,
+ queue_id);
+}
+
+static void sxe2_tx_tail_init(struct sxe2_adapter *adapter, struct sxe2_tx_queue *txq)
+{
+ txq->tdt_reg_addr = sxe2_tx_doorbell_addr_get(adapter, txq->queue_id);
+ SXE2_PCI_REG_WRITE_WC(txq->tdt_reg_addr, 0);
+}
+
+void __rte_cold sxe2_tx_queue_reset(struct sxe2_tx_queue *txq)
+{
+ uint16_t prev, i;
+ volatile union sxe2_tx_data_desc *txd;
+ static const union sxe2_tx_data_desc zeroed_desc = {{0}};
+ struct sxe2_tx_buffer *tx_buffer = txq->buffer_ring;
+
+ for (i = 0; i < txq->ring_depth; i++)
+ txq->desc_ring[i] = zeroed_desc;
+
+ prev = txq->ring_depth - 1;
+ for (i = 0; i < txq->ring_depth; i++) {
+ txd = &txq->desc_ring[i];
+ if (txd == NULL)
+ continue;
+
+ txd->wb.dd = rte_cpu_to_le_64(SXE2_TX_DESC_DTYPE_DESC_DONE);
+ tx_buffer[i].mbuf = NULL;
+ tx_buffer[i].last_id = i;
+ tx_buffer[prev].next_id = i;
+ prev = i;
+ }
+
+ txq->desc_used_num = 0;
+ txq->desc_free_num = txq->ring_depth - 1;
+ txq->next_use = 0;
+ txq->next_clean = txq->ring_depth - 1;
+ txq->next_dd = txq->rs_thresh - 1;
+ txq->next_rs = txq->rs_thresh - 1;
+}
+
+void __rte_cold sxe2_tx_queue_mbufs_release(struct sxe2_tx_queue *txq)
+{
+ uint32_t i;
+
+ if (txq != NULL && txq->buffer_ring != NULL) {
+ for (i = 0; i < txq->ring_depth; i++) {
+ if (txq->buffer_ring[i].mbuf != NULL) {
+ rte_pktmbuf_free_seg(txq->buffer_ring[i].mbuf);
+ txq->buffer_ring[i].mbuf = NULL;
+ }
+ }
+ }
+}
+
+static void sxe2_tx_buffer_ring_free(struct sxe2_tx_queue *txq)
+{
+ if (txq != NULL && txq->buffer_ring != NULL)
+ rte_free(txq->buffer_ring);
+}
+
+const struct sxe2_txq_ops sxe2_default_txq_ops = {
+ .queue_reset = sxe2_tx_queue_reset,
+ .mbufs_release = sxe2_tx_queue_mbufs_release,
+ .buffer_ring_free = sxe2_tx_buffer_ring_free,
+};
+
+static struct sxe2_txq_ops sxe2_tx_default_ops_get(void)
+{
+ return sxe2_default_txq_ops;
+}
+
+static int32_t sxe2_txq_arg_validate(struct rte_eth_dev *dev, uint16_t ring_depth,
+ uint16_t *rs_thresh, uint16_t *free_thresh, const struct rte_eth_txconf *tx_conf)
+{
+ int32_t ret = 0;
+
+ if ((ring_depth % SXE2_TX_DESC_RING_ALIGN) != 0 ||
+ ring_depth > SXE2_MAX_RING_DESC ||
+ ring_depth < SXE2_MIN_RING_DESC) {
+ PMD_LOG_ERR(TX, "number:%u of receive descriptors is invalid", ring_depth);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ *free_thresh = (uint16_t)((tx_conf->tx_free_thresh) ?
+ tx_conf->tx_free_thresh : DEFAULT_TX_FREE_THRESH);
+ *rs_thresh = (uint16_t)((tx_conf->tx_rs_thresh) ?
+ tx_conf->tx_rs_thresh : DEFAULT_TX_RS_THRESH);
+
+ if (*rs_thresh >= (ring_depth - 2)) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be less than the number "
+ "of tx descriptors minus 2. (tx_rs_thresh:%u port:%u)",
+ *rs_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (*free_thresh >= (ring_depth - 3)) {
+ PMD_LOG_ERR(TX, "tx_free_thresh must be less than the number "
+ "of tx descriptors minus 3. (tx_free_thresh:%u port:%u)",
+ *free_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if (*rs_thresh > *free_thresh) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be less than or equal to "
+ "tx_free_thresh. (tx_free_thresh:%u tx_rs_thresh:%u port:%u)",
+ *free_thresh, *rs_thresh, dev->data->port_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ if ((ring_depth % *rs_thresh) != 0) {
+ PMD_LOG_ERR(TX, "tx_rs_thresh must be a divisor of the "
+ "number of tx descriptors. (tx_rs_thresh:%u port:%d ring_depth:%u)",
+ *rs_thresh, dev->data->port_id, ring_depth);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_txq_info *qinfo)
+{
+ struct sxe2_tx_queue *txq = NULL;
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ PMD_LOG_WARN(TX, "tx queue:%u is NULL", queue_id);
+ goto end;
+ }
+
+ qinfo->nb_desc = txq->ring_depth;
+
+ qinfo->conf.tx_thresh.pthresh = txq->pthresh;
+ qinfo->conf.tx_thresh.hthresh = txq->hthresh;
+ qinfo->conf.tx_thresh.wthresh = txq->wthresh;
+ qinfo->conf.tx_free_thresh = txq->free_thresh;
+ qinfo->conf.tx_rs_thresh = txq->rs_thresh;
+ qinfo->conf.offloads = txq->offloads;
+ qinfo->conf.tx_deferred_start = txq->tx_deferred_start;
+
+end:
+ return;
+}
+
+int32_t __rte_cold sxe2_tx_queue_stop(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_tx_queue *txq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ if (dev->data->tx_queue_state[queue_id] ==
+ RTE_ETH_QUEUE_STATE_STOPPED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ ret = 0;
+ goto l_end;
+ }
+
+ ret = sxe2_drv_txq_switch(adapter, txq, false);
+ if (ret) {
+ PMD_LOG_ERR(TX, "Failed to switch tx queue %u off",
+ queue_id);
+ goto l_end;
+ }
+
+ txq->ops.mbufs_release(txq);
+ txq->ops.queue_reset(txq);
+ dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+static void __rte_cold sxe2_tx_queue_free(struct sxe2_tx_queue *txq)
+{
+ if (txq != NULL) {
+ txq->ops.mbufs_release(txq);
+ txq->ops.buffer_ring_free(txq);
+
+ rte_memzone_free(txq->mz);
+ rte_free(txq);
+ }
+}
+
+void __rte_cold sxe2_tx_queue_release(struct rte_eth_dev *dev, uint16_t queue_idx)
+{
+ (void)sxe2_tx_queue_stop(dev, queue_idx);
+ sxe2_tx_queue_free(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+}
+
+void __rte_cold sxe2_all_txqs_release(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ uint16_t nb_txq;
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ if (data->tx_queues[nb_txq] == NULL)
+ continue;
+
+ sxe2_tx_queue_release(dev, nb_txq);
+ data->tx_queues[nb_txq] = NULL;
+ }
+ data->nb_tx_queues = 0;
+}
+
+static struct sxe2_tx_queue
+*sxe2_tx_queue_alloc(struct rte_eth_dev *dev, uint16_t queue_idx,
+ uint16_t ring_depth, uint32_t socket_id)
+{
+ struct sxe2_tx_queue *txq;
+ const struct rte_memzone *tz;
+
+ if (dev->data->tx_queues[queue_idx]) {
+ sxe2_tx_queue_release(dev, queue_idx);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
+ txq = rte_zmalloc_socket("tx_queue", sizeof(struct sxe2_tx_queue),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "tx queue:%d alloc failed", queue_idx);
+ goto l_end;
+ }
+
+ tz = rte_eth_dma_zone_reserve(dev, "tx_dma", queue_idx,
+ sizeof(union sxe2_tx_data_desc) * SXE2_MAX_RING_DESC,
+ SXE2_DESC_ADDR_ALIGN, socket_id);
+ if (tz == NULL) {
+ PMD_LOG_ERR(TX, "tx desc ring alloc failed, queue_id:%d", queue_idx);
+ rte_free(txq);
+ txq = NULL;
+ goto l_end;
+ }
+
+ txq->buffer_ring = rte_zmalloc_socket("tx_buffer_ring",
+ sizeof(struct sxe2_tx_buffer) * ring_depth,
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq->buffer_ring == NULL) {
+ PMD_LOG_ERR(TX, "tx buffer alloc failed, queue_id:%d", queue_idx);
+ rte_memzone_free(tz);
+ rte_free(txq);
+ txq = NULL;
+ goto l_end;
+ }
+
+ txq->mz = tz;
+ txq->base_addr = tz->iova;
+ txq->desc_ring = (volatile union sxe2_tx_data_desc *)tz->addr;
+
+l_end:
+ return txq;
+}
+
+int32_t __rte_cold sxe2_tx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ int32_t ret = 0;
+ uint16_t tx_rs_thresh;
+ uint16_t tx_free_thresh;
+ struct sxe2_tx_queue *txq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ struct sxe2_vsi *vsi = adapter->vsi_ctxt.main_vsi;
+ uint64_t offloads;
+ PMD_INIT_FUNC_TRACE();
+
+ ret = sxe2_txq_arg_validate(dev, nb_desc, &tx_rs_thresh, &tx_free_thresh, tx_conf);
+ if (ret) {
+ PMD_LOG_ERR(TX, "tx queue:%u arg validate failed", queue_idx);
+ goto end;
+ }
+
+ offloads = tx_conf->offloads | dev->data->dev_conf.txmode.offloads;
+
+ txq = sxe2_tx_queue_alloc(dev, queue_idx, nb_desc, socket_id);
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "failed to alloc sxe2vf tx queue:%u resource", queue_idx);
+ ret = -ENOMEM;
+ goto end;
+ }
+
+ txq->vlan_flag = SXE2_TX_FLAGS_VLAN_TAG_LOC_L2TAG1;
+ txq->ring_depth = nb_desc;
+ txq->rs_thresh = tx_rs_thresh;
+ txq->free_thresh = tx_free_thresh;
+ txq->pthresh = tx_conf->tx_thresh.pthresh;
+ txq->hthresh = tx_conf->tx_thresh.hthresh;
+ txq->wthresh = tx_conf->tx_thresh.wthresh;
+ txq->queue_id = queue_idx;
+ txq->idx_in_func = vsi->txqs.base_idx_in_func + queue_idx;
+ txq->port_id = dev->data->port_id;
+ txq->offloads = offloads;
+ txq->tx_deferred_start = tx_conf->tx_deferred_start;
+ txq->vsi = vsi;
+ txq->ops = sxe2_tx_default_ops_get();
+ txq->ops.queue_reset(txq);
+
+ dev->data->tx_queues[queue_idx] = txq;
+ ret = 0;
+
+end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_tx_queue_start(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+ int32_t ret = 0;
+ struct sxe2_tx_queue *txq;
+ struct sxe2_adapter *adapter = SXE2_DEV_PRIVATE_TO_ADAPTER(dev);
+ PMD_INIT_FUNC_TRACE();
+
+ if (dev->data->tx_queue_state[queue_id] == RTE_ETH_QUEUE_STATE_STARTED) {
+ ret = 0;
+ goto l_end;
+ }
+
+ txq = dev->data->tx_queues[queue_id];
+ if (txq == NULL) {
+ PMD_LOG_ERR(TX, "tx queue:%u is not available or setup", queue_id);
+ ret = -EINVAL;
+ goto l_end;
+ }
+
+ ret = sxe2_drv_txq_ctxt_cfg(adapter, txq, 1);
+ if (ret) {
+ PMD_LOG_ERR(TX, "tx queue:%u config ctxt fail", queue_id);
+
+ (void)sxe2_drv_txq_switch(adapter, txq, false);
+ txq->ops.mbufs_release(txq);
+ txq->ops.queue_reset(txq);
+ goto l_end;
+ }
+
+ sxe2_tx_tail_init(adapter, txq);
+
+ dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+ ret = 0;
+
+l_end:
+ return ret;
+}
+
+int32_t __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ struct sxe2_tx_queue *txq;
+ uint16_t nb_txq;
+ uint16_t nb_started_txq;
+ int32_t ret;
+ PMD_INIT_FUNC_TRACE();
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ txq = dev->data->tx_queues[nb_txq];
+ if (!txq || txq->tx_deferred_start)
+ continue;
+
+ ret = sxe2_tx_queue_start(dev, nb_txq);
+ if (ret) {
+ PMD_LOG_ERR(TX, "Fail to start tx queue %u", nb_txq);
+ goto l_free_started_queue;
+ }
+ }
+ ret = 0;
+ goto l_end;
+
+l_free_started_queue:
+ for (nb_started_txq = 0; nb_started_txq <= nb_txq; nb_started_txq++)
+ (void)sxe2_tx_queue_stop(dev, nb_started_txq);
+
+l_end:
+ return ret;
+}
+
+void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev)
+{
+ struct rte_eth_dev_data *data = dev->data;
+ uint16_t nb_txq;
+ int32_t ret;
+
+ for (nb_txq = 0; nb_txq < data->nb_tx_queues; nb_txq++) {
+ ret = sxe2_tx_queue_stop(dev, nb_txq);
+ if (ret) {
+ PMD_LOG_WARN(TX, "Fail to stop tx queue %u", nb_txq);
+ continue;
+ }
+ }
+}
diff --git a/drivers/net/sxe2/sxe2_tx.h b/drivers/net/sxe2/sxe2_tx.h
new file mode 100644
index 0000000000..c929b1bee2
--- /dev/null
+++ b/drivers/net/sxe2/sxe2_tx.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C), 2025, Wuxi Stars Micro System Technologies Co., Ltd.
+ */
+
+#ifndef __SXE2_TX_H__
+#define __SXE2_TX_H__
+#include "sxe2_queue.h"
+
+void __rte_cold sxe2_tx_queue_reset(struct sxe2_tx_queue *txq);
+
+int32_t __rte_cold sxe2_tx_queue_start(struct rte_eth_dev *dev, uint16_t queue_id);
+
+void sxe2_tx_queue_mbufs_release(struct sxe2_tx_queue *txq);
+
+int32_t __rte_cold sxe2_tx_queue_stop(struct rte_eth_dev *dev, uint16_t queue_id);
+
+int32_t __rte_cold sxe2_tx_queue_setup(struct rte_eth_dev *dev,
+ uint16_t queue_idx, uint16_t nb_desc, uint32_t socket_id,
+ const struct rte_eth_txconf *tx_conf);
+
+void __rte_cold sxe2_tx_queue_release(struct rte_eth_dev *dev, uint16_t queue_idx);
+
+void __rte_cold sxe2_all_txqs_release(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_tx_queue_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+ struct rte_eth_txq_info *qinfo);
+
+int32_t __rte_cold sxe2_txqs_all_start(struct rte_eth_dev *dev);
+
+void __rte_cold sxe2_txqs_all_stop(struct rte_eth_dev *dev);
+
+#endif /* __SXE2_TX_H__ */
--
2.47.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox