* [TEST PATCH v9] eal/x86: optimize memcpy of small sizes
From: Morten Brørup @ 2026-05-21 10:54 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
In-Reply-To: <20260429103548.220354-1-mb@smartsharesystems.com>
TEST: Were the Intel drivers the only ones triggering the warnings with mingw in Github?
Depends-on: series-38174 ("remove use of rte_memcpy from net/intel")
Venlig hilsen / Kind regards,
-Morten Brørup
The implementation for copying up to 64 bytes does not depend on address
alignment with the size of the CPU's vector registers. Nonetheless, the
exact same code for copying up to 64 bytes was present in both the aligned
copy function and all the CPU vector register size specific variants of
the unaligned copy functions.
With this patch, the implementation for copying up to 64 bytes was
consolidated into one instance, located in the common copy function,
before checking alignment requirements.
This provides three benefits:
1. No copy-paste in the source code.
2. A performance gain for copying up to 64 bytes, because the
address alignment check is avoided in this case.
3. Reduced instruction memory footprint, because the compiler only
generates one instance of the function for copying up to 64 bytes, instead
of two instances (one in the unaligned copy function, and one in the
aligned copy function).
Furthermore, __rte_restrict was added to source and destination addresses.
And finally, the missing implementation of rte_mov48() was added.
Regarding performance...
The memcpy performance test (cache-to-cache copy) shows:
Copying up to 15 bytes takes ca. 4.5 cycles, versus ca. 6.5 cycles before.
Copying 8 bytes takes 4 cycles, versus 7 cycles before.
Copying 16 bytes takes 2 cycles, versus 4 cycles before.
Copying 64 bytes takes 4 cycles, versus 7 cycles before.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@huawei.com>
---
v9:
* Removed new functions rte_mov16_to_32() and rte_mov32_to_64(), and moved
their implementations into rte_memcpy() instead.
There is no need for such public functions, and having them separate did
not improve source code readability.
* Kept acks from Bruce and Konstantin (both given to v7).
v8:
* Reverted the first branch from size <= 16 back to size < 16, restored
the original rte_mov15_or_less() function, and removed the new
rte_mov16_or_less() function.
When rte_memcpy() is used for copying an array of pointers, and the
number of pointers to copy is low (size <= 64 bytes), it is more likely
that the number of pointers to copy is 1 than 2.
The rte_mov15_or_less() implementation handles copying 8 bytes more
efficiently than the rte_mov16_or_less() implementation, which copied
the 8-byte pointer twice.
Also note that with rte_mov15_or_less(), the compiler can optimize away
the branches handling n & 1, n & 2 and n & 4 when it is known at compile
time that (8-byte) pointers are being copied. (For 32-bit architecture,
the n & 4 will not be optimized away when copying pointers.)
This reversion also makes the patch less revolutionary and more
incremental.
* Removed a lot of code for handling compile time known sizes. (Bruce)
The rte_memcpy() function should not be used for small copies with
compile time known sizes, so handling it is considered superfluous.
Removing it improves source code readability. And reduces the size of
the patch.
* Kept acks from Bruce and Konstantin (both given to v7).
v7:
* Updated patch description. Mainly to clarify that the changes related to
copying up to 64 bytes simply replaces multiple instances of copy-pasted
code with one common instance.
* Fixed copy of compile time known 16 bytes in rte_mov17_to_32(). (Vipin)
* Rebased.
v6:
* Went back to using rte_uintN_alias structures for copying instead of
using memcpy(). They were there for a reason.
(Inspired by the discussion about optimizing the checksum function.)
* Removed note about copying uninitialized data.
* Added __rte_restrict to source and destination addresses.
Updated function descriptions from "should" to "must" not overlap.
* Changed rte_mov48() AVX implementation to copy 32+16 bytes instead of
copying 32 + 32 overlapping bytes. (Konstantin)
* Ignoring "-Wstringop-overflow" is not needed, so it was removed.
v5:
* Reverted v4: Replace SSE2 _mm_loadu_si128() with SSE3 _mm_lddqu_si128().
It was slower.
* Improved some comments. (Konstantin Ananyev)
* Moved the size range 17..32 inside the size <= 64 branch, so when
building for SSE, the generated code can start copying the first
16 bytes before comparing if the size is greater than 32 or not.
* Just require RTE_MEMCPY_AVX for using rte_mov32() in rte_mov33_to_64().
v4:
* Replace SSE2 _mm_loadu_si128() with SSE3 _mm_lddqu_si128().
v3:
* Fixed typo in comment.
v2:
* Updated patch title to reflect that the performance is improved.
* Use the design pattern of two overlapping stores for small copies too.
* Expanded first branch from size < 16 to size <= 16.
* Handle more compile time constant copy sizes.
---
lib/eal/x86/include/rte_memcpy.h | 250 +++++++++++++------------------
1 file changed, 102 insertions(+), 148 deletions(-)
diff --git a/lib/eal/x86/include/rte_memcpy.h b/lib/eal/x86/include/rte_memcpy.h
index 46d34b8081..8ed8c55010 100644
--- a/lib/eal/x86/include/rte_memcpy.h
+++ b/lib/eal/x86/include/rte_memcpy.h
@@ -22,11 +22,6 @@
extern "C" {
#endif
-#if defined(RTE_TOOLCHAIN_GCC) && (GCC_VERSION >= 100000)
-#pragma GCC diagnostic push
-#pragma GCC diagnostic ignored "-Wstringop-overflow"
-#endif
-
/*
* GCC older than version 11 doesn't compile AVX properly, so use SSE instead.
* There are no problems with AVX2.
@@ -40,9 +35,6 @@ extern "C" {
/**
* Copy bytes from one location to another. The locations must not overlap.
*
- * @note This is implemented as a macro, so it's address should not be taken
- * and care is needed as parameter expressions may be evaluated multiple times.
- *
* @param dst
* Pointer to the destination of the data.
* @param src
@@ -53,15 +45,15 @@ extern "C" {
* Pointer to the destination data.
*/
static __rte_always_inline void *
-rte_memcpy(void *dst, const void *src, size_t n);
+rte_memcpy(void *__rte_restrict dst, const void *__rte_restrict src, size_t n);
/**
* Copy bytes from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
* Use with n <= 15.
*/
static __rte_always_inline void *
-rte_mov15_or_less(void *dst, const void *src, size_t n)
+rte_mov15_or_less(void *__rte_restrict dst, const void *__rte_restrict src, size_t n)
{
/**
* Use the following structs to avoid violating C standard
@@ -103,10 +95,10 @@ rte_mov15_or_less(void *dst, const void *src, size_t n)
/**
* Copy 16 bytes from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static __rte_always_inline void
-rte_mov16(uint8_t *dst, const uint8_t *src)
+rte_mov16(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src)
{
__m128i xmm0;
@@ -116,10 +108,10 @@ rte_mov16(uint8_t *dst, const uint8_t *src)
/**
* Copy 32 bytes from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static __rte_always_inline void
-rte_mov32(uint8_t *dst, const uint8_t *src)
+rte_mov32(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src)
{
#if defined RTE_MEMCPY_AVX
__m256i ymm0;
@@ -132,12 +124,29 @@ rte_mov32(uint8_t *dst, const uint8_t *src)
#endif
}
+/**
+ * Copy 48 bytes from one location to another,
+ * locations must not overlap.
+ */
+static __rte_always_inline void
+rte_mov48(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src)
+{
+#if defined RTE_MEMCPY_AVX
+ rte_mov32((uint8_t *)dst, (const uint8_t *)src);
+ rte_mov16((uint8_t *)dst + 32, (const uint8_t *)src + 32);
+#else /* SSE implementation */
+ rte_mov16((uint8_t *)dst + 0 * 16, (const uint8_t *)src + 0 * 16);
+ rte_mov16((uint8_t *)dst + 1 * 16, (const uint8_t *)src + 1 * 16);
+ rte_mov16((uint8_t *)dst + 2 * 16, (const uint8_t *)src + 2 * 16);
+#endif
+}
+
/**
* Copy 64 bytes from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static __rte_always_inline void
-rte_mov64(uint8_t *dst, const uint8_t *src)
+rte_mov64(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src)
{
#if defined __AVX512F__ && defined RTE_MEMCPY_AVX512
__m512i zmm0;
@@ -152,10 +161,10 @@ rte_mov64(uint8_t *dst, const uint8_t *src)
/**
* Copy 128 bytes from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static __rte_always_inline void
-rte_mov128(uint8_t *dst, const uint8_t *src)
+rte_mov128(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src)
{
rte_mov64(dst + 0 * 64, src + 0 * 64);
rte_mov64(dst + 1 * 64, src + 1 * 64);
@@ -163,10 +172,10 @@ rte_mov128(uint8_t *dst, const uint8_t *src)
/**
* Copy 256 bytes from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static __rte_always_inline void
-rte_mov256(uint8_t *dst, const uint8_t *src)
+rte_mov256(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src)
{
rte_mov128(dst + 0 * 128, src + 0 * 128);
rte_mov128(dst + 1 * 128, src + 1 * 128);
@@ -182,10 +191,10 @@ rte_mov256(uint8_t *dst, const uint8_t *src)
/**
* Copy 128-byte blocks from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static __rte_always_inline void
-rte_mov128blocks(uint8_t *dst, const uint8_t *src, size_t n)
+rte_mov128blocks(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src, size_t n)
{
__m512i zmm0, zmm1;
@@ -202,10 +211,10 @@ rte_mov128blocks(uint8_t *dst, const uint8_t *src, size_t n)
/**
* Copy 512-byte blocks from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static inline void
-rte_mov512blocks(uint8_t *dst, const uint8_t *src, size_t n)
+rte_mov512blocks(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src, size_t n)
{
__m512i zmm0, zmm1, zmm2, zmm3, zmm4, zmm5, zmm6, zmm7;
@@ -232,45 +241,22 @@ rte_mov512blocks(uint8_t *dst, const uint8_t *src, size_t n)
}
}
+/**
+ * Copy bytes from one location to another,
+ * locations must not overlap.
+ * Use with n > 64.
+ */
static __rte_always_inline void *
-rte_memcpy_generic(void *dst, const void *src, size_t n)
+rte_memcpy_generic_more_than_64(void *__rte_restrict dst, const void *__rte_restrict src,
+ size_t n)
{
void *ret = dst;
size_t dstofss;
size_t bits;
- /**
- * Copy less than 16 bytes
- */
- if (n < 16) {
- return rte_mov15_or_less(dst, src, n);
- }
-
/**
* Fast way when copy size doesn't exceed 512 bytes
*/
- if (__rte_constant(n) && n == 32) {
- rte_mov32((uint8_t *)dst, (const uint8_t *)src);
- return ret;
- }
- if (n <= 32) {
- rte_mov16((uint8_t *)dst, (const uint8_t *)src);
- if (__rte_constant(n) && n == 16)
- return ret; /* avoid (harmless) duplicate copy */
- rte_mov16((uint8_t *)dst - 16 + n,
- (const uint8_t *)src - 16 + n);
- return ret;
- }
- if (__rte_constant(n) && n == 64) {
- rte_mov64((uint8_t *)dst, (const uint8_t *)src);
- return ret;
- }
- if (n <= 64) {
- rte_mov32((uint8_t *)dst, (const uint8_t *)src);
- rte_mov32((uint8_t *)dst - 32 + n,
- (const uint8_t *)src - 32 + n);
- return ret;
- }
if (n <= 512) {
if (n >= 256) {
n -= 256;
@@ -351,10 +337,10 @@ rte_memcpy_generic(void *dst, const void *src, size_t n)
/**
* Copy 128-byte blocks from one location to another,
- * locations should not overlap.
+ * locations must not overlap.
*/
static __rte_always_inline void
-rte_mov128blocks(uint8_t *dst, const uint8_t *src, size_t n)
+rte_mov128blocks(uint8_t *__rte_restrict dst, const uint8_t *__rte_restrict src, size_t n)
{
__m256i ymm0, ymm1, ymm2, ymm3;
@@ -381,41 +367,22 @@ rte_mov128blocks(uint8_t *dst, const uint8_t *src, size_t n)
}
}
+/**
+ * Copy bytes from one location to another,
+ * locations must not overlap.
+ * Use with n > 64.
+ */
static __rte_always_inline void *
-rte_memcpy_generic(void *dst, const void *src, size_t n)
+rte_memcpy_generic_more_than_64(void *__rte_restrict dst, const void *__rte_restrict src,
+ size_t n)
{
void *ret = dst;
size_t dstofss;
size_t bits;
- /**
- * Copy less than 16 bytes
- */
- if (n < 16) {
- return rte_mov15_or_less(dst, src, n);
- }
-
/**
* Fast way when copy size doesn't exceed 256 bytes
*/
- if (__rte_constant(n) && n == 32) {
- rte_mov32((uint8_t *)dst, (const uint8_t *)src);
- return ret;
- }
- if (n <= 32) {
- rte_mov16((uint8_t *)dst, (const uint8_t *)src);
- if (__rte_constant(n) && n == 16)
- return ret; /* avoid (harmless) duplicate copy */
- rte_mov16((uint8_t *)dst - 16 + n,
- (const uint8_t *)src - 16 + n);
- return ret;
- }
- if (n <= 64) {
- rte_mov32((uint8_t *)dst, (const uint8_t *)src);
- rte_mov32((uint8_t *)dst - 32 + n,
- (const uint8_t *)src - 32 + n);
- return ret;
- }
if (n <= 256) {
if (n >= 128) {
n -= 128;
@@ -482,7 +449,7 @@ rte_memcpy_generic(void *dst, const void *src, size_t n)
/**
* Macro for copying unaligned block from one location to another with constant load offset,
* 47 bytes leftover maximum,
- * locations should not overlap.
+ * locations must not overlap.
* Requirements:
* - Store is aligned
* - Load offset is <offset>, which must be immediate value within [1, 15]
@@ -542,7 +509,7 @@ rte_memcpy_generic(void *dst, const void *src, size_t n)
/**
* Macro for copying unaligned block from one location to another,
* 47 bytes leftover maximum,
- * locations should not overlap.
+ * locations must not overlap.
* Use switch here because the aligning instruction requires immediate value for shift count.
* Requirements:
* - Store is aligned
@@ -573,38 +540,23 @@ rte_memcpy_generic(void *dst, const void *src, size_t n)
} \
}
+/**
+ * Copy bytes from one location to another,
+ * locations must not overlap.
+ * Use with n > 64.
+ */
static __rte_always_inline void *
-rte_memcpy_generic(void *dst, const void *src, size_t n)
+rte_memcpy_generic_more_than_64(void *__rte_restrict dst, const void *__rte_restrict src,
+ size_t n)
{
__m128i xmm0, xmm1, xmm2, xmm3, xmm4, xmm5, xmm6, xmm7, xmm8;
void *ret = dst;
size_t dstofss;
size_t srcofs;
- /**
- * Copy less than 16 bytes
- */
- if (n < 16) {
- return rte_mov15_or_less(dst, src, n);
- }
-
/**
* Fast way when copy size doesn't exceed 512 bytes
*/
- if (n <= 32) {
- rte_mov16((uint8_t *)dst, (const uint8_t *)src);
- if (__rte_constant(n) && n == 16)
- return ret; /* avoid (harmless) duplicate copy */
- rte_mov16((uint8_t *)dst - 16 + n, (const uint8_t *)src - 16 + n);
- return ret;
- }
- if (n <= 64) {
- rte_mov32((uint8_t *)dst, (const uint8_t *)src);
- if (n > 48)
- rte_mov16((uint8_t *)dst + 32, (const uint8_t *)src + 32);
- rte_mov16((uint8_t *)dst - 16 + n, (const uint8_t *)src - 16 + n);
- return ret;
- }
if (n <= 128) {
goto COPY_BLOCK_128_BACK15;
}
@@ -696,44 +648,17 @@ rte_memcpy_generic(void *dst, const void *src, size_t n)
#endif /* __AVX512F__ */
+/**
+ * Copy bytes from one vector register size aligned location to another,
+ * locations must not overlap.
+ * Use with n > 64.
+ */
static __rte_always_inline void *
-rte_memcpy_aligned(void *dst, const void *src, size_t n)
+rte_memcpy_aligned_more_than_64(void *__rte_restrict dst, const void *__rte_restrict src,
+ size_t n)
{
void *ret = dst;
- /* Copy size < 16 bytes */
- if (n < 16) {
- return rte_mov15_or_less(dst, src, n);
- }
-
- /* Copy 16 <= size <= 32 bytes */
- if (__rte_constant(n) && n == 32) {
- rte_mov32((uint8_t *)dst, (const uint8_t *)src);
- return ret;
- }
- if (n <= 32) {
- rte_mov16((uint8_t *)dst, (const uint8_t *)src);
- if (__rte_constant(n) && n == 16)
- return ret; /* avoid (harmless) duplicate copy */
- rte_mov16((uint8_t *)dst - 16 + n,
- (const uint8_t *)src - 16 + n);
-
- return ret;
- }
-
- /* Copy 32 < size <= 64 bytes */
- if (__rte_constant(n) && n == 64) {
- rte_mov64((uint8_t *)dst, (const uint8_t *)src);
- return ret;
- }
- if (n <= 64) {
- rte_mov32((uint8_t *)dst, (const uint8_t *)src);
- rte_mov32((uint8_t *)dst - 32 + n,
- (const uint8_t *)src - 32 + n);
-
- return ret;
- }
-
/* Copy 64 bytes blocks */
for (; n > 64; n -= 64) {
rte_mov64((uint8_t *)dst, (const uint8_t *)src);
@@ -749,20 +674,49 @@ rte_memcpy_aligned(void *dst, const void *src, size_t n)
}
static __rte_always_inline void *
-rte_memcpy(void *dst, const void *src, size_t n)
+rte_memcpy(void *__rte_restrict dst, const void *__rte_restrict src, size_t n)
{
+ /* Fast way when copy size doesn't exceed 64 bytes. */
+ if (n < 16)
+ return rte_mov15_or_less(dst, src, n);
+ if (n <= 32) {
+ if (__rte_constant(n) && n == 32) {
+ rte_mov32((uint8_t *)dst, (const uint8_t *)src);
+ return dst;
+ }
+ rte_mov16((uint8_t *)dst, (const uint8_t *)src);
+ if (__rte_constant(n) && n == 16)
+ return dst; /* avoid (harmless) duplicate copy */
+ rte_mov16((uint8_t *)dst - 16 + n, (const uint8_t *)src - 16 + n);
+ return dst;
+ }
+ if (n <= 64) {
+ if (__rte_constant(n) && n == 64) {
+ rte_mov64((uint8_t *)dst, (const uint8_t *)src);
+ return dst;
+ }
+#if defined RTE_MEMCPY_AVX
+ rte_mov32((uint8_t *)dst, (const uint8_t *)src);
+ rte_mov32((uint8_t *)dst - 32 + n, (const uint8_t *)src - 32 + n);
+#else /* SSE implementation */
+ rte_mov16((uint8_t *)dst + 0 * 16, (const uint8_t *)src + 0 * 16);
+ rte_mov16((uint8_t *)dst + 1 * 16, (const uint8_t *)src + 1 * 16);
+ if (n > 48)
+ rte_mov16((uint8_t *)dst + 2 * 16, (const uint8_t *)src + 2 * 16);
+ rte_mov16((uint8_t *)dst - 16 + n, (const uint8_t *)src - 16 + n);
+#endif
+ return dst;
+ }
+
+ /* Implementation for size > 64 bytes depends on alignment with vector register size. */
if (!(((uintptr_t)dst | (uintptr_t)src) & ALIGNMENT_MASK))
- return rte_memcpy_aligned(dst, src, n);
+ return rte_memcpy_aligned_more_than_64(dst, src, n);
else
- return rte_memcpy_generic(dst, src, n);
+ return rte_memcpy_generic_more_than_64(dst, src, n);
}
#undef ALIGNMENT_MASK
-#if defined(RTE_TOOLCHAIN_GCC) && (GCC_VERSION >= 100000)
-#pragma GCC diagnostic pop
-#endif
-
#ifdef __cplusplus
}
#endif
--
2.43.0
^ permalink raw reply related
* [PATCH v2] app/test-pmd: add generic PROG action parser support
From: Megha Ajmera @ 2026-05-21 10:13 UTC (permalink / raw)
To: bruce.richardson, cristian.dumitrescu, praveen.shetty, dev
In-Reply-To: <20260521055612.508916-1-megha.ajmera@intel.com>
Add parser support for a generic PROG flow action in testpmd.
The update adds CLI tokens and parsing logic for program name and
argument tuples (name, size, value), enabling programmable action
configuration through the flow command interface.
Example flow rule:
flow create 0 ingress pattern eth / end actions prog name my_prog
argument name arg0 size 4 value 10 / end
Signed-off-by: Megha Ajmera <megha.ajmera@intel.com>
Signed-off-by: Praveen Shetty <praveen.shetty@intel.com>
---
v2:
* Fixed compilation warning.
app/test-pmd/cmdline_flow.c | 584 +++++++++++++++++++++++++++++++++++-
1 file changed, 581 insertions(+), 3 deletions(-)
diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index ebc036b14b..daf1de7146 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -714,6 +714,13 @@ enum index {
ACTION_SET_META,
ACTION_SET_META_DATA,
ACTION_SET_META_MASK,
+ /* ACTION PROG */
+ ACTION_PROG,
+ ACTION_PROG_NAME,
+ ACTION_PROG_ARGUMENT,
+ ACTION_PROG_ARGUMENT_NAME,
+ ACTION_PROG_ARGUMENT_SIZE,
+ ACTION_PROG_ARGUMENT_VALUE,
ACTION_SET_IPV4_DSCP,
ACTION_SET_IPV4_DSCP_VALUE,
ACTION_SET_IPV6_DSCP,
@@ -981,6 +988,24 @@ struct action_sample_data {
struct rte_flow_action_sample conf;
uint32_t idx;
};
+
+#define ACTION_PROG_MAX_ARGS 32
+#define ACTION_PROG_NAME_LEN_MAX 64
+#define ACTION_PROG_ARG_NAME_LEN_MAX 16
+
+struct action_prog_argument_data {
+ char name[ACTION_PROG_ARG_NAME_LEN_MAX];
+ uint32_t length;
+ uint32_t size;
+ uint64_t value;
+};
+
+struct action_prog_data {
+ char name[ACTION_PROG_NAME_LEN_MAX];
+ uint32_t args_num;
+ uint32_t length;
+ struct action_prog_argument_data args[ACTION_PROG_MAX_ARGS];
+};
/** Storage for struct rte_flow_action_sample. */
struct raw_sample_conf {
struct rte_flow_action data[ACTION_SAMPLE_ACTIONS_NUM];
@@ -2310,6 +2335,7 @@ static const enum index next_action[] = {
ACTION_RAW_DECAP,
ACTION_SET_TAG,
ACTION_SET_META,
+ ACTION_PROG,
ACTION_SET_IPV4_DSCP,
ACTION_SET_IPV6_DSCP,
ACTION_AGE,
@@ -2565,6 +2591,23 @@ static const enum index action_set_meta[] = {
ZERO,
};
+/** ACTION PROG */
+static const enum index next_action_prog[] = {
+ ACTION_PROG_NAME,
+ ACTION_PROG_ARGUMENT,
+ ACTION_NEXT,
+ ZERO,
+};
+
+static const enum index next_prog_arg[] = {
+ ACTION_PROG_ARGUMENT_NAME,
+ ACTION_PROG_ARGUMENT_SIZE,
+ ACTION_PROG_ARGUMENT_VALUE,
+ ACTION_PROG_ARGUMENT,
+ ACTION_NEXT,
+ ZERO,
+};
+
static const enum index action_set_ipv4_dscp[] = {
ACTION_SET_IPV4_DSCP_VALUE,
ACTION_NEXT,
@@ -2803,6 +2846,27 @@ static int parse_vc_action_set_meta(struct context *ctx,
const struct token *token, const char *str,
unsigned int len, void *buf,
unsigned int size);
+static int parse_vc_action_prog(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size);
+static int parse_vc_action_prog_argument(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static int parse_vc_action_prog_argument_name(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static int parse_vc_action_prog_argument_size(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static int parse_vc_action_prog_argument_value(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static void free_action_prog_converted(struct rte_flow_action *actions,
+ const uint32_t *converted_idx,
+ uint32_t converted_idx_num);
static int parse_vc_action_sample(struct context *ctx,
const struct token *token, const char *str,
unsigned int len, void *buf,
@@ -7816,6 +7880,48 @@ static const struct token token_list[] = {
(struct rte_flow_action_set_meta, mask)),
.call = parse_vc_conf,
},
+ [ACTION_PROG] = {
+ .name = "prog",
+ .help = "Program action: action prog name <name> [argument ...]",
+ .priv = PRIV_ACTION(PROG,
+ sizeof(struct action_prog_data)),
+ .next = NEXT(next_action_prog),
+ .call = parse_vc_action_prog,
+ },
+ [ACTION_PROG_NAME] = {
+ .name = "name",
+ .help = "Action name",
+ .next = NEXT(next_action_prog, NEXT_ENTRY(COMMON_STRING)),
+ .args = ARGS(ARGS_ENTRY_ARB(0, 0),
+ ARGS_ENTRY(struct action_prog_data, length),
+ ARGS_ENTRY_ARB(0,
+ ACTION_PROG_NAME_LEN_MAX)),
+ .call = parse_vc_conf,
+ },
+ [ACTION_PROG_ARGUMENT] = {
+ .name = "argument",
+ .help = "Keyword: argument",
+ .next = NEXT(next_prog_arg),
+ .call = parse_vc_action_prog_argument,
+ },
+ [ACTION_PROG_ARGUMENT_NAME] = {
+ .name = "name",
+ .help = "Argument Name",
+ .next = NEXT(next_prog_arg, NEXT_ENTRY(COMMON_STRING)),
+ .call = parse_vc_action_prog_argument_name,
+ },
+ [ACTION_PROG_ARGUMENT_SIZE] = {
+ .name = "size",
+ .help = "Argument size (bytes)",
+ .next = NEXT(next_prog_arg, NEXT_ENTRY(COMMON_UNSIGNED)),
+ .call = parse_vc_action_prog_argument_size,
+ },
+ [ACTION_PROG_ARGUMENT_VALUE] = {
+ .name = "value",
+ .help = "Argument value",
+ .next = NEXT(next_prog_arg, NEXT_ENTRY(COMMON_UNSIGNED)),
+ .call = parse_vc_action_prog_argument_value,
+ },
[ACTION_SET_IPV4_DSCP] = {
.name = "set_ipv4_dscp",
.help = "set DSCP value",
@@ -10415,6 +10521,449 @@ parse_vc_action_set_meta(struct context *ctx, const struct token *token,
return len;
}
+/** Parse PROG action */
+static int
+parse_vc_action_prog(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ int ret;
+
+ ret = parse_vc(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ return ret;
+
+ if (!out->args.vc.actions_n)
+ return -1;
+
+ /* Point to selected object. */
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+
+ prog_data = ctx->object;
+ prog_data->args_num = 0;
+
+ return ret;
+}
+
+/** Called when args keyword is encountered */
+static int
+parse_vc_action_prog_argument(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(size);
+
+ /* Token name must match. */
+ if (parse_default(ctx, token, str, len, NULL, 0) < 0)
+ return -1;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ return len;
+
+ if (!out->args.vc.actions_n)
+ return len;
+
+ /* Point to selected object. */
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+
+ prog_data = ctx->object;
+ if (prog_data->args_num >= ACTION_PROG_MAX_ARGS)
+ return -1;
+ prog_data->args_num++;
+
+ return len;
+}
+
+static int __prog_argument_name_args_push(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size, uint32_t arg_ind)
+{
+ static struct arg arg_addr[ACTION_PROG_MAX_ARGS];
+ static struct arg arg_len[ACTION_PROG_MAX_ARGS];
+ static struct arg arg_data[ACTION_PROG_MAX_ARGS];
+ /* Calculate the base size based on the actual structure size */
+ uint32_t prog_data_base_size = offsetof(struct action_prog_data, args);
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(len);
+ RTE_SET_USED(buf);
+ RTE_SET_USED(size);
+
+ arg_addr[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, name) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg_addr[arg_ind].size = 0;
+
+ if (push_args(ctx, &arg_addr[arg_ind]))
+ return -1;
+ arg_len[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, length) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg_len[arg_ind].size = sizeof(((struct action_prog_argument_data *)0)->length);
+
+ if (push_args(ctx, &arg_len[arg_ind]))
+ return -1;
+ arg_data[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, name) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg_data[arg_ind].size =
+ sizeof(((struct action_prog_argument_data *)0)->name);
+
+ if (push_args(ctx, &arg_data[arg_ind]))
+ return -1;
+
+ return 0;
+}
+
+static int __prog_argument_size_args_push(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size, uint32_t arg_ind)
+{
+ static struct arg arg[ACTION_PROG_MAX_ARGS];
+ uint32_t prog_data_base_size = offsetof(struct action_prog_data, args);
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(len);
+ RTE_SET_USED(buf);
+ RTE_SET_USED(size);
+
+ arg[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, size) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg[arg_ind].size = sizeof(((struct action_prog_argument_data *)0)->size);
+
+ if (push_args(ctx, &arg[arg_ind]))
+ return -1;
+
+ return 0;
+}
+
+static int __prog_argument_value_args_push(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size, uint32_t arg_ind)
+{
+ static struct arg arg[ACTION_PROG_MAX_ARGS];
+ uint32_t prog_data_base_size = offsetof(struct action_prog_data, args);
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(len);
+ RTE_SET_USED(buf);
+ RTE_SET_USED(size);
+
+ arg[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, value) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg[arg_ind].size = sizeof(((struct action_prog_argument_data *)0)->value);
+
+ if (push_args(ctx, &arg[arg_ind]))
+ return -1;
+
+ return 0;
+}
+
+static int
+parse_vc_action_prog_argument_name(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ uint32_t arg_ind;
+ uint32_t max_arg_ind = (ACTION_PROG_MAX_ARGS - 1);
+ int ret;
+
+ ret = parse_vc_conf(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ goto error_push;
+
+ if (!ctx->object)
+ goto error_push;
+
+ prog_data = ctx->object;
+ if (prog_data->args_num == 0)
+ goto error_push;
+ arg_ind = prog_data->args_num - 1;
+ if (__prog_argument_name_args_push(ctx, token, str, len, buf, size,
+ arg_ind) < 0)
+ return -1;
+
+ return ret;
+
+error_push:
+ return __prog_argument_name_args_push(ctx, token, str, len, buf, size,
+ max_arg_ind);
+}
+
+static int
+parse_vc_action_prog_argument_size(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ uint32_t arg_ind;
+ uint32_t max_arg_ind = (ACTION_PROG_MAX_ARGS - 1);
+ int ret;
+
+ ret = parse_vc_conf(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ goto error_push;
+
+ if (!ctx->object)
+ goto error_push;
+
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+ prog_data = ctx->object;
+ if (prog_data->args_num == 0)
+ goto error_push;
+ arg_ind = prog_data->args_num - 1;
+ if (__prog_argument_size_args_push(ctx, token, str, len, buf, size,
+ arg_ind) < 0)
+ return -1;
+
+ return ret;
+
+error_push:
+ return __prog_argument_size_args_push(ctx, token, str, len, buf, size,
+ max_arg_ind);
+}
+
+static int
+parse_vc_action_prog_argument_value(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ uint32_t arg_ind;
+ uint32_t max_arg_ind = (ACTION_PROG_MAX_ARGS - 1);
+ int ret;
+
+ RTE_SET_USED(size);
+
+ ret = parse_vc_conf(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ goto error_push;
+
+ if (!ctx->object)
+ goto error_push;
+
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+ prog_data = ctx->object;
+ if (prog_data->args_num == 0)
+ goto error_push;
+ arg_ind = prog_data->args_num - 1;
+ if (__prog_argument_value_args_push(ctx, token, str, len, buf, size,
+ arg_ind) < 0)
+ return -1;
+
+ return ret;
+
+error_push:
+ return __prog_argument_value_args_push(ctx, token, str, len, buf, size,
+ max_arg_ind);
+}
+
+/**
+ * Convert action_prog_data to rte_flow_action_prog.
+ * Converts PROG actions from action_prog_data format to rte_flow_action_prog format.
+ *
+ * @param actions The flow actions array
+ * @param action_count Optional count of actions. If 0, will count until END action
+ * @return 0 on success, negative value on failure
+ */
+static int
+convert_action_prog_to_rte_flow(struct rte_flow_action *actions,
+ uint32_t prog_action_count,
+ uint32_t *converted_idx,
+ uint32_t converted_idx_cap,
+ uint32_t *converted_idx_num)
+{
+ uint32_t i = 0;
+ uint32_t j;
+ uint32_t k;
+ const struct action_prog_data *prog_data;
+ struct rte_flow_action_prog *prog;
+ struct rte_flow_action_prog_argument *args;
+ uint8_t *value;
+ bool arg_error;
+ int ret = 0;
+
+ if (converted_idx_num)
+ *converted_idx_num = 0;
+
+ /* If action_count is 0, count the actions until END action */
+ if (prog_action_count == 0) {
+ while (actions[i].type != RTE_FLOW_ACTION_TYPE_END)
+ i++;
+ prog_action_count = i + 1; /* Include END action */
+ i = 0; /* Reset counter for the processing loop */
+ }
+
+ /* Process all actions */
+ for (i = 0; i < prog_action_count; i++) {
+ if (actions[i].type == RTE_FLOW_ACTION_TYPE_PROG) {
+ prog_data = (const struct action_prog_data *)actions[i].conf;
+ if (!prog_data) {
+ fprintf(stderr, "Prog action found but no data provided\n");
+ ret = -EINVAL;
+ continue;
+ }
+
+ prog = calloc(1, sizeof(struct rte_flow_action_prog));
+ if (!prog) {
+ fprintf(stderr, "Failed to allocate memory for prog action\n");
+ ret = -ENOMEM;
+ continue;
+ }
+
+ prog->name = strdup(prog_data->name);
+ if (!prog->name) {
+ fprintf(stderr, "Failed to allocate memory for prog name\n");
+ free(prog);
+ ret = -ENOMEM;
+ continue;
+ }
+
+ prog->args_num = prog_data->args_num;
+
+ if (prog->args_num > 0) {
+ args = calloc(prog->args_num,
+ sizeof(struct rte_flow_action_prog_argument));
+ if (!args) {
+ fprintf(stderr,
+ "Failed to allocate memory for prog arguments\n");
+ free((void *)(uintptr_t)prog->name);
+ free(prog);
+ ret = -ENOMEM;
+ continue;
+ }
+
+ arg_error = false;
+ for (j = 0; j < prog->args_num; j++) {
+ args[j].name = strdup(prog_data->args[j].name);
+ if (!args[j].name) {
+ fprintf(stderr,
+ "Failed to allocate memory for argument name\n");
+ ret = -ENOMEM;
+ arg_error = true;
+ break;
+ }
+
+ args[j].size = prog_data->args[j].size;
+ if (args[j].size == 0)
+ continue;
+ if (args[j].size > sizeof(prog_data->args[j].value)) {
+ free((void *)(uintptr_t)args[j].name);
+ arg_error = true;
+ ret = -EINVAL;
+ break;
+ }
+
+ value = malloc(args[j].size);
+ if (!value) {
+ fprintf(stderr,
+ "Failed to allocate memory for argument value\n");
+ free((void *)(uintptr_t)args[j].name);
+ ret = -ENOMEM;
+ arg_error = true;
+ break;
+ }
+
+ memcpy(value,
+ &prog_data->args[j].value,
+ args[j].size);
+ args[j].value = value;
+ }
+
+ if (arg_error) {
+ /* Free all allocated resources */
+ for (k = 0; k < j; k++) {
+ free((void *)(uintptr_t)args[k].name);
+ free((void *)(uintptr_t)args[k].value);
+ }
+ free(args);
+ free((void *)(uintptr_t)prog->name);
+ free(prog);
+ continue;
+ }
+
+ prog->args = args;
+ }
+
+ actions[i].conf = prog;
+ if (converted_idx && converted_idx_num &&
+ *converted_idx_num < converted_idx_cap)
+ converted_idx[(*converted_idx_num)++] = i;
+ }
+ }
+
+ return ret;
+}
+
+static void
+free_action_prog_converted(struct rte_flow_action *actions,
+ const uint32_t *converted_idx,
+ uint32_t converted_idx_num)
+{
+ uint32_t i;
+
+ for (i = 0; i < converted_idx_num; i++) {
+ uint32_t idx = converted_idx[i];
+ struct rte_flow_action_prog *prog;
+ uint32_t j;
+
+ if (actions[idx].type != RTE_FLOW_ACTION_TYPE_PROG)
+ continue;
+
+ prog = (struct rte_flow_action_prog *)(uintptr_t)actions[idx].conf;
+ if (!prog)
+ continue;
+
+ for (j = 0; j < prog->args_num; j++) {
+ free((void *)(uintptr_t)prog->args[j].name);
+ free((void *)(uintptr_t)prog->args[j].value);
+ }
+ free((void *)(uintptr_t)prog->args);
+ free((void *)(uintptr_t)prog->name);
+ free(prog);
+ }
+}
+
static int
parse_vc_action_sample(struct context *ctx, const struct token *token,
const char *str, unsigned int len, void *buf,
@@ -13380,6 +13929,8 @@ indirect_action_list_conf_get(uint32_t conf_id)
static void
cmd_flow_parsed(const struct buffer *in)
{
+ int ret;
+
switch (in->command) {
case INFO:
port_flow_get_info(in->port);
@@ -13561,9 +14112,36 @@ cmd_flow_parsed(const struct buffer *in)
&in->args.vc.tunnel_ops);
break;
case CREATE:
- port_flow_create(in->port, &in->args.vc.attr,
- in->args.vc.pattern, in->args.vc.actions,
- &in->args.vc.tunnel_ops, in->args.vc.user_id);
+ {
+ uint32_t *converted_idx;
+ uint32_t converted_idx_num = 0;
+
+ converted_idx = calloc(in->args.vc.actions_n,
+ sizeof(*converted_idx));
+ if (!converted_idx) {
+ fprintf(stderr,
+ "Warning: Failed to allocate conversion index list\n");
+ break;
+ }
+
+ /* Convert from action_prog_data to rte_flow_action_prog. */
+ ret = convert_action_prog_to_rte_flow(in->args.vc.actions,
+ 0,
+ converted_idx,
+ in->args.vc.actions_n,
+ &converted_idx_num);
+ if (ret < 0)
+ fprintf(stderr,
+ "Warning: Failed to convert program action data: %s\n",
+ strerror(-ret));
+ port_flow_create(in->port, &in->args.vc.attr,
+ in->args.vc.pattern, in->args.vc.actions,
+ &in->args.vc.tunnel_ops, in->args.vc.user_id);
+ free_action_prog_converted(in->args.vc.actions,
+ converted_idx,
+ converted_idx_num);
+ free(converted_idx);
+ }
break;
case DESTROY:
port_flow_destroy(in->port, in->args.destroy.rule_n,
--
2.34.1
^ permalink raw reply related
* Re: [PATCH v3] net/mlx5: add validation for indirect actions
From: Dariusz Sosnowski @ 2026-05-21 10:00 UTC (permalink / raw)
To: Rayane Boussanni; +Cc: dev, rasland
In-Reply-To: <20260514193359.195017-1-rboussanni@gmail.com>
On Thu, May 14, 2026 at 03:33:59PM -0400, Rayane Boussanni wrote:
> This patch implements missing validation logic for RSS and Connection
> Tracking (ConnTrack) indirect actions in the Hardware Steering (HWS)
> flow engine.
>
> Previously, these actions were accepted without being validated
> against hardware capabilities, which could lead to unexpected behavior
> when applying flow rules. The specialist validation functions
> (mlx5_hw_validate_action_rss and mlx5_hw_validate_action_conntrack)
> already existed but were not wired up to the indirect action handler.
>
> The signature of flow_hw_validate_action_indirect was updated to
> include the actions template attributes (attr), allowing it to pass
> the necessary traffic direction context (ingress/egress/transfer)
> to the underlying validation specialists. For indirect RSS, only the
> template attributes are validated, as the RSS configuration itself is
> already validated when the indirect action handle is created.
>
> Reported-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Signed-off-by: Rayane Boussanni <rboussanni@gmail.com>
Acked-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Thank you for the contribution.
Could you please mark the older versions of this patch as Superseded in Patchwork?
- https://patches.dpdk.org/project/dpdk/patch/20260416001052.2624-1-rboussanni@gmail.com/
- https://patches.dpdk.org/project/dpdk/patch/20260417150153.63149-1-rboussanni@gmail.com/
- https://patches.dpdk.org/project/dpdk/patch/20260417222104.66543-1-rboussanni@gmail.com/
Best regards,
Dariusz Sosnowski
^ permalink raw reply
* Re: [RFC PATCH 0/8] remove use of rte_memcpy from net/intel
From: Bruce Richardson @ 2026-05-21 9:48 UTC (permalink / raw)
To: Morten Brørup; +Cc: stephen, dev
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35F65883@smartserver.smartshare.dk>
On Tue, May 19, 2026 at 07:57:47PM +0200, Morten Brørup wrote:
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Tuesday, 19 May 2026 18.06
> >
> > This RFC proposed to replace all instances of rte_memcpy in Intel
> > (and former-Intel) net drivers with just regular memcpy. This is
> > done on the basis that the memcpy use is not datapath, but is used
> > for flow configuration, virt-channel (to firmware or PF) messaging
> > and other control path functions.
> >
>
> Series-acked-by: Morten Brørup <mb@smartsharesystems.com>
>
Series applied to dpdk-next-net-intel, [including ack from Rosen Xu on the
last patch, received offlist.]
/Bruce
^ permalink raw reply
* Re: [PATCH v3] net/mlx5: prepend implicit items in sync flow creation path
From: Dariusz Sosnowski @ 2026-05-21 9:34 UTC (permalink / raw)
To: Maxime Peim; +Cc: dev
In-Reply-To: <20260427123217.510662-1-maxime.peim@gmail.com>
On Mon, Apr 27, 2026 at 02:32:17PM +0200, Maxime Peim wrote:
> In eSwitch mode, the async (template) flow creation path automatically
> prepends implicit match items to scope flow rules to the correct
> representor port:
> - Ingress: REPRESENTED_PORT item matching dev->data->port_id
> - Egress: REG_C_0 TAG item matching the port's tx tag value
>
> The sync path (flow_hw_list_create) was missing this logic, causing all
> flow rules created via the non-template API to match traffic from all
> ports rather than being scoped to the specific representor.
>
> Add the same implicit item prepending to flow_hw_list_create, right
> after pattern validation and before any branching (sample/RSS/single/
> prefix), mirroring the behavior of flow_hw_pattern_template_create
> and flow_hw_get_rule_items. The ingress case prepends
> REPRESENTED_PORT with the current port_id; the egress case prepends
> MLX5_RTE_FLOW_ITEM_TYPE_TAG with REG_C_0 value/mask (skipped when
> user provides an explicit SQ item).
>
> Also fix a pre-existing bug where 'return split' on metadata split
> failure returned a negative int cast to uintptr_t, which callers
> would treat as a valid flow handle instead of an error.
>
> Fixes: e38776c36c8a ("net/mlx5: introduce HWS for non-template flow API")
> Fixes: 821a6a5cc495 ("net/mlx5: add metadata split for compatibility")
> Signed-off-by: Maxime Peim <maxime.peim@gmail.com>
Since you sent v4 to the mailing list,
could you please mark v3 of this patch as superseded in Patchwork [1]?
[1]: https://patches.dpdk.org/project/dpdk/patch/20260427123217.510662-1-maxime.peim@gmail.com/
Best regards,
Dariusz Sosnowski
^ permalink raw reply
* Re: [PATCH v4] net/mlx5: prepend implicit items in sync flow creation path
From: Dariusz Sosnowski @ 2026-05-21 9:30 UTC (permalink / raw)
To: Maxime Peim; +Cc: dev, viacheslavo, bingz, orika, suanmingm, matan
In-Reply-To: <20260511150854.1398044-1-maxime.peim@gmail.com>
On Mon, May 11, 2026 at 05:08:50PM +0200, Maxime Peim wrote:
> In eSwitch mode, the async (template) flow creation path automatically
> prepends implicit match items to scope flow rules to the correct
> representor port:
> - Ingress: REPRESENTED_PORT item matching dev->data->port_id
> - Egress: REG_C_0 TAG item matching the port's tx tag value
>
> The sync path (flow_hw_list_create) was missing this logic, causing all
> flow rules created via the non-template API to match traffic from all
> ports rather than being scoped to the specific representor.
>
> Add the same implicit item prepending to flow_hw_list_create, right
> after pattern validation and before any branching (sample/RSS/single/
> prefix), mirroring the behavior of flow_hw_pattern_template_create
> and flow_hw_get_rule_items. The ingress case prepends
> REPRESENTED_PORT with the current port_id; the egress case prepends
> MLX5_RTE_FLOW_ITEM_TYPE_TAG with REG_C_0 value/mask (skipped when
> user provides an explicit SQ item).
>
> Also fix a pre-existing bug where 'return split' on metadata split
> failure returned a negative int cast to uintptr_t, which callers
> would treat as a valid flow handle instead of an error.
>
> Fixes: e38776c36c8a ("net/mlx5: introduce HWS for non-template flow API")
> Fixes: 821a6a5cc495 ("net/mlx5: add metadata split for compatibility")
> Signed-off-by: Maxime Peim <maxime.peim@gmail.com>
> ---
> v3:
> - Factor the implicit-item prepend logic out of
> flow_hw_pattern_template_create() into a new helper
> flow_hw_adjust_pattern() and reuse it from flow_hw_list_create(),
> instead of duplicating the prepend logic inline in the sync path.
> - Zero-initialize item_flags in both callers. The validator is
> read-modify-write on item_flags (reads MLX5_FLOW_LAYER_TUNNEL on
> the first iteration), so leaving it uninitialized was UB.
> - Call __flow_hw_pattern_validate() with nt_flow=true from the sync
> path (was effectively nt_flow=false via the wrapper), restoring the
> previous behavior that skips GENEVE_OPT TLV parser validation on
> the non-template path.
> - Document flow_hw_adjust_pattern(): the dual role of the nt_flow
> parameter (template spec-left-zero vs. sync spec-filled + validator
> flag), the three-way return, and the caller's ownership of
> *copied_items across every exit path.
> - Clarify the "omitting implicit REG_C_0 match" debug log now that
> the helper runs on both the template and sync paths.
> - Add Fixes: tags for the two original commits.
>
> v4:
> - Fix items in case splitted metadata are not needed.
>
> drivers/net/mlx5/mlx5_flow_hw.c | 194 ++++++++++++++++++++++----------
> 1 file changed, 132 insertions(+), 62 deletions(-)
>
> diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
> index bca5b2769e..6b3fcb43a7 100644
> --- a/drivers/net/mlx5/mlx5_flow_hw.c
> +++ b/drivers/net/mlx5/mlx5_flow_hw.c
> @@ -9255,33 +9255,40 @@ pattern_template_validate(struct rte_eth_dev *dev,
> return -ret;
> }
>
> -/**
> - * Create flow item template.
> +/*
> + * Validate the user-supplied items and, in eSwitch mode, prepend the implicit
> + * scoping item so the rule/template is bound to the current representor port:
> + * - ingress -> RTE_FLOW_ITEM_TYPE_REPRESENTED_PORT (dev->data->port_id)
> + * - egress -> MLX5_RTE_FLOW_ITEM_TYPE_TAG on REG_C_0 (tx vport tag),
> + * skipped when the user already supplied an SQ item.
> *
> - * @param[in] dev
> - * Pointer to the rte_eth_dev structure.
> - * @param[in] attr
> - * Pointer to the item template attributes.
> - * @param[in] items
> - * The template item pattern.
> - * @param[out] error
> - * Pointer to error structure.
> + * @param nt_flow
> + * Selects between the two call paths that share this helper:
> + * false -> pattern template creation (async API). The prepended item's
> + * spec is left zeroed so mlx5dr matches any value; the live
> + * port_id / tx-tag value is substituted later by
> + * flow_hw_get_rule_items() at rule-create time.
> + * true -> sync (non-template) flow creation. The prepended item's spec
> + * is filled immediately with the live values, and the flag is
> + * forwarded to __flow_hw_pattern_validate() so that validation
> + * paths gated on nt_flow (e.g. GENEVE_OPT TLV parser creation)
> + * take the non-template branch.
> *
> - * @return
> - * Item template pointer on success, NULL otherwise and rte_errno is set.
> + * Return / ownership:
> + * - NULL on validation or allocation failure (error populated).
> + * - `items` unchanged when no prepending is required; *copied_items == NULL.
> + * - A newly-allocated array otherwise; also stored in *copied_items. The
> + * caller must mlx5_free(*copied_items) on every path (it is safe to call
> + * with NULL). Do not free the returned pointer directly.
> */
> -static struct rte_flow_pattern_template *
> -flow_hw_pattern_template_create(struct rte_eth_dev *dev,
> - const struct rte_flow_pattern_template_attr *attr,
> - const struct rte_flow_item items[],
> - bool external,
> - struct rte_flow_error *error)
> +static const struct rte_flow_item *
> +flow_hw_adjust_pattern(struct rte_eth_dev *dev, const struct rte_flow_pattern_template_attr *attr,
> + bool nt_flow, const struct rte_flow_item *items, uint64_t *item_flags,
> + uint64_t *nb_items, struct rte_flow_item **copied_items,
> + struct rte_flow_error *error)
It seems that there's an issue with dangling pointer in flow_hw_adjust_pattern():
- flow_hw_adjust_pattern() defines TAG/REPRESENTED_PORT item spec/mask on
its local stack frame,
- If prepending is needed, then flow_hw_prepend_item() will copy only
the rte_flow_item struct. spec/mask is not copied.
- After flow_hw_adjust_pattern() finishes,
"copied_items" will have an item with spec/mask pointing to invalid stack data.
This wasn't previously a problem in flow_hw_pattern_template_create(),
because items' spec/mask were only needed during the duration of that
function.
Could you please fix the above? It'll require extending amount of memory
allocated by flow_hw_prepend_item(), ideally everything should be done
in single allocation, so freeing copied_items would still require
single call to mlx5_free().
ASAN, with detect_stack_use_after_return enabled, report is below.
# repro steps
meson setup build -Dbuildtype=debug -Db_sanitize=address -Denable_drivers=bus/auxiliary,common/mlx5,net/mlx5 -Denable_apps=test-pmd
meson compile -C build -j $(nproc --ignore 1)
env ASAN_OPTIONS=detect_stack_use_after_return=1 ./build/app/dpdk-testpmd -a 08:00.0,dv_flow_en=2,representor=vf0-1 -- --flow-isolate-all -i
testpmd> flow create 0 group 0 priority 0 ingress pattern end actions jump group 1 / end
# ASAN report
=================================================================
==59105==ERROR: AddressSanitizer: stack-use-after-return on address 0x71e744675730 at pc 0x5d70bed42c6d bp 0x7fff52519b40 sp 0x7fff52519b30
READ of size 2 at 0x71e744675730 thread T0
#0 0x5d70bed42c6c in flow_dv_translate_item_represented_port ../drivers/net/mlx5/mlx5_flow_dv.c:11059
#1 0x5d70bed5c9b1 in flow_dv_translate_items ../drivers/net/mlx5/mlx5_flow_dv.c:14301
#2 0x5d70bed60620 in flow_dv_translate_items_nta ../drivers/net/mlx5/mlx5_flow_dv.c:14646
#3 0x5d70bed60fb5 in mlx5_flow_dv_translate_items_hws_impl ../drivers/net/mlx5/mlx5_flow_dv.c:14714
#4 0x5d70c1288ea2 in mlx5_flow_hw_create_flow ../drivers/net/mlx5/mlx5_flow_hw.c:14134
#5 0x5d70c128e45c in flow_hw_list_create ../drivers/net/mlx5/mlx5_flow_hw.c:14394
#6 0x5d70beca71b9 in mlx5_flow_list_create ../drivers/net/mlx5/mlx5_flow.c:8068
#7 0x5d70beca6e33 in mlx5_flow_create ../drivers/net/mlx5/mlx5_flow.c:8044
#8 0x5d70be95670e in rte_flow_create ../lib/ethdev/rte_flow.c:426
#9 0x5d70bdcaf19a in port_flow_create ../app/test-pmd/config.c:3869
#10 0x5d70bdc3983d in cmd_flow_parsed ../app/test-pmd/cmdline_flow.c:13564
#11 0x5d70bdc3a376 in cmd_flow_cb ../app/test-pmd/cmdline_flow.c:13634
#12 0x5d70be8ad00a in __cmdline_parse ../lib/cmdline/cmdline_parse.c:296
#13 0x5d70be8ad0b8 in cmdline_parse ../lib/cmdline/cmdline_parse.c:305
#14 0x5d70be8a87ef in cmdline_valid_buffer ../lib/cmdline/cmdline.c:25
#15 0x5d70be8b405e in rdline_char_in ../lib/cmdline/cmdline_rdline.c:470
#16 0x5d70be8a9097 in cmdline_in ../lib/cmdline/cmdline.c:154
#17 0x5d70be8a932f in cmdline_interact ../lib/cmdline/cmdline.c:202
#18 0x5d70bdc12af6 in prompt ../app/test-pmd/cmdline.c:14586
#19 0x5d70bdd5d636 in main ../app/test-pmd/testpmd.c:4792
#20 0x71e746c2a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
#21 0x71e746c2a28a in __libc_start_main_impl ../csu/libc-start.c:360
#22 0x5d70bdbc0944 in _start (/auto/swgwork9/dsosnowski/code/nvidia/dpdk/build/app/dpdk-testpmd+0x358944) (BuildId: 04ff6d26a7ff0a65637cb32f087bae007e00c965)
Address 0x71e744675730 is located in stack of thread T0 at offset 48 in frame
#0 0x5d70c115c601 in flow_hw_adjust_pattern ../drivers/net/mlx5/mlx5_flow_hw.c:9289
This frame has 5 object(s):
[48, 50) 'port_spec' (line 9291) <== Memory access at offset 48 is inside this variable
[64, 72) 'tag_v' (line 9296)
[96, 104) 'tag_m' (line 9300)
[128, 160) 'port' (line 9292)
[192, 224) 'tag' (line 9304)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-use-after-return ../drivers/net/mlx5/mlx5_flow_dv.c:11059 in flow_dv_translate_item_represented_port
Shadow bytes around the buggy address:
0x71e744675480: f5 f5 f5 f5 00 00 00 00 00 00 00 00 00 00 00 00
0x71e744675500: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
0x71e744675580: f5 f5 f5 f5 f5 f5 f5 f5 00 00 00 00 00 00 00 00
0x71e744675600: f1 f1 f1 f1 f1 f1 02 f2 00 f2 f2 f2 00 f2 f2 f2
0x71e744675680: 00 f2 f2 f2 00 f3 f3 f3 00 00 00 00 00 00 00 00
=>0x71e744675700: f5 f5 f5 f5 f5 f5[f5]f5 f5 f5 f5 f5 f5 f5 f5 f5
0x71e744675780: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
0x71e744675800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x71e744675880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x71e744675900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x71e744675980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==59105==ABORTING
> {
> struct mlx5_priv *priv = dev->data->dev_private;
> - struct rte_flow_pattern_template *it;
> - struct rte_flow_item *copied_items = NULL;
> - const struct rte_flow_item *tmpl_items;
> - uint64_t orig_item_nb, item_flags = 0;
> + struct rte_flow_item_ethdev port_spec = {.port_id = dev->data->port_id};
> struct rte_flow_item port = {
> .type = RTE_FLOW_ITEM_TYPE_REPRESENTED_PORT,
> .mask = &rte_flow_item_ethdev_mask,
> @@ -9298,39 +9305,89 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
> .type = (enum rte_flow_item_type)MLX5_RTE_FLOW_ITEM_TYPE_TAG,
> .spec = &tag_v,
> .mask = &tag_m,
> - .last = NULL
> + .last = NULL,
> };
> - int it_items_size;
> - unsigned int i = 0;
> int rc;
>
> + if (!copied_items || !item_flags || !nb_items)
> + return NULL;
> +
> + if (nt_flow) {
> + port.spec = &port_spec;
> + tag_v.data = flow_hw_tx_tag_regc_value(dev);
> + }
> +
> + /*
> + * item_flags must be zero-initialized: __flow_hw_pattern_validate()
> + * OR-accumulates bits into it and reads it (MLX5_FLOW_LAYER_TUNNEL)
> + * on the first iteration.
> + */
> + *item_flags = 0;
> +
> /* Validate application items only */
> - rc = flow_hw_pattern_validate(dev, attr, items, &item_flags, error);
> + rc = __flow_hw_pattern_validate(dev, attr, items, item_flags, nt_flow, error);
> if (rc < 0)
> return NULL;
> - orig_item_nb = rc;
> - if (priv->sh->config.dv_esw_en &&
> - attr->ingress && !attr->egress && !attr->transfer) {
> - copied_items = flow_hw_prepend_item(items, orig_item_nb, &port, error);
> - if (!copied_items)
> + *nb_items = rc;
> +
> + if (priv->sh->config.dv_esw_en && attr->ingress && !attr->egress && !attr->transfer) {
> + *copied_items = flow_hw_prepend_item(items, *nb_items, &port, error);
> + if (!*copied_items)
> return NULL;
> - tmpl_items = copied_items;
> - } else if (priv->sh->config.dv_esw_en &&
> - !attr->ingress && attr->egress && !attr->transfer) {
> - if (item_flags & MLX5_FLOW_ITEM_SQ) {
> - DRV_LOG(DEBUG, "Port %u omitting implicit REG_C_0 match for egress "
> - "pattern template", dev->data->port_id);
> - tmpl_items = items;
> - goto setup_pattern_template;
> + return *copied_items;
> + } else if (priv->sh->config.dv_esw_en && !attr->ingress && attr->egress &&
> + !attr->transfer) {
> + if (*item_flags & MLX5_FLOW_ITEM_SQ) {
> + DRV_LOG(DEBUG,
> + "Port %u: explicit SQ item present, omitting implicit "
> + "REG_C_0 match for egress pattern",
> + dev->data->port_id);
> + return items;
> }
> - copied_items = flow_hw_prepend_item(items, orig_item_nb, &tag, error);
> - if (!copied_items)
> + *copied_items = flow_hw_prepend_item(items, *nb_items, &tag, error);
> + if (!*copied_items)
> return NULL;
> - tmpl_items = copied_items;
> - } else {
> - tmpl_items = items;
> + return *copied_items;
> }
> -setup_pattern_template:
> + return items;
> +}
> +
> +/**
> + * Create flow item template.
> + *
> + * @param[in] dev
> + * Pointer to the rte_eth_dev structure.
> + * @param[in] attr
> + * Pointer to the item template attributes.
> + * @param[in] items
> + * The template item pattern.
> + * @param[out] error
> + * Pointer to error structure.
> + *
> + * @return
> + * Item template pointer on success, NULL otherwise and rte_errno is set.
> + */
> +static struct rte_flow_pattern_template *
> +flow_hw_pattern_template_create(struct rte_eth_dev *dev,
> + const struct rte_flow_pattern_template_attr *attr,
> + const struct rte_flow_item items[],
> + bool external,
> + struct rte_flow_error *error)
> +{
> + struct mlx5_priv *priv = dev->data->dev_private;
> + struct rte_flow_pattern_template *it;
> + struct rte_flow_item *copied_items = NULL;
> + const struct rte_flow_item *tmpl_items;
> + int it_items_size;
> + uint64_t orig_item_nb, item_flags;
> + unsigned int i = 0;
> + int rc;
> +
> + tmpl_items = flow_hw_adjust_pattern(dev, attr, false, items, &orig_item_nb, &item_flags,
> + &copied_items, error);
This is something I missed on v3, sorry about that.
flow_hw_adjust_pattern() takes:
- "item_flags" as 5th argument,
- "nb_items" as 6th argument.
But both call sites of flow_hw_adjust_pattern() pass them
in reverse order.
Please rearrange the call sites.
> + if (!tmpl_items)
> + return NULL;
> +
> it = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*it), 0, SOCKET_ID_ANY);
> if (!it) {
> rte_flow_error_set(error, ENOMEM,
> @@ -14272,7 +14329,6 @@ static uintptr_t flow_hw_list_create(struct rte_eth_dev *dev,
> struct rte_flow_hw *prfx_flow = NULL;
> const struct rte_flow_action *qrss = NULL;
> const struct rte_flow_action *mark = NULL;
> - uint64_t item_flags = 0;
> uint64_t action_flags = mlx5_flow_hw_action_flags_get(actions, &qrss, &mark,
> &encap_idx, &actions_n, error);
> struct mlx5_flow_hw_split_resource resource = {
> @@ -14289,20 +14345,27 @@ static uintptr_t flow_hw_list_create(struct rte_eth_dev *dev,
> .egress = attr->egress,
> .transfer = attr->transfer,
> };
> -
> - /* Validate application items only */
> - ret = __flow_hw_pattern_validate(dev, &pattern_template_attr, items,
> - &item_flags, true, error);
> - if (ret < 0)
> - return 0;
> + struct rte_flow_item *copied_items = NULL;
> + const struct rte_flow_item *prepend_items;
> + uint64_t orig_item_nb, item_flags;
>
> RTE_SET_USED(encap_idx);
> if (!error)
> error = &shadow_error;
> +
> + prepend_items = flow_hw_adjust_pattern(dev, &pattern_template_attr, true, items,
> + &orig_item_nb, &item_flags, &copied_items, error);
> + if (!prepend_items)
> + return 0;
> +
> split = mlx5_flow_nta_split_metadata(dev, attr, actions, qrss, action_flags,
> actions_n, external, &resource, error);
> - if (split < 0)
> - return split;
> + if (split < 0) {
> + mlx5_free(copied_items);
> + return 0;
> + } else if (!split) {
> + resource.suffix.items = prepend_items;
> + }
>
> /* Update the metadata copy table - MLX5_FLOW_MREG_CP_TABLE_GROUP */
> if (((attr->ingress && attr->group != MLX5_FLOW_MREG_CP_TABLE_GROUP) ||
> @@ -14313,23 +14376,26 @@ static uintptr_t flow_hw_list_create(struct rte_eth_dev *dev,
> goto free;
> }
> if (action_flags & MLX5_FLOW_ACTION_SAMPLE) {
> - flow = mlx5_nta_sample_flow_list_create(dev, type, attr, items, actions,
> + flow = mlx5_nta_sample_flow_list_create(dev, type, attr, prepend_items, actions,
> item_flags, action_flags, error);
> - if (flow != NULL)
> + if (flow != NULL) {
> + mlx5_free(copied_items);
> return (uintptr_t)flow;
> + }
> goto free;
> }
> if (action_flags & MLX5_FLOW_ACTION_RSS) {
> const struct rte_flow_action_rss
> *rss_conf = mlx5_flow_nta_locate_rss(dev, actions, error);
> - flow = mlx5_flow_nta_handle_rss(dev, attr, items, actions, rss_conf,
> - item_flags, action_flags, external,
> - type, error);
> + flow = mlx5_flow_nta_handle_rss(dev, attr, prepend_items, actions, rss_conf,
> + item_flags, action_flags, external, type, error);
> if (flow) {
> flow->nt2hws->rix_mreg_copy = cpy_idx;
> cpy_idx = 0;
> - if (!split)
> + if (!split) {
> + mlx5_free(copied_items);
> return (uintptr_t)flow;
> + }
> goto prefix_flow;
> }
> goto free;
> @@ -14343,12 +14409,14 @@ static uintptr_t flow_hw_list_create(struct rte_eth_dev *dev,
> if (flow) {
> flow->nt2hws->rix_mreg_copy = cpy_idx;
> cpy_idx = 0;
> - if (!split)
> + if (!split) {
> + mlx5_free(copied_items);
> return (uintptr_t)flow;
> + }
> /* Fall Through to prefix flow creation. */
> }
> prefix_flow:
> - ret = mlx5_flow_hw_create_flow(dev, type, attr, items, resource.prefix.actions,
> + ret = mlx5_flow_hw_create_flow(dev, type, attr, prepend_items, resource.prefix.actions,
> item_flags, action_flags, external, &prfx_flow, error);
> if (ret)
> goto free;
> @@ -14357,6 +14425,7 @@ static uintptr_t flow_hw_list_create(struct rte_eth_dev *dev,
> flow->nt2hws->chaned_flow = 1;
> SLIST_INSERT_AFTER(prfx_flow, flow, nt2hws->next);
> mlx5_flow_nta_split_resource_free(dev, &resource);
> + mlx5_free(copied_items);
> return (uintptr_t)prfx_flow;
> }
> free:
> @@ -14368,6 +14437,7 @@ static uintptr_t flow_hw_list_create(struct rte_eth_dev *dev,
> mlx5_flow_nta_del_copy_action(dev, cpy_idx);
> if (split > 0)
> mlx5_flow_nta_split_resource_free(dev, &resource);
> + mlx5_free(copied_items);
> return 0;
> }
>
^ permalink raw reply
* [PATCH] common/cnxk: fix VFIO MSI-X interrupt setup
From: pbhagavatula @ 2026-05-21 8:26 UTC (permalink / raw)
To: jerinj, Nithin Dabilpuram, Kiran Kumar K, Sunil Kumar Kori,
Satha Rao, Harman Kalra, Stephen Hemminger, Tejasree Kondoj
Cc: dev, Pavan Nikhilesh, stable
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Use heap allocation sized by the configured maximum interrupt count for the
VFIO irq_set buffer, correct handling when irq.count is zero, and use a
minimal stack buffer for per-vector configuration.
Fixes: 1fb9f4ab14b3 ("common/cnxk: remove VLA in interrupt configuration")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
drivers/common/cnxk/roc_platform.c | 39 +++++++++++++++++++-----------
1 file changed, 25 insertions(+), 14 deletions(-)
diff --git a/drivers/common/cnxk/roc_platform.c b/drivers/common/cnxk/roc_platform.c
index 1fdbf8f05143..59d662fab43b 100644
--- a/drivers/common/cnxk/roc_platform.c
+++ b/drivers/common/cnxk/roc_platform.c
@@ -13,13 +13,11 @@
#if defined(__linux__)
#include <inttypes.h>
+#include <stdlib.h>
#include <sys/eventfd.h>
#include <sys/ioctl.h>
#include <unistd.h>
-#define MSIX_IRQ_SET_BUF_LEN \
- (sizeof(struct vfio_irq_set) + sizeof(int) * PLT_MAX_RXTX_INTR_VEC_ID)
-
static int
irq_get_info(struct plt_intr_handle *intr_handle)
{
@@ -39,11 +37,12 @@ irq_get_info(struct plt_intr_handle *intr_handle)
irq.count, PLT_MAX_RXTX_INTR_VEC_ID);
if (irq.count == 0) {
- plt_err("HW max=%d > PLT_MAX_RXTX_INTR_VEC_ID: %d", irq.count,
- PLT_MAX_RXTX_INTR_VEC_ID);
- plt_intr_max_intr_set(intr_handle, PLT_MAX_RXTX_INTR_VEC_ID);
+ plt_warn("VFIO MSI-X irq.count is 0; using PLT_MAX_RXTX_INTR_VEC_ID=%u",
+ PLT_MAX_RXTX_INTR_VEC_ID);
+ if (plt_intr_max_intr_set(intr_handle, PLT_MAX_RXTX_INTR_VEC_ID))
+ return -1;
} else {
- if (plt_intr_max_intr_set(intr_handle, irq.count))
+ if (plt_intr_max_intr_set(intr_handle, (int)irq.count))
return -1;
}
@@ -53,7 +52,7 @@ irq_get_info(struct plt_intr_handle *intr_handle)
static int
irq_config(struct plt_intr_handle *intr_handle, unsigned int vec)
{
- char irq_set_buf[MSIX_IRQ_SET_BUF_LEN];
+ char irq_set_buf[sizeof(struct vfio_irq_set) + sizeof(int32_t)];
struct vfio_irq_set *irq_set;
int len, rc, vfio_dev_fd;
int32_t *fd_ptr;
@@ -89,23 +88,34 @@ irq_config(struct plt_intr_handle *intr_handle, unsigned int vec)
static int
irq_init(struct plt_intr_handle *intr_handle)
{
- char irq_set_buf[MSIX_IRQ_SET_BUF_LEN];
+ int max_intr, len, rc, vfio_dev_fd;
struct vfio_irq_set *irq_set;
- int len, rc, vfio_dev_fd;
+ char *irq_set_buf = NULL;
int32_t *fd_ptr;
- uint32_t i;
+ int i;
- len = sizeof(struct vfio_irq_set) + sizeof(int32_t) * plt_intr_max_intr_get(intr_handle);
+ max_intr = plt_intr_max_intr_get(intr_handle);
+ if (max_intr <= 0) {
+ plt_err("Invalid max_intr %d for irq init", max_intr);
+ return -EINVAL;
+ }
+
+ len = sizeof(struct vfio_irq_set) + sizeof(int32_t) * max_intr;
+ irq_set_buf = malloc((size_t)len);
+ if (irq_set_buf == NULL) {
+ plt_err("Failed to alloc irq_set_buf len=%d", len);
+ return -ENOMEM;
+ }
irq_set = (struct vfio_irq_set *)irq_set_buf;
irq_set->argsz = len;
irq_set->start = 0;
- irq_set->count = plt_intr_max_intr_get(intr_handle);
+ irq_set->count = (uint32_t)max_intr;
irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER;
irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX;
fd_ptr = (int32_t *)&irq_set->data[0];
- for (i = 0; i < irq_set->count; i++)
+ for (i = 0; i < max_intr; i++)
fd_ptr[i] = -1;
vfio_dev_fd = plt_intr_dev_fd_get(intr_handle);
@@ -113,6 +123,7 @@ irq_init(struct plt_intr_handle *intr_handle)
if (rc)
plt_err("Failed to set irqs vector rc=%d", rc);
+ free(irq_set_buf);
return rc;
}
--
2.50.1 (Apple Git-155)
^ permalink raw reply related
* Re: [PATCH v2 00/23] Consolidate bus driver infrastructure
From: Bruce Richardson @ 2026-05-21 8:13 UTC (permalink / raw)
To: David Marchand; +Cc: dev, thomas, stephen
In-Reply-To: <20260506155201.2709810-1-david.marchand@redhat.com>
On Wed, May 06, 2026 at 05:51:32PM +0200, David Marchand wrote:
> This is a continuation of the work I started on the bus infrastructure,
> but this time, a lot of the changes were done by a AI "friend".
> It is still an unfinished topic as the current series focuses on probing
> only. The detaching/cleanup aspect is postponed to another release/time.
>
> My AI "friend" really *sucked* at git and at separating unrelated changes,
> so it required quite a lot of massage/polishing afterwards.
> But it seems good enough now for upstream submission.
>
> I would like to see this series merged in 26.07, so that we have enough
> time to stabilize it before the next LTS.
> And seeing how it affects drivers, it is probably better to merge it
> the sooner possible (so Thomas does not have to solve too many conflicts
> when pulling next-* subtrees after, especially wrt the last patch).
>
>
> This series refactors the DPDK bus infrastructure to consolidate common
> operations and reduce code duplication across all bus drivers.
> Currently, each bus implements its own specific device/driver lists,
> probe logic, and lookup functions.
> This series moves these common patterns into the EAL bus layer,
> providing generic helpers that all buses can use.
>
> The refactoring removes approximately 1,400 lines of duplicated code across
> the codebase while maintaining full functional equivalence.
>
> Key changes:
> - Factorize device and driver lists into struct rte_bus
> - Implement generic probe, device/driver lookup, and iteration helpers in EAL
> - Introduce conversion macros (RTE_BUS_DEVICE, RTE_BUS_DRIVER, RTE_CLASS_TO_BUS_DEVICE)
> to safely convert between generic and bus-specific types
> - Remove bus-specific device/driver types from most driver code
> - Move probe logic from individual buses to rte_bus_generic_probe()
> - Separate NXP-specific metadata from generic bus structures
>
> Benefits:
> - Significant code reduction (~1,400 lines removed)
> - Consistent behavior across all bus types
> - Simplified bus driver implementation
> - Easier maintenance and future enhancements
>
> The series is structured as a progressive refactoring:
> - Remove redundant checks and helpers (patches 1-5)
> - Add conversion macros and factorize lists (patches 6-8)
> - Consolidate device/driver lookup and iteration (patches 9-11)
> - Refactor probe logic (patches 12-15)
> - Remove bus-specific types from drivers (patches 16-23)
>
> Note on ABI:
> This series breaks the ABI for drivers (changes to rte_pci_device,
> rte_pci_driver, and similar structures for other buses). However, the DPDK
> ABI policy does not provide guarantees for driver-level interfaces.
>
>
> --
> David Marchand
>
> Changes since v1:
> - fix typo in Windows code for net/mlx5,
>
In general a fan of cleanups in code, and this all seems reasonable.
With a fix added for idxd driver,
Series-Acked-by: Bruce Richardson <bruce.richardson@intel.com>
^ permalink raw reply
* [DPDK/ethdev Bug 1948] bnxt: bnxt_hwrm_send_message timeout
From: bugzilla @ 2026-05-21 8:03 UTC (permalink / raw)
To: dev
http://bugs.dpdk.org/show_bug.cgi?id=1948
Bug ID: 1948
Summary: bnxt: bnxt_hwrm_send_message timeout
Product: DPDK
Version: 26.03
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: ethdev
Assignee: dev@dpdk.org
Reporter: oleksandrn@interfacemasters.com
Target Milestone: ---
Randomly, on rte_eth_dev_configure, see this error, it seems like after that,
traffic doesn't work on this port anymore(or at least returned counters are 0)
Happens somewhat rarely, like 1 out of 10. The problem is that dev_configure
doesn't return failure, so not sure if there is a way to detect this
automatically; maybe rte_eth_link_get can be used.
I also see that there is a similar old Bug 1214
BNXT: bnxt_hwrm_send_message(): Error(timeout) sending msg 0x002a, seq_id 598
BNXT: bnxt_get_hwrm_link_config(): Get link config failed with rc -110
BNXT: bnxt_hwrm_send_message(): Error(timeout) sending msg 0x002a, seq_id 621
BNXT: bnxt_get_hwrm_link_config(): Get link config failed with rc -110
Part Number : BCM957608-P2100GQF20
Chip Number : BCM57608
Chip Name : THOR2
Description : Broadcom BCM57608 2x100G PCIe Ethernet
NIC
Firmware Name : PRIMATE_FW
Firmware Version : 237.1.148.0
RoCE Firmware Version : 237.1.148.0
HWRM Interface Spec : 1.10.3
Kong mailbox channel : Not Applicable
Active Package Version : 237.1.148.0
Package Version on NVM : 237.1.148.0
Active NVM config version : 236.0.2
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply
* Re: [PATCH v2 19/23] dma/idxd: remove specific bus type
From: David Marchand @ 2026-05-21 7:46 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev, thomas, stephen, Kevin Laatz
In-Reply-To: <ag3tWZn0f4jIfVWo@bricha3-mobl1.ger.corp.intel.com>
On Wed, 20 May 2026 at 19:20, Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Wed, May 06, 2026 at 05:51:51PM +0200, David Marchand wrote:
> > There is nothing that requires a specific bus type.
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
>
> This patch exposes an underlying issue with the DSA driver. When applied,
> attempting to probe the workqueues gives the output:
>
> "EAL: Device wq0.0 is already probed"
>
> For each device. This is because the device is not zeroed in the scan
> function, leaving a non-NULL driver pointer. See inline below.
Thanks for testing!
I think this issue surfaces with patch 15 "bus: implement probe in
EAL", where the dsa bus starts using the generic helper that checks
for rte_dev_is_probed.
But well, nothing is broken atm, no need for you to send a separate fix.
I'll add this fix in the v3.
--
David Marchand
^ permalink raw reply
* [PATCH] app/test-pmd: add generic PROG action parser support
From: Megha Ajmera @ 2026-05-21 5:56 UTC (permalink / raw)
To: bruce.richardson, cristian.dumitrescu, praveen.shetty, dev
Add parser support for a generic PROG flow action in testpmd.
The update adds CLI tokens and parsing logic for program name and
argument tuples (name, size, value), enabling programmable action
configuration through the flow command interface.
Example flow rule:
flow create 0 ingress pattern eth / end actions prog name my_prog
argument name arg0 size 4 value 10 / end
Signed-off-by: Megha Ajmera <megha.ajmera@intel.com>
Signed-off-by: Praveen Shetty <praveen.shetty@intel.com>
---
app/test-pmd/cmdline_flow.c | 584 +++++++++++++++++++++++++++++++++++-
1 file changed, 581 insertions(+), 3 deletions(-)
diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index ebc036b14b..b429ff7267 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -714,6 +714,13 @@ enum index {
ACTION_SET_META,
ACTION_SET_META_DATA,
ACTION_SET_META_MASK,
+ /* ACTION PROG */
+ ACTION_PROG,
+ ACTION_PROG_NAME,
+ ACTION_PROG_ARGUMENT,
+ ACTION_PROG_ARGUMENT_NAME,
+ ACTION_PROG_ARGUMENT_SIZE,
+ ACTION_PROG_ARGUMENT_VALUE,
ACTION_SET_IPV4_DSCP,
ACTION_SET_IPV4_DSCP_VALUE,
ACTION_SET_IPV6_DSCP,
@@ -981,6 +988,24 @@ struct action_sample_data {
struct rte_flow_action_sample conf;
uint32_t idx;
};
+
+#define ACTION_PROG_MAX_ARGS 32
+#define ACTION_PROG_NAME_LEN_MAX 64
+#define ACTION_PROG_ARG_NAME_LEN_MAX 16
+
+struct action_prog_argument_data {
+ char name[ACTION_PROG_ARG_NAME_LEN_MAX];
+ uint32_t length;
+ uint32_t size;
+ uint64_t value;
+};
+
+struct action_prog_data {
+ char name[ACTION_PROG_NAME_LEN_MAX];
+ uint32_t args_num;
+ uint32_t length;
+ struct action_prog_argument_data args[ACTION_PROG_MAX_ARGS];
+};
/** Storage for struct rte_flow_action_sample. */
struct raw_sample_conf {
struct rte_flow_action data[ACTION_SAMPLE_ACTIONS_NUM];
@@ -2310,6 +2335,7 @@ static const enum index next_action[] = {
ACTION_RAW_DECAP,
ACTION_SET_TAG,
ACTION_SET_META,
+ ACTION_PROG,
ACTION_SET_IPV4_DSCP,
ACTION_SET_IPV6_DSCP,
ACTION_AGE,
@@ -2565,6 +2591,23 @@ static const enum index action_set_meta[] = {
ZERO,
};
+/** ACTION PROG */
+static const enum index next_action_prog[] = {
+ ACTION_PROG_NAME,
+ ACTION_PROG_ARGUMENT,
+ ACTION_NEXT,
+ ZERO,
+};
+
+static const enum index next_prog_arg[] = {
+ ACTION_PROG_ARGUMENT_NAME,
+ ACTION_PROG_ARGUMENT_SIZE,
+ ACTION_PROG_ARGUMENT_VALUE,
+ ACTION_PROG_ARGUMENT,
+ ACTION_NEXT,
+ ZERO,
+};
+
static const enum index action_set_ipv4_dscp[] = {
ACTION_SET_IPV4_DSCP_VALUE,
ACTION_NEXT,
@@ -2803,6 +2846,27 @@ static int parse_vc_action_set_meta(struct context *ctx,
const struct token *token, const char *str,
unsigned int len, void *buf,
unsigned int size);
+static int parse_vc_action_prog(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size);
+static int parse_vc_action_prog_argument(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static int parse_vc_action_prog_argument_name(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static int parse_vc_action_prog_argument_size(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static int parse_vc_action_prog_argument_value(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size);
+static void free_action_prog_converted(struct rte_flow_action *actions,
+ const uint32_t *converted_idx,
+ uint32_t converted_idx_num);
static int parse_vc_action_sample(struct context *ctx,
const struct token *token, const char *str,
unsigned int len, void *buf,
@@ -7816,6 +7880,48 @@ static const struct token token_list[] = {
(struct rte_flow_action_set_meta, mask)),
.call = parse_vc_conf,
},
+ [ACTION_PROG] = {
+ .name = "prog",
+ .help = "Program action: action prog name <name> [argument ...]",
+ .priv = PRIV_ACTION(PROG,
+ sizeof(struct action_prog_data)),
+ .next = NEXT(next_action_prog),
+ .call = parse_vc_action_prog,
+ },
+ [ACTION_PROG_NAME] = {
+ .name = "name",
+ .help = "Action name",
+ .next = NEXT(next_action_prog, NEXT_ENTRY(COMMON_STRING)),
+ .args = ARGS(ARGS_ENTRY_ARB(0, 0),
+ ARGS_ENTRY(struct action_prog_data, length),
+ ARGS_ENTRY_ARB(0,
+ ACTION_PROG_NAME_LEN_MAX)),
+ .call = parse_vc_conf,
+ },
+ [ACTION_PROG_ARGUMENT] = {
+ .name = "argument",
+ .help = "Keyword: argument",
+ .next = NEXT(next_prog_arg),
+ .call = parse_vc_action_prog_argument,
+ },
+ [ACTION_PROG_ARGUMENT_NAME] = {
+ .name = "name",
+ .help = "Argument Name",
+ .next = NEXT(next_prog_arg, NEXT_ENTRY(COMMON_STRING)),
+ .call = parse_vc_action_prog_argument_name,
+ },
+ [ACTION_PROG_ARGUMENT_SIZE] = {
+ .name = "size",
+ .help = "Argument size (bytes)",
+ .next = NEXT(next_prog_arg, NEXT_ENTRY(COMMON_UNSIGNED)),
+ .call = parse_vc_action_prog_argument_size,
+ },
+ [ACTION_PROG_ARGUMENT_VALUE] = {
+ .name = "value",
+ .help = "Argument value",
+ .next = NEXT(next_prog_arg, NEXT_ENTRY(COMMON_UNSIGNED)),
+ .call = parse_vc_action_prog_argument_value,
+ },
[ACTION_SET_IPV4_DSCP] = {
.name = "set_ipv4_dscp",
.help = "set DSCP value",
@@ -10415,6 +10521,449 @@ parse_vc_action_set_meta(struct context *ctx, const struct token *token,
return len;
}
+/** Parse PROG action */
+static int
+parse_vc_action_prog(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ int ret;
+
+ ret = parse_vc(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ return ret;
+
+ if (!out->args.vc.actions_n)
+ return -1;
+
+ /* Point to selected object. */
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+
+ prog_data = ctx->object;
+ prog_data->args_num = 0;
+
+ return ret;
+}
+
+/** Called when args keyword is encountered */
+static int
+parse_vc_action_prog_argument(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(size);
+
+ /* Token name must match. */
+ if (parse_default(ctx, token, str, len, NULL, 0) < 0)
+ return -1;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ return len;
+
+ if (!out->args.vc.actions_n)
+ return len;
+
+ /* Point to selected object. */
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+
+ prog_data = ctx->object;
+ if (prog_data->args_num >= ACTION_PROG_MAX_ARGS)
+ return -1;
+ prog_data->args_num++;
+
+ return len;
+}
+
+static int __prog_argument_name_args_push(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size, uint32_t arg_ind)
+{
+ static struct arg arg_addr[ACTION_PROG_MAX_ARGS];
+ static struct arg arg_len[ACTION_PROG_MAX_ARGS];
+ static struct arg arg_data[ACTION_PROG_MAX_ARGS];
+ /* Calculate the base size based on the actual structure size */
+ uint32_t prog_data_base_size = offsetof(struct action_prog_data, args);
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(len);
+ RTE_SET_USED(buf);
+ RTE_SET_USED(size);
+
+ arg_addr[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, name) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg_addr[arg_ind].size = 0;
+
+ if (push_args(ctx, &arg_addr[arg_ind]))
+ return -1;
+ arg_len[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, length) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg_len[arg_ind].size = sizeof(((struct action_prog_argument_data *)0)->length);
+
+ if (push_args(ctx, &arg_len[arg_ind]))
+ return -1;
+ arg_data[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, name) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg_data[arg_ind].size =
+ sizeof(((struct action_prog_argument_data *)0)->name);
+
+ if (push_args(ctx, &arg_data[arg_ind]))
+ return -1;
+
+ return 0;
+}
+
+static int __prog_argument_size_args_push(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size, uint32_t arg_ind)
+{
+ static struct arg arg[ACTION_PROG_MAX_ARGS];
+ uint32_t prog_data_base_size = offsetof(struct action_prog_data, args);
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(len);
+ RTE_SET_USED(buf);
+ RTE_SET_USED(size);
+
+ arg[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, size) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg[arg_ind].size = sizeof(((struct action_prog_argument_data *)0)->size);
+
+ if (push_args(ctx, &arg[arg_ind]))
+ return -1;
+
+ return 0;
+}
+
+static int __prog_argument_value_args_push(struct context *ctx, const struct token *token,
+ const char *str, unsigned int len, void *buf,
+ unsigned int size, uint32_t arg_ind)
+{
+ static struct arg arg[ACTION_PROG_MAX_ARGS];
+ uint32_t prog_data_base_size = offsetof(struct action_prog_data, args);
+
+ RTE_SET_USED(token);
+ RTE_SET_USED(str);
+ RTE_SET_USED(len);
+ RTE_SET_USED(buf);
+ RTE_SET_USED(size);
+
+ arg[arg_ind].offset = prog_data_base_size +
+ offsetof(struct action_prog_argument_data, value) +
+ (arg_ind * sizeof(struct action_prog_argument_data));
+ arg[arg_ind].size = sizeof(((struct action_prog_argument_data *)0)->value);
+
+ if (push_args(ctx, &arg[arg_ind]))
+ return -1;
+
+ return 0;
+}
+
+static int
+parse_vc_action_prog_argument_name(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ uint32_t arg_ind;
+ uint32_t max_arg_ind = (ACTION_PROG_MAX_ARGS - 1);
+ int ret;
+
+ ret = parse_vc_conf(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ goto error_push;
+
+ if (!ctx->object)
+ goto error_push;
+
+ prog_data = ctx->object;
+ if (prog_data->args_num == 0)
+ goto error_push;
+ arg_ind = prog_data->args_num - 1;
+ if (__prog_argument_name_args_push(ctx, token, str, len, buf, size,
+ arg_ind) < 0)
+ return -1;
+
+ return ret;
+
+error_push:
+ return __prog_argument_name_args_push(ctx, token, str, len, buf, size,
+ max_arg_ind);
+}
+
+static int
+parse_vc_action_prog_argument_size(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ uint32_t arg_ind;
+ uint32_t max_arg_ind = (ACTION_PROG_MAX_ARGS - 1);
+ int ret;
+
+ ret = parse_vc_conf(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ goto error_push;
+
+ if (!ctx->object)
+ goto error_push;
+
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+ prog_data = ctx->object;
+ if (prog_data->args_num == 0)
+ goto error_push;
+ arg_ind = prog_data->args_num - 1;
+ if (__prog_argument_size_args_push(ctx, token, str, len, buf, size,
+ arg_ind) < 0)
+ return -1;
+
+ return ret;
+
+error_push:
+ return __prog_argument_size_args_push(ctx, token, str, len, buf, size,
+ max_arg_ind);
+}
+
+static int
+parse_vc_action_prog_argument_value(struct context *ctx,
+ const struct token *token,
+ const char *str, unsigned int len,
+ void *buf, unsigned int size)
+{
+ struct buffer *out = buf;
+ struct action_prog_data *prog_data;
+ uint32_t arg_ind;
+ uint32_t max_arg_ind = (ACTION_PROG_MAX_ARGS - 1);
+ int ret;
+
+ RTE_SET_USED(size);
+
+ ret = parse_vc_conf(ctx, token, str, len, buf, size);
+ if (ret < 0)
+ return ret;
+
+ /* Nothing else to do if there is no buffer. */
+ if (!out)
+ goto error_push;
+
+ if (!ctx->object)
+ goto error_push;
+
+ ctx->object = out->args.vc.data;
+ ctx->objmask = NULL;
+ prog_data = ctx->object;
+ if (prog_data->args_num == 0)
+ goto error_push;
+ arg_ind = prog_data->args_num - 1;
+ if (__prog_argument_value_args_push(ctx, token, str, len, buf, size,
+ arg_ind) < 0)
+ return -1;
+
+ return ret;
+
+error_push:
+ return __prog_argument_value_args_push(ctx, token, str, len, buf, size,
+ max_arg_ind);
+}
+
+/**
+ * Convert action_prog_data to rte_flow_action_prog.
+ * Converts PROG actions from action_prog_data format to rte_flow_action_prog format.
+ *
+ * @param actions The flow actions array
+ * @param action_count Optional count of actions. If 0, will count until END action
+ * @return 0 on success, negative value on failure
+ */
+static int
+convert_action_prog_to_rte_flow(struct rte_flow_action *actions,
+ uint32_t prog_action_count,
+ uint32_t *converted_idx,
+ uint32_t converted_idx_cap,
+ uint32_t *converted_idx_num)
+{
+ uint32_t i = 0;
+ uint32_t j;
+ uint32_t k;
+ const struct action_prog_data *prog_data;
+ struct rte_flow_action_prog *prog;
+ struct rte_flow_action_prog_argument *args;
+ uint8_t *value;
+ bool arg_error;
+ int ret = 0;
+
+ if (converted_idx_num)
+ *converted_idx_num = 0;
+
+ /* If action_count is 0, count the actions until END action */
+ if (prog_action_count == 0) {
+ while (actions[i].type != RTE_FLOW_ACTION_TYPE_END)
+ i++;
+ prog_action_count = i + 1; /* Include END action */
+ i = 0; /* Reset counter for the processing loop */
+ }
+
+ /* Process all actions */
+ for (i = 0; i < prog_action_count; i++) {
+ if (actions[i].type == RTE_FLOW_ACTION_TYPE_PROG) {
+ prog_data = (const struct action_prog_data *)actions[i].conf;
+ if (!prog_data) {
+ fprintf(stderr, "Prog action found but no data provided\n");
+ ret = -EINVAL;
+ continue;
+ }
+
+ prog = calloc(1, sizeof(struct rte_flow_action_prog));
+ if (!prog) {
+ fprintf(stderr, "Failed to allocate memory for prog action\n");
+ ret = -ENOMEM;
+ continue;
+ }
+
+ prog->name = strdup(prog_data->name);
+ if (!prog->name) {
+ fprintf(stderr, "Failed to allocate memory for prog name\n");
+ free(prog);
+ ret = -ENOMEM;
+ continue;
+ }
+
+ prog->args_num = prog_data->args_num;
+
+ if (prog->args_num > 0) {
+ args = calloc(prog->args_num,
+ sizeof(struct rte_flow_action_prog_argument));
+ if (!args) {
+ fprintf(stderr,
+ "Failed to allocate memory for prog arguments\n");
+ free((void *)(uintptr_t)prog->name);
+ free(prog);
+ ret = -ENOMEM;
+ continue;
+ }
+
+ arg_error = false;
+ for (j = 0; j < prog->args_num; j++) {
+ args[j].name = strdup(prog_data->args[j].name);
+ if (!args[j].name) {
+ fprintf(stderr,
+ "Failed to allocate memory for argument name\n");
+ ret = -ENOMEM;
+ arg_error = true;
+ break;
+ }
+
+ args[j].size = prog_data->args[j].size;
+ if (args[j].size == 0)
+ continue;
+ if (args[j].size > sizeof(prog_data->args[j].value)) {
+ free((void *)(uintptr_t)args[j].name);
+ arg_error = true;
+ ret = -EINVAL;
+ break;
+ }
+
+ value = malloc(args[j].size);
+ if (!value) {
+ fprintf(stderr,
+ "Failed to allocate memory for argument value\n");
+ free((void *)(uintptr_t)args[j].name);
+ ret = -ENOMEM;
+ arg_error = true;
+ break;
+ }
+
+ memcpy(value,
+ &prog_data->args[j].value,
+ args[j].size);
+ args[j].value = value;
+ }
+
+ if (arg_error) {
+ /* Free all allocated resources */
+ for (k = 0; k < j; k++) {
+ free((void *)(uintptr_t)args[k].name);
+ free((void *)(uintptr_t)args[k].value);
+ }
+ free(args);
+ free((void *)(uintptr_t)prog->name);
+ free(prog);
+ continue;
+ }
+
+ prog->args = args;
+ }
+
+ actions[i].conf = prog;
+ if (converted_idx && converted_idx_num &&
+ *converted_idx_num < converted_idx_cap)
+ converted_idx[(*converted_idx_num)++] = i;
+ }
+ }
+
+ return ret;
+}
+
+static void
+free_action_prog_converted(struct rte_flow_action *actions,
+ const uint32_t *converted_idx,
+ uint32_t converted_idx_num)
+{
+ uint32_t i;
+
+ for (i = 0; i < converted_idx_num; i++) {
+ uint32_t idx = converted_idx[i];
+ struct rte_flow_action_prog *prog;
+ uint32_t j;
+
+ if (actions[idx].type != RTE_FLOW_ACTION_TYPE_PROG)
+ continue;
+
+ prog = actions[idx].conf;
+ if (!prog)
+ continue;
+
+ for (j = 0; j < prog->args_num; j++) {
+ free((void *)(uintptr_t)prog->args[j].name);
+ free((void *)(uintptr_t)prog->args[j].value);
+ }
+ free(prog->args);
+ free((void *)(uintptr_t)prog->name);
+ free(prog);
+ }
+}
+
static int
parse_vc_action_sample(struct context *ctx, const struct token *token,
const char *str, unsigned int len, void *buf,
@@ -13380,6 +13929,8 @@ indirect_action_list_conf_get(uint32_t conf_id)
static void
cmd_flow_parsed(const struct buffer *in)
{
+ int ret;
+
switch (in->command) {
case INFO:
port_flow_get_info(in->port);
@@ -13561,9 +14112,36 @@ cmd_flow_parsed(const struct buffer *in)
&in->args.vc.tunnel_ops);
break;
case CREATE:
- port_flow_create(in->port, &in->args.vc.attr,
- in->args.vc.pattern, in->args.vc.actions,
- &in->args.vc.tunnel_ops, in->args.vc.user_id);
+ {
+ uint32_t *converted_idx;
+ uint32_t converted_idx_num = 0;
+
+ converted_idx = calloc(in->args.vc.actions_n,
+ sizeof(*converted_idx));
+ if (!converted_idx) {
+ fprintf(stderr,
+ "Warning: Failed to allocate conversion index list\n");
+ break;
+ }
+
+ /* Convert from action_prog_data to rte_flow_action_prog. */
+ ret = convert_action_prog_to_rte_flow(in->args.vc.actions,
+ 0,
+ converted_idx,
+ in->args.vc.actions_n,
+ &converted_idx_num);
+ if (ret < 0)
+ fprintf(stderr,
+ "Warning: Failed to convert program action data: %s\n",
+ strerror(-ret));
+ port_flow_create(in->port, &in->args.vc.attr,
+ in->args.vc.pattern, in->args.vc.actions,
+ &in->args.vc.tunnel_ops, in->args.vc.user_id);
+ free_action_prog_converted(in->args.vc.actions,
+ converted_idx,
+ converted_idx_num);
+ free(converted_idx);
+ }
break;
case DESTROY:
port_flow_destroy(in->port, in->args.destroy.rule_n,
--
2.34.1
^ permalink raw reply related
* Re: Memory management BoF summary (DPDK Summit 2026)
From: Mattias Rönnblom @ 2026-05-21 4:56 UTC (permalink / raw)
To: Morten Brørup, dev; +Cc: techboard, mattias.ronnblom
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35F6587B@smartserver.smartshare.dk>
On 5/18/26 10:35, Morten Brørup wrote:
> There are two ways of allocating/freeing memory objects in DPDK:
> 1. From the memory heap, i.e. rte_malloc() etc.
> 2. From a memory pool, i.e. rte_mempool_get() etc.
>
> The current memory heap in DPDK has two issues:
> - It is too slow to be used in the fast path.
> - There may be contention between control threads and
> EAL threads for the same spinlocks in the heap.
>
> The memory pools in the DPDK has multiple issues:
> - It is designed for fixed size objects,
> so individual mempools must be allocated for each type of object.
> (In theory, mempools could be allocated per object size,
> and shared across modules.
> But that would require coordination across modules, including
> the total number of objects required by these modules,
> the choice of underlying mempool driver,
> and the optimal mempool cache size.)
> - The number of objects in each mempool is fixed,
> so applications have to size their mempools to worst case usage.
> - The memory for the objects in a mempool is pre-allocated.
> In summary, mempools consume significantly more memory than effectively
> being used.
>
> We discussed the need for a new memory heap system,
> and converged on the following features:
> - Providing functions to allocate and free memory objects of varying
> size, like the rte_malloc library, but usable in the fast path.
> - Perhaps also a need for bulk alloc/free functions,
> like the rte_mempool library.
> - Building on top of memzones.
> - No dependency on the current DPDK heap, so it can cleanly replace it.
> This is a stretch requirement.
> - NUMA aware.
> - Using slabs for various block sizes like in the Linux Kernel,
> possibly with object size 2^N.
> - Using per-lcore caches to reduce contention,
> resembling the mempool per-lcore caches.
>
> Mattias Rönnblom has implemented a prototype with the above properties,
> and this was a good starting point for the discussion.
>
I'm working to get an OK from the appropriate internal stakeholders,
then I'll post an RFC.
Thanks a lot for the excellent summary.
> Discussion:
> - It does not seem to be a requirement to be able to free the memory
> used by the heap back to the memzones.
> - It is possible having header-less objects.
> However, if the headers are relatively small compared to the size of
> the objects themselves, having a header may offer some benefits.
> - The mempool library has implementation details optimizing for
> spreading usage across memory channels, cache alignment, etc..
> Such performance optimization details might need consideration.
> - There is a lower limit to how small objects will be allocated.
> E.g. objects smaller than the size of a pointer is unlikely.
> - It is acceptable to return an object larger than the size requested,
> e.g. returning an object of 16 bytes when 12 bytes were requested.
> - Preventing false sharing of allocated objects between CPU caches
> may not be necessary, but must be considered.
> - Optimally, the new heap should replace the existing heap.
> As seen with the timer wheel library previously proposed, replacing
> core libraries in DPDK is difficult, so adding the new heap library
> in parallel with the existing heap, i.e. with separate APIs, might be
> a path of less resistance.
> It will then be an application choice to use this new memory heap.
> And long term, it can replace the existing heap, if appropriate.
>
> Mbuf discussion:
> - As a stretch goal, mbufs should be dynamically allocated from
> the new heap, instead of being pre-allocated in mempools.
> - This might be achievable either
> - by replacing the mempool library with a wrapper to the heap, or
> - by providing a new mempool driver using the heap, as an
> alternative to the existing ring and stack mempool drivers.
Would be very interesting to see a prototype of a mempool driver for a
new small-object allocator.
> - In this context, it is important to note that mbuf objects have some
> pre-initialized fields when held in their respective mempools.
> If allocating an mbuf directly from the heap, these fields must
> somehow be initialized.
> We did not discuss solutions for this, but noted that using the new
> heap for mbufs should be considered in the design choices.
>
>
> Venlig hilsen / Kind regards,
> -Morten Brørup
^ permalink raw reply
* [RFC 7/7] config: use RTE_FORCE_INTRINSICS on all platforms
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev
Cc: Stephen Hemminger, Wathsala Vithanage, Bruce Richardson, Bibo Mao,
Sun Yuechi, David Christensen, Konstantin Ananyev
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
Next step is to deprecate the rte_atomicNN_*() family. Rather than
maintaining both the inline asm and intrinsic fallbacks, drop the
asm paths and use intrinsics everywhere. The RTE_FORCE_INTRINSICS
config option is removed.
This also retires the asm-based byteorder bswap and spinlock
implementations, which were guarded by the same option.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
config/arm/meson.build | 1 -
config/loongarch/meson.build | 1 -
config/meson.build | 3 -
config/riscv/meson.build | 1 -
doc/guides/rel_notes/release_26_07.rst | 5 +
lib/eal/arm/include/rte_atomic_32.h | 3 -
lib/eal/arm/include/rte_atomic_64.h | 3 -
lib/eal/arm/include/rte_byteorder.h | 3 -
lib/eal/arm/include/rte_spinlock.h | 3 -
lib/eal/include/generic/rte_atomic.h | 58 -------
lib/eal/include/generic/rte_byteorder.h | 2 -
lib/eal/include/generic/rte_spinlock.h | 10 --
lib/eal/loongarch/include/rte_atomic.h | 3 -
lib/eal/loongarch/include/rte_spinlock.h | 3 -
lib/eal/ppc/include/rte_atomic.h | 173 ---------------------
lib/eal/ppc/include/rte_byteorder.h | 13 --
lib/eal/ppc/include/rte_spinlock.h | 26 ----
lib/eal/riscv/include/rte_atomic.h | 3 -
lib/eal/riscv/include/rte_spinlock.h | 3 -
lib/eal/x86/include/rte_atomic.h | 172 ---------------------
lib/eal/x86/include/rte_atomic_32.h | 188 -----------------------
lib/eal/x86/include/rte_atomic_64.h | 157 -------------------
lib/eal/x86/include/rte_byteorder.h | 49 ------
lib/eal/x86/include/rte_spinlock.h | 49 ------
24 files changed, 5 insertions(+), 927 deletions(-)
diff --git a/config/arm/meson.build b/config/arm/meson.build
index 5a9c16b9b1..08fff73599 100644
--- a/config/arm/meson.build
+++ b/config/arm/meson.build
@@ -837,7 +837,6 @@ socs = {
}
dpdk_conf.set('RTE_ARCH_ARM', 1)
-dpdk_conf.set('RTE_FORCE_INTRINSICS', 1)
update_flags = false
soc_flags = []
diff --git a/config/loongarch/meson.build b/config/loongarch/meson.build
index 99dabef203..1623cdb571 100644
--- a/config/loongarch/meson.build
+++ b/config/loongarch/meson.build
@@ -6,7 +6,6 @@ if not dpdk_conf.get('RTE_ARCH_64')
endif
dpdk_conf.set('RTE_ARCH', 'loongarch')
dpdk_conf.set('RTE_ARCH_LOONGARCH', 1)
-dpdk_conf.set('RTE_FORCE_INTRINSICS', 1)
machine_args_generic = [
['default', ['-march=loongarch64']],
diff --git a/config/meson.build b/config/meson.build
index 9ba7b9a338..934abf04f2 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -29,9 +29,6 @@ is_ms_compiler = is_windows and (cc.get_id() == 'msvc')
is_ms_linker = is_windows and (cc.get_id() == 'clang' or is_ms_compiler)
if is_ms_compiler
- # force the use of intrinsics the MSVC compiler (except x86)
- # does not support inline assembly
- dpdk_conf.set('RTE_FORCE_INTRINSICS', 1)
# force the use of C++11 memory model in lib/ring
dpdk_conf.set('RTE_USE_C11_MEM_MODEL', true)
diff --git a/config/riscv/meson.build b/config/riscv/meson.build
index a06429a1e2..5dba613973 100644
--- a/config/riscv/meson.build
+++ b/config/riscv/meson.build
@@ -16,7 +16,6 @@ endif
dpdk_conf.set('RTE_ARCH', 'riscv')
dpdk_conf.set('RTE_ARCH_RISCV', 1)
-dpdk_conf.set('RTE_FORCE_INTRINSICS', 1)
# common flags to all riscv builds, with lowest priority
flags_common = [
diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
index f012d47a4b..9378cf1d36 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -92,6 +92,11 @@ API Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* **Changed to use stdatomic intrinsics on all platforms.**
+
+ The config option ``RTE_FORCE_INTRINSICS`` has been removed.
+ Architecture specific code has been replaced with stdatomic.
+
ABI Changes
-----------
diff --git a/lib/eal/arm/include/rte_atomic_32.h b/lib/eal/arm/include/rte_atomic_32.h
index 3809ddefb7..a5ee63a2c7 100644
--- a/lib/eal/arm/include/rte_atomic_32.h
+++ b/lib/eal/arm/include/rte_atomic_32.h
@@ -5,9 +5,6 @@
#ifndef _RTE_ATOMIC_ARM32_H_
#define _RTE_ATOMIC_ARM32_H_
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
#include "generic/rte_atomic.h"
diff --git a/lib/eal/arm/include/rte_atomic_64.h b/lib/eal/arm/include/rte_atomic_64.h
index c9b41f6212..01412020e7 100644
--- a/lib/eal/arm/include/rte_atomic_64.h
+++ b/lib/eal/arm/include/rte_atomic_64.h
@@ -6,9 +6,6 @@
#ifndef _RTE_ATOMIC_ARM64_H_
#define _RTE_ATOMIC_ARM64_H_
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
#include "generic/rte_atomic.h"
#include <rte_branch_prediction.h>
diff --git a/lib/eal/arm/include/rte_byteorder.h b/lib/eal/arm/include/rte_byteorder.h
index a0aaff4a28..ffaaf726a4 100644
--- a/lib/eal/arm/include/rte_byteorder.h
+++ b/lib/eal/arm/include/rte_byteorder.h
@@ -5,9 +5,6 @@
#ifndef _RTE_BYTEORDER_ARM_H_
#define _RTE_BYTEORDER_ARM_H_
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
#include <stdint.h>
#include <rte_common.h>
diff --git a/lib/eal/arm/include/rte_spinlock.h b/lib/eal/arm/include/rte_spinlock.h
index a5d01b0d21..47820e5e1a 100644
--- a/lib/eal/arm/include/rte_spinlock.h
+++ b/lib/eal/arm/include/rte_spinlock.h
@@ -5,9 +5,6 @@
#ifndef _RTE_SPINLOCK_ARM_H_
#define _RTE_SPINLOCK_ARM_H_
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
#include <rte_common.h>
#include "generic/rte_spinlock.h"
diff --git a/lib/eal/include/generic/rte_atomic.h b/lib/eal/include/generic/rte_atomic.h
index 4e9d230f85..06b6acf9eb 100644
--- a/lib/eal/include/generic/rte_atomic.h
+++ b/lib/eal/include/generic/rte_atomic.h
@@ -171,13 +171,11 @@ rte_smp_rmb(void)
static inline int
rte_atomic16_cmpset(volatile uint16_t *dst, uint16_t exp, uint16_t src);
-#ifdef RTE_FORCE_INTRINSICS
static inline int
rte_atomic16_cmpset(volatile uint16_t *dst, uint16_t exp, uint16_t src)
{
return __sync_bool_compare_and_swap(dst, exp, src);
}
-#endif
/**
* Atomic exchange.
@@ -197,13 +195,11 @@ rte_atomic16_cmpset(volatile uint16_t *dst, uint16_t exp, uint16_t src)
static inline uint16_t
rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val);
-#ifdef RTE_FORCE_INTRINSICS
static inline uint16_t
rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val)
{
return rte_atomic_exchange_explicit(dst, val, rte_memory_order_seq_cst);
}
-#endif
/**
* The atomic counter structure.
@@ -296,13 +292,11 @@ rte_atomic16_sub(rte_atomic16_t *v, int16_t dec)
static inline void
rte_atomic16_inc(rte_atomic16_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic16_inc(rte_atomic16_t *v)
{
rte_atomic16_add(v, 1);
}
-#endif
/**
* Atomically decrement a counter by one.
@@ -313,13 +307,11 @@ rte_atomic16_inc(rte_atomic16_t *v)
static inline void
rte_atomic16_dec(rte_atomic16_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic16_dec(rte_atomic16_t *v)
{
rte_atomic16_sub(v, 1);
}
-#endif
/**
* Atomically add a 16-bit value to a counter and return the result.
@@ -375,13 +367,11 @@ rte_atomic16_sub_return(rte_atomic16_t *v, int16_t dec)
*/
static inline int rte_atomic16_inc_and_test(rte_atomic16_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic16_inc_and_test(rte_atomic16_t *v)
{
return rte_atomic_fetch_add_explicit((volatile __rte_atomic int16_t *)&v->cnt, 1,
rte_memory_order_seq_cst) + 1 == 0;
}
-#endif
/**
* Atomically decrement a 16-bit counter by one and test.
@@ -396,13 +386,11 @@ static inline int rte_atomic16_inc_and_test(rte_atomic16_t *v)
*/
static inline int rte_atomic16_dec_and_test(rte_atomic16_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic16_dec_and_test(rte_atomic16_t *v)
{
return rte_atomic_fetch_sub_explicit((volatile __rte_atomic int16_t *)&v->cnt, 1,
rte_memory_order_seq_cst) - 1 == 0;
}
-#endif
/**
* Atomically test and set a 16-bit atomic counter.
@@ -417,12 +405,10 @@ static inline int rte_atomic16_dec_and_test(rte_atomic16_t *v)
*/
static inline int rte_atomic16_test_and_set(rte_atomic16_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic16_test_and_set(rte_atomic16_t *v)
{
return rte_atomic16_cmpset((volatile uint16_t *)&v->cnt, 0, 1);
}
-#endif
/**
* Atomically set a 16-bit counter to 0.
@@ -456,13 +442,11 @@ static inline void rte_atomic16_clear(rte_atomic16_t *v)
static inline int
rte_atomic32_cmpset(volatile uint32_t *dst, uint32_t exp, uint32_t src);
-#ifdef RTE_FORCE_INTRINSICS
static inline int
rte_atomic32_cmpset(volatile uint32_t *dst, uint32_t exp, uint32_t src)
{
return __sync_bool_compare_and_swap(dst, exp, src);
}
-#endif
/**
* Atomic exchange.
@@ -482,13 +466,11 @@ rte_atomic32_cmpset(volatile uint32_t *dst, uint32_t exp, uint32_t src)
static inline uint32_t
rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val);
-#ifdef RTE_FORCE_INTRINSICS
static inline uint32_t
rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val)
{
return rte_atomic_exchange_explicit(dst, val, rte_memory_order_seq_cst);
}
-#endif
/**
* The atomic counter structure.
@@ -581,13 +563,11 @@ rte_atomic32_sub(rte_atomic32_t *v, int32_t dec)
static inline void
rte_atomic32_inc(rte_atomic32_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic32_inc(rte_atomic32_t *v)
{
rte_atomic32_add(v, 1);
}
-#endif
/**
* Atomically decrement a counter by one.
@@ -598,13 +578,11 @@ rte_atomic32_inc(rte_atomic32_t *v)
static inline void
rte_atomic32_dec(rte_atomic32_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic32_dec(rte_atomic32_t *v)
{
rte_atomic32_sub(v,1);
}
-#endif
/**
* Atomically add a 32-bit value to a counter and return the result.
@@ -660,13 +638,11 @@ rte_atomic32_sub_return(rte_atomic32_t *v, int32_t dec)
*/
static inline int rte_atomic32_inc_and_test(rte_atomic32_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic32_inc_and_test(rte_atomic32_t *v)
{
return rte_atomic_fetch_add_explicit((volatile __rte_atomic int32_t *)&v->cnt, 1,
rte_memory_order_seq_cst) + 1 == 0;
}
-#endif
/**
* Atomically decrement a 32-bit counter by one and test.
@@ -681,13 +657,11 @@ static inline int rte_atomic32_inc_and_test(rte_atomic32_t *v)
*/
static inline int rte_atomic32_dec_and_test(rte_atomic32_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic32_dec_and_test(rte_atomic32_t *v)
{
return rte_atomic_fetch_sub_explicit((volatile __rte_atomic int32_t *)&v->cnt, 1,
rte_memory_order_seq_cst) - 1 == 0;
}
-#endif
/**
* Atomically test and set a 32-bit atomic counter.
@@ -702,12 +676,10 @@ static inline int rte_atomic32_dec_and_test(rte_atomic32_t *v)
*/
static inline int rte_atomic32_test_and_set(rte_atomic32_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic32_test_and_set(rte_atomic32_t *v)
{
return rte_atomic32_cmpset((volatile uint32_t *)&v->cnt, 0, 1);
}
-#endif
/**
* Atomically set a 32-bit counter to 0.
@@ -740,13 +712,11 @@ static inline void rte_atomic32_clear(rte_atomic32_t *v)
static inline int
rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src);
-#ifdef RTE_FORCE_INTRINSICS
static inline int
rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src)
{
return __sync_bool_compare_and_swap(dst, exp, src);
}
-#endif
/**
* Atomic exchange.
@@ -766,13 +736,11 @@ rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src)
static inline uint64_t
rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val);
-#ifdef RTE_FORCE_INTRINSICS
static inline uint64_t
rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val)
{
return rte_atomic_exchange_explicit(dst, val, rte_memory_order_seq_cst);
}
-#endif
/**
* The atomic counter structure.
@@ -795,7 +763,6 @@ typedef struct {
static inline void
rte_atomic64_init(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic64_init(rte_atomic64_t *v)
{
@@ -812,7 +779,6 @@ rte_atomic64_init(rte_atomic64_t *v)
}
#endif
}
-#endif
/**
* Atomically read a 64-bit counter.
@@ -825,7 +791,6 @@ rte_atomic64_init(rte_atomic64_t *v)
static inline int64_t
rte_atomic64_read(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int64_t
rte_atomic64_read(rte_atomic64_t *v)
{
@@ -844,7 +809,6 @@ rte_atomic64_read(rte_atomic64_t *v)
return tmp;
#endif
}
-#endif
/**
* Atomically set a 64-bit counter.
@@ -857,7 +821,6 @@ rte_atomic64_read(rte_atomic64_t *v)
static inline void
rte_atomic64_set(rte_atomic64_t *v, int64_t new_value);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic64_set(rte_atomic64_t *v, int64_t new_value)
{
@@ -874,7 +837,6 @@ rte_atomic64_set(rte_atomic64_t *v, int64_t new_value)
}
#endif
}
-#endif
/**
* Atomically add a 64-bit value to a counter.
@@ -887,14 +849,12 @@ rte_atomic64_set(rte_atomic64_t *v, int64_t new_value)
static inline void
rte_atomic64_add(rte_atomic64_t *v, int64_t inc);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic64_add(rte_atomic64_t *v, int64_t inc)
{
rte_atomic_fetch_add_explicit((volatile __rte_atomic int64_t *)&v->cnt, inc,
rte_memory_order_seq_cst);
}
-#endif
/**
* Atomically subtract a 64-bit value from a counter.
@@ -907,14 +867,12 @@ rte_atomic64_add(rte_atomic64_t *v, int64_t inc)
static inline void
rte_atomic64_sub(rte_atomic64_t *v, int64_t dec);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic64_sub(rte_atomic64_t *v, int64_t dec)
{
rte_atomic_fetch_sub_explicit((volatile __rte_atomic int64_t *)&v->cnt, dec,
rte_memory_order_seq_cst);
}
-#endif
/**
* Atomically increment a 64-bit counter by one and test.
@@ -925,13 +883,11 @@ rte_atomic64_sub(rte_atomic64_t *v, int64_t dec)
static inline void
rte_atomic64_inc(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic64_inc(rte_atomic64_t *v)
{
rte_atomic64_add(v, 1);
}
-#endif
/**
* Atomically decrement a 64-bit counter by one and test.
@@ -942,13 +898,11 @@ rte_atomic64_inc(rte_atomic64_t *v)
static inline void
rte_atomic64_dec(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_atomic64_dec(rte_atomic64_t *v)
{
rte_atomic64_sub(v, 1);
}
-#endif
/**
* Add a 64-bit value to an atomic counter and return the result.
@@ -966,14 +920,12 @@ rte_atomic64_dec(rte_atomic64_t *v)
static inline int64_t
rte_atomic64_add_return(rte_atomic64_t *v, int64_t inc);
-#ifdef RTE_FORCE_INTRINSICS
static inline int64_t
rte_atomic64_add_return(rte_atomic64_t *v, int64_t inc)
{
return rte_atomic_fetch_add_explicit((volatile __rte_atomic int64_t *)&v->cnt, inc,
rte_memory_order_seq_cst) + inc;
}
-#endif
/**
* Subtract a 64-bit value from an atomic counter and return the result.
@@ -991,14 +943,12 @@ rte_atomic64_add_return(rte_atomic64_t *v, int64_t inc)
static inline int64_t
rte_atomic64_sub_return(rte_atomic64_t *v, int64_t dec);
-#ifdef RTE_FORCE_INTRINSICS
static inline int64_t
rte_atomic64_sub_return(rte_atomic64_t *v, int64_t dec)
{
return rte_atomic_fetch_sub_explicit((volatile __rte_atomic int64_t *)&v->cnt, dec,
rte_memory_order_seq_cst) - dec;
}
-#endif
/**
* Atomically increment a 64-bit counter by one and test.
@@ -1013,12 +963,10 @@ rte_atomic64_sub_return(rte_atomic64_t *v, int64_t dec)
*/
static inline int rte_atomic64_inc_and_test(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic64_inc_and_test(rte_atomic64_t *v)
{
return rte_atomic64_add_return(v, 1) == 0;
}
-#endif
/**
* Atomically decrement a 64-bit counter by one and test.
@@ -1033,12 +981,10 @@ static inline int rte_atomic64_inc_and_test(rte_atomic64_t *v)
*/
static inline int rte_atomic64_dec_and_test(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic64_dec_and_test(rte_atomic64_t *v)
{
return rte_atomic64_sub_return(v, 1) == 0;
}
-#endif
/**
* Atomically test and set a 64-bit atomic counter.
@@ -1053,12 +999,10 @@ static inline int rte_atomic64_dec_and_test(rte_atomic64_t *v)
*/
static inline int rte_atomic64_test_and_set(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline int rte_atomic64_test_and_set(rte_atomic64_t *v)
{
return rte_atomic64_cmpset((volatile uint64_t *)&v->cnt, 0, 1);
}
-#endif
/**
* Atomically set a 64-bit counter to 0.
@@ -1068,12 +1012,10 @@ static inline int rte_atomic64_test_and_set(rte_atomic64_t *v)
*/
static inline void rte_atomic64_clear(rte_atomic64_t *v);
-#ifdef RTE_FORCE_INTRINSICS
static inline void rte_atomic64_clear(rte_atomic64_t *v)
{
rte_atomic64_set(v, 0);
}
-#endif
#endif
diff --git a/lib/eal/include/generic/rte_byteorder.h b/lib/eal/include/generic/rte_byteorder.h
index 7973d6326f..e8b5f0ab86 100644
--- a/lib/eal/include/generic/rte_byteorder.h
+++ b/lib/eal/include/generic/rte_byteorder.h
@@ -239,7 +239,6 @@ static uint64_t rte_be_to_cpu_64(rte_be64_t x);
#endif /* __DOXYGEN__ */
-#ifdef RTE_FORCE_INTRINSICS
#ifndef RTE_TOOLCHAIN_MSVC
#define rte_bswap16(x) __builtin_bswap16(x)
@@ -253,7 +252,6 @@ static uint64_t rte_be_to_cpu_64(rte_be64_t x);
#define rte_bswap64(x) _byteswap_uint64(x)
#endif
-#endif
#ifdef __cplusplus
}
diff --git a/lib/eal/include/generic/rte_spinlock.h b/lib/eal/include/generic/rte_spinlock.h
index dd3d2d046c..13916f88b3 100644
--- a/lib/eal/include/generic/rte_spinlock.h
+++ b/lib/eal/include/generic/rte_spinlock.h
@@ -13,8 +13,6 @@
* This kind of lock simply waits in a loop
* repeatedly checking until the lock becomes available.
*
- * Some functions may have an architecture-specific implementation
- * if RTE_FORCE_INTRINSICS is disabled.
* The hardware transactional memory (lock elision) functions have _tm suffix
* and are implemented in architecture-specific files.
*
@@ -22,9 +20,7 @@
*/
#include <rte_lcore.h>
-#ifdef RTE_FORCE_INTRINSICS
#include <rte_common.h>
-#endif
#include <rte_debug.h>
#include <rte_lock_annotations.h>
#include <rte_pause.h>
@@ -68,7 +64,6 @@ static inline void
rte_spinlock_lock(rte_spinlock_t *sl)
__rte_acquire_capability(sl);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_spinlock_lock(rte_spinlock_t *sl)
__rte_no_thread_safety_analysis
@@ -82,7 +77,6 @@ rte_spinlock_lock(rte_spinlock_t *sl)
exp = 0;
}
}
-#endif
/**
* Release the spinlock.
@@ -94,14 +88,12 @@ static inline void
rte_spinlock_unlock(rte_spinlock_t *sl)
__rte_release_capability(sl);
-#ifdef RTE_FORCE_INTRINSICS
static inline void
rte_spinlock_unlock(rte_spinlock_t *sl)
__rte_no_thread_safety_analysis
{
rte_atomic_store_explicit(&sl->locked, 0, rte_memory_order_release);
}
-#endif
/**
* Try to take the lock.
@@ -116,7 +108,6 @@ static inline int
rte_spinlock_trylock(rte_spinlock_t *sl)
__rte_try_acquire_capability(true, sl);
-#ifdef RTE_FORCE_INTRINSICS
static inline int
rte_spinlock_trylock(rte_spinlock_t *sl)
__rte_no_thread_safety_analysis
@@ -125,7 +116,6 @@ rte_spinlock_trylock(rte_spinlock_t *sl)
return rte_atomic_compare_exchange_strong_explicit(&sl->locked, &exp, 1,
rte_memory_order_acquire, rte_memory_order_relaxed);
}
-#endif
/**
* Test if the lock is taken.
diff --git a/lib/eal/loongarch/include/rte_atomic.h b/lib/eal/loongarch/include/rte_atomic.h
index 49e0c67020..ed42e36843 100644
--- a/lib/eal/loongarch/include/rte_atomic.h
+++ b/lib/eal/loongarch/include/rte_atomic.h
@@ -5,9 +5,6 @@
#ifndef RTE_ATOMIC_LOONGARCH_H
#define RTE_ATOMIC_LOONGARCH_H
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
#include <rte_common.h>
#include "generic/rte_atomic.h"
diff --git a/lib/eal/loongarch/include/rte_spinlock.h b/lib/eal/loongarch/include/rte_spinlock.h
index 38f00f631d..bc9569b8e3 100644
--- a/lib/eal/loongarch/include/rte_spinlock.h
+++ b/lib/eal/loongarch/include/rte_spinlock.h
@@ -12,9 +12,6 @@
extern "C" {
#endif
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
static inline int rte_tm_supported(void)
{
diff --git a/lib/eal/ppc/include/rte_atomic.h b/lib/eal/ppc/include/rte_atomic.h
index 1da5afccbf..0e64db2a35 100644
--- a/lib/eal/ppc/include/rte_atomic.h
+++ b/lib/eal/ppc/include/rte_atomic.h
@@ -37,179 +37,6 @@ rte_atomic_thread_fence(rte_memory_order memorder)
}
/*------------------------- 16 bit atomic operations -------------------------*/
-#ifndef RTE_FORCE_INTRINSICS
-static inline int
-rte_atomic16_cmpset(volatile uint16_t *dst, uint16_t exp, uint16_t src)
-{
- return rte_atomic_compare_exchange_strong_explicit(dst, &exp, src, rte_memory_order_acquire,
- rte_memory_order_acquire) ? 1 : 0;
-}
-
-static inline int rte_atomic16_test_and_set(rte_atomic16_t *v)
-{
- return rte_atomic16_cmpset((volatile uint16_t *)&v->cnt, 0, 1);
-}
-
-static inline void
-rte_atomic16_inc(rte_atomic16_t *v)
-{
- rte_atomic_fetch_add_explicit(&v->cnt, 1, rte_memory_order_acquire);
-}
-
-static inline void
-rte_atomic16_dec(rte_atomic16_t *v)
-{
- rte_atomic_fetch_sub_explicit(&v->cnt, 1, rte_memory_order_acquire);
-}
-
-static inline int rte_atomic16_inc_and_test(rte_atomic16_t *v)
-{
- return rte_atomic_fetch_add_explicit(&v->cnt, 1, rte_memory_order_acquire) + 1 == 0;
-}
-
-static inline int rte_atomic16_dec_and_test(rte_atomic16_t *v)
-{
- return rte_atomic_fetch_sub_explicit(&v->cnt, 1, rte_memory_order_acquire) - 1 == 0;
-}
-
-static inline uint16_t
-rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val)
-{
- return __atomic_exchange_2(dst, val, rte_memory_order_seq_cst);
-}
-
-/*------------------------- 32 bit atomic operations -------------------------*/
-
-static inline int
-rte_atomic32_cmpset(volatile uint32_t *dst, uint32_t exp, uint32_t src)
-{
- return rte_atomic_compare_exchange_strong_explicit(dst, &exp, src, rte_memory_order_acquire,
- rte_memory_order_acquire) ? 1 : 0;
-}
-
-static inline int rte_atomic32_test_and_set(rte_atomic32_t *v)
-{
- return rte_atomic32_cmpset((volatile uint32_t *)&v->cnt, 0, 1);
-}
-
-static inline void
-rte_atomic32_inc(rte_atomic32_t *v)
-{
- rte_atomic_fetch_add_explicit(&v->cnt, 1, rte_memory_order_acquire);
-}
-
-static inline void
-rte_atomic32_dec(rte_atomic32_t *v)
-{
- rte_atomic_fetch_sub_explicit(&v->cnt, 1, rte_memory_order_acquire);
-}
-
-static inline int rte_atomic32_inc_and_test(rte_atomic32_t *v)
-{
- return rte_atomic_fetch_add_explicit(&v->cnt, 1, rte_memory_order_acquire) + 1 == 0;
-}
-
-static inline int rte_atomic32_dec_and_test(rte_atomic32_t *v)
-{
- return rte_atomic_fetch_sub_explicit(&v->cnt, 1, rte_memory_order_acquire) - 1 == 0;
-}
-
-static inline uint32_t
-rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val)
-{
- return __atomic_exchange_4(dst, val, rte_memory_order_seq_cst);
-}
-
-/*------------------------- 64 bit atomic operations -------------------------*/
-
-static inline int
-rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src)
-{
- return rte_atomic_compare_exchange_strong_explicit(dst, &exp, src, rte_memory_order_acquire,
- rte_memory_order_acquire) ? 1 : 0;
-}
-
-static inline void
-rte_atomic64_init(rte_atomic64_t *v)
-{
- v->cnt = 0;
-}
-
-static inline int64_t
-rte_atomic64_read(rte_atomic64_t *v)
-{
- return v->cnt;
-}
-
-static inline void
-rte_atomic64_set(rte_atomic64_t *v, int64_t new_value)
-{
- v->cnt = new_value;
-}
-
-static inline void
-rte_atomic64_add(rte_atomic64_t *v, int64_t inc)
-{
- rte_atomic_fetch_add_explicit(&v->cnt, inc, rte_memory_order_acquire);
-}
-
-static inline void
-rte_atomic64_sub(rte_atomic64_t *v, int64_t dec)
-{
- rte_atomic_fetch_sub_explicit(&v->cnt, dec, rte_memory_order_acquire);
-}
-
-static inline void
-rte_atomic64_inc(rte_atomic64_t *v)
-{
- rte_atomic_fetch_add_explicit(&v->cnt, 1, rte_memory_order_acquire);
-}
-
-static inline void
-rte_atomic64_dec(rte_atomic64_t *v)
-{
- rte_atomic_fetch_sub_explicit(&v->cnt, 1, rte_memory_order_acquire);
-}
-
-static inline int64_t
-rte_atomic64_add_return(rte_atomic64_t *v, int64_t inc)
-{
- return rte_atomic_fetch_add_explicit(&v->cnt, inc, rte_memory_order_acquire) + inc;
-}
-
-static inline int64_t
-rte_atomic64_sub_return(rte_atomic64_t *v, int64_t dec)
-{
- return rte_atomic_fetch_sub_explicit(&v->cnt, dec, rte_memory_order_acquire) - dec;
-}
-
-static inline int rte_atomic64_inc_and_test(rte_atomic64_t *v)
-{
- return rte_atomic_fetch_add_explicit(&v->cnt, 1, rte_memory_order_acquire) + 1 == 0;
-}
-
-static inline int rte_atomic64_dec_and_test(rte_atomic64_t *v)
-{
- return rte_atomic_fetch_sub_explicit(&v->cnt, 1, rte_memory_order_acquire) - 1 == 0;
-}
-
-static inline int rte_atomic64_test_and_set(rte_atomic64_t *v)
-{
- return rte_atomic64_cmpset((volatile uint64_t *)&v->cnt, 0, 1);
-}
-
-static inline void rte_atomic64_clear(rte_atomic64_t *v)
-{
- v->cnt = 0;
-}
-
-static inline uint64_t
-rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val)
-{
- return __atomic_exchange_8(dst, val, rte_memory_order_seq_cst);
-}
-
-#endif
#ifdef __cplusplus
}
diff --git a/lib/eal/ppc/include/rte_byteorder.h b/lib/eal/ppc/include/rte_byteorder.h
index 6c11fce9dc..e1e74f83e8 100644
--- a/lib/eal/ppc/include/rte_byteorder.h
+++ b/lib/eal/ppc/include/rte_byteorder.h
@@ -49,19 +49,6 @@ static inline uint64_t rte_arch_bswap64(uint64_t _x)
((_x << 40) & (0xffULL << 48)) | ((_x << 56));
}
-#ifndef RTE_FORCE_INTRINSICS
-#define rte_bswap16(x) ((uint16_t)(__builtin_constant_p(x) ? \
- rte_constant_bswap16(x) : \
- rte_arch_bswap16(x)))
-
-#define rte_bswap32(x) ((uint32_t)(__builtin_constant_p(x) ? \
- rte_constant_bswap32(x) : \
- rte_arch_bswap32(x)))
-
-#define rte_bswap64(x) ((uint64_t)(__builtin_constant_p(x) ? \
- rte_constant_bswap64(x) : \
- rte_arch_bswap64(x)))
-#endif
/* Power 8 have both little endian and big endian mode
* Power 7 only support big endian
diff --git a/lib/eal/ppc/include/rte_spinlock.h b/lib/eal/ppc/include/rte_spinlock.h
index 6d242db35d..76afa52413 100644
--- a/lib/eal/ppc/include/rte_spinlock.h
+++ b/lib/eal/ppc/include/rte_spinlock.h
@@ -15,32 +15,6 @@ extern "C" {
/* Fixme: Use intrinsics to implement the spinlock on Power architecture */
-#ifndef RTE_FORCE_INTRINSICS
-
-static inline void
-rte_spinlock_lock(rte_spinlock_t *sl)
- __rte_no_thread_safety_analysis
-{
- while (__sync_lock_test_and_set(&sl->locked, 1))
- while (sl->locked)
- rte_pause();
-}
-
-static inline void
-rte_spinlock_unlock(rte_spinlock_t *sl)
- __rte_no_thread_safety_analysis
-{
- __sync_lock_release(&sl->locked);
-}
-
-static inline int
-rte_spinlock_trylock(rte_spinlock_t *sl)
- __rte_no_thread_safety_analysis
-{
- return __sync_lock_test_and_set(&sl->locked, 1) == 0;
-}
-
-#endif
static inline int rte_tm_supported(void)
{
diff --git a/lib/eal/riscv/include/rte_atomic.h b/lib/eal/riscv/include/rte_atomic.h
index dd10ad5127..bc7d446df5 100644
--- a/lib/eal/riscv/include/rte_atomic.h
+++ b/lib/eal/riscv/include/rte_atomic.h
@@ -8,9 +8,6 @@
#ifndef RTE_ATOMIC_RISCV_H
#define RTE_ATOMIC_RISCV_H
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
#include <stdint.h>
#include <rte_common.h>
diff --git a/lib/eal/riscv/include/rte_spinlock.h b/lib/eal/riscv/include/rte_spinlock.h
index 5fe4980e44..5df97ac5ca 100644
--- a/lib/eal/riscv/include/rte_spinlock.h
+++ b/lib/eal/riscv/include/rte_spinlock.h
@@ -8,9 +8,6 @@
#ifndef RTE_SPINLOCK_RISCV_H
#define RTE_SPINLOCK_RISCV_H
-#ifndef RTE_FORCE_INTRINSICS
-# error Platform must be built with RTE_FORCE_INTRINSICS
-#endif
#include <rte_common.h>
#include "generic/rte_spinlock.h"
diff --git a/lib/eal/x86/include/rte_atomic.h b/lib/eal/x86/include/rte_atomic.h
index a850b0257c..f4d39ce4fe 100644
--- a/lib/eal/x86/include/rte_atomic.h
+++ b/lib/eal/x86/include/rte_atomic.h
@@ -102,178 +102,6 @@ rte_atomic_thread_fence(rte_memory_order memorder)
extern "C" {
#endif
-#ifndef RTE_FORCE_INTRINSICS
-static inline int
-rte_atomic16_cmpset(volatile uint16_t *dst, uint16_t exp, uint16_t src)
-{
- uint8_t res;
-
- asm volatile(
- MPLOCKED
- "cmpxchgw %[src], %[dst];"
- "sete %[res];"
- : [res] "=a" (res), /* output */
- [dst] "=m" (*dst)
- : [src] "r" (src), /* input */
- "a" (exp),
- "m" (*dst)
- : "memory"); /* no-clobber list */
- return res;
-}
-
-static inline uint16_t
-rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val)
-{
- asm volatile(
- MPLOCKED
- "xchgw %0, %1;"
- : "=r" (val), "=m" (*dst)
- : "0" (val), "m" (*dst)
- : "memory"); /* no-clobber list */
- return val;
-}
-
-static inline int rte_atomic16_test_and_set(rte_atomic16_t *v)
-{
- return rte_atomic16_cmpset((volatile uint16_t *)&v->cnt, 0, 1);
-}
-
-static inline void
-rte_atomic16_inc(rte_atomic16_t *v)
-{
- asm volatile(
- MPLOCKED
- "incw %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : "m" (v->cnt) /* input */
- );
-}
-
-static inline void
-rte_atomic16_dec(rte_atomic16_t *v)
-{
- asm volatile(
- MPLOCKED
- "decw %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : "m" (v->cnt) /* input */
- );
-}
-
-static inline int rte_atomic16_inc_and_test(rte_atomic16_t *v)
-{
- uint8_t ret;
-
- asm volatile(
- MPLOCKED
- "incw %[cnt] ; "
- "sete %[ret]"
- : [cnt] "+m" (v->cnt), /* output */
- [ret] "=qm" (ret)
- );
- return ret != 0;
-}
-
-static inline int rte_atomic16_dec_and_test(rte_atomic16_t *v)
-{
- uint8_t ret;
-
- asm volatile(MPLOCKED
- "decw %[cnt] ; "
- "sete %[ret]"
- : [cnt] "+m" (v->cnt), /* output */
- [ret] "=qm" (ret)
- );
- return ret != 0;
-}
-
-/*------------------------- 32 bit atomic operations -------------------------*/
-
-static inline int
-rte_atomic32_cmpset(volatile uint32_t *dst, uint32_t exp, uint32_t src)
-{
- uint8_t res;
-
- asm volatile(
- MPLOCKED
- "cmpxchgl %[src], %[dst];"
- "sete %[res];"
- : [res] "=a" (res), /* output */
- [dst] "=m" (*dst)
- : [src] "r" (src), /* input */
- "a" (exp),
- "m" (*dst)
- : "memory"); /* no-clobber list */
- return res;
-}
-
-static inline uint32_t
-rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val)
-{
- asm volatile(
- MPLOCKED
- "xchgl %0, %1;"
- : "=r" (val), "=m" (*dst)
- : "0" (val), "m" (*dst)
- : "memory"); /* no-clobber list */
- return val;
-}
-
-static inline int rte_atomic32_test_and_set(rte_atomic32_t *v)
-{
- return rte_atomic32_cmpset((volatile uint32_t *)&v->cnt, 0, 1);
-}
-
-static inline void
-rte_atomic32_inc(rte_atomic32_t *v)
-{
- asm volatile(
- MPLOCKED
- "incl %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : "m" (v->cnt) /* input */
- );
-}
-
-static inline void
-rte_atomic32_dec(rte_atomic32_t *v)
-{
- asm volatile(
- MPLOCKED
- "decl %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : "m" (v->cnt) /* input */
- );
-}
-
-static inline int rte_atomic32_inc_and_test(rte_atomic32_t *v)
-{
- uint8_t ret;
-
- asm volatile(
- MPLOCKED
- "incl %[cnt] ; "
- "sete %[ret]"
- : [cnt] "+m" (v->cnt), /* output */
- [ret] "=qm" (ret)
- );
- return ret != 0;
-}
-
-static inline int rte_atomic32_dec_and_test(rte_atomic32_t *v)
-{
- uint8_t ret;
-
- asm volatile(MPLOCKED
- "decl %[cnt] ; "
- "sete %[ret]"
- : [cnt] "+m" (v->cnt), /* output */
- [ret] "=qm" (ret)
- );
- return ret != 0;
-}
-
-#endif /* !RTE_FORCE_INTRINSICS */
#ifdef __cplusplus
}
diff --git a/lib/eal/x86/include/rte_atomic_32.h b/lib/eal/x86/include/rte_atomic_32.h
index 0f25863aa5..37d139f30d 100644
--- a/lib/eal/x86/include/rte_atomic_32.h
+++ b/lib/eal/x86/include/rte_atomic_32.h
@@ -20,193 +20,5 @@
/*------------------------- 64 bit atomic operations -------------------------*/
-#ifndef RTE_FORCE_INTRINSICS
-static inline int
-rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src)
-{
- uint8_t res;
- union {
- struct {
- uint32_t l32;
- uint32_t h32;
- };
- uint64_t u64;
- } _exp, _src;
-
- _exp.u64 = exp;
- _src.u64 = src;
-
-#ifndef __PIC__
- asm volatile (
- MPLOCKED
- "cmpxchg8b (%[dst]);"
- "setz %[res];"
- : [res] "=a" (res) /* result in eax */
- : [dst] "S" (dst), /* esi */
- "b" (_src.l32), /* ebx */
- "c" (_src.h32), /* ecx */
- "a" (_exp.l32), /* eax */
- "d" (_exp.h32) /* edx */
- : "memory" ); /* no-clobber list */
-#else
- asm volatile (
- "xchgl %%ebx, %%edi;\n"
- MPLOCKED
- "cmpxchg8b (%[dst]);"
- "setz %[res];"
- "xchgl %%ebx, %%edi;\n"
- : [res] "=a" (res) /* result in eax */
- : [dst] "S" (dst), /* esi */
- "D" (_src.l32), /* ebx */
- "c" (_src.h32), /* ecx */
- "a" (_exp.l32), /* eax */
- "d" (_exp.h32) /* edx */
- : "memory" ); /* no-clobber list */
-#endif
-
- return res;
-}
-
-static inline uint64_t
-rte_atomic64_exchange(volatile uint64_t *dest, uint64_t val)
-{
- uint64_t old;
-
- do {
- old = *dest;
- } while (rte_atomic64_cmpset(dest, old, val) == 0);
-
- return old;
-}
-
-static inline void
-rte_atomic64_init(rte_atomic64_t *v)
-{
- int success = 0;
- uint64_t tmp;
-
- while (success == 0) {
- tmp = v->cnt;
- success = rte_atomic64_cmpset((volatile uint64_t *)&v->cnt,
- tmp, 0);
- }
-}
-
-static inline int64_t
-rte_atomic64_read(rte_atomic64_t *v)
-{
- int success = 0;
- uint64_t tmp;
-
- while (success == 0) {
- tmp = v->cnt;
- /* replace the value by itself */
- success = rte_atomic64_cmpset((volatile uint64_t *)&v->cnt,
- tmp, tmp);
- }
- return tmp;
-}
-
-static inline void
-rte_atomic64_set(rte_atomic64_t *v, int64_t new_value)
-{
- int success = 0;
- uint64_t tmp;
-
- while (success == 0) {
- tmp = v->cnt;
- success = rte_atomic64_cmpset((volatile uint64_t *)&v->cnt,
- tmp, new_value);
- }
-}
-
-static inline void
-rte_atomic64_add(rte_atomic64_t *v, int64_t inc)
-{
- int success = 0;
- uint64_t tmp;
-
- while (success == 0) {
- tmp = v->cnt;
- success = rte_atomic64_cmpset((volatile uint64_t *)&v->cnt,
- tmp, tmp + inc);
- }
-}
-
-static inline void
-rte_atomic64_sub(rte_atomic64_t *v, int64_t dec)
-{
- int success = 0;
- uint64_t tmp;
-
- while (success == 0) {
- tmp = v->cnt;
- success = rte_atomic64_cmpset((volatile uint64_t *)&v->cnt,
- tmp, tmp - dec);
- }
-}
-
-static inline void
-rte_atomic64_inc(rte_atomic64_t *v)
-{
- rte_atomic64_add(v, 1);
-}
-
-static inline void
-rte_atomic64_dec(rte_atomic64_t *v)
-{
- rte_atomic64_sub(v, 1);
-}
-
-static inline int64_t
-rte_atomic64_add_return(rte_atomic64_t *v, int64_t inc)
-{
- int success = 0;
- uint64_t tmp;
-
- while (success == 0) {
- tmp = v->cnt;
- success = rte_atomic64_cmpset((volatile uint64_t *)&v->cnt,
- tmp, tmp + inc);
- }
-
- return tmp + inc;
-}
-
-static inline int64_t
-rte_atomic64_sub_return(rte_atomic64_t *v, int64_t dec)
-{
- int success = 0;
- uint64_t tmp;
-
- while (success == 0) {
- tmp = v->cnt;
- success = rte_atomic64_cmpset((volatile uint64_t *)&v->cnt,
- tmp, tmp - dec);
- }
-
- return tmp - dec;
-}
-
-static inline int rte_atomic64_inc_and_test(rte_atomic64_t *v)
-{
- return rte_atomic64_add_return(v, 1) == 0;
-}
-
-static inline int rte_atomic64_dec_and_test(rte_atomic64_t *v)
-{
- return rte_atomic64_sub_return(v, 1) == 0;
-}
-
-static inline int rte_atomic64_test_and_set(rte_atomic64_t *v)
-{
- return rte_atomic64_cmpset((volatile uint64_t *)&v->cnt, 0, 1);
-}
-
-static inline void rte_atomic64_clear(rte_atomic64_t *v)
-{
- rte_atomic64_set(v, 0);
-}
-#endif
#endif /* _RTE_ATOMIC_I686_H_ */
diff --git a/lib/eal/x86/include/rte_atomic_64.h b/lib/eal/x86/include/rte_atomic_64.h
index 0a7a2131e0..1cd12695a2 100644
--- a/lib/eal/x86/include/rte_atomic_64.h
+++ b/lib/eal/x86/include/rte_atomic_64.h
@@ -22,163 +22,6 @@
/*------------------------- 64 bit atomic operations -------------------------*/
-#ifndef RTE_FORCE_INTRINSICS
-static inline int
-rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src)
-{
- uint8_t res;
-
-
- asm volatile(
- MPLOCKED
- "cmpxchgq %[src], %[dst];"
- "sete %[res];"
- : [res] "=a" (res), /* output */
- [dst] "=m" (*dst)
- : [src] "r" (src), /* input */
- "a" (exp),
- "m" (*dst)
- : "memory"); /* no-clobber list */
-
- return res;
-}
-
-static inline uint64_t
-rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val)
-{
- asm volatile(
- MPLOCKED
- "xchgq %0, %1;"
- : "=r" (val), "=m" (*dst)
- : "0" (val), "m" (*dst)
- : "memory"); /* no-clobber list */
- return val;
-}
-
-static inline void
-rte_atomic64_init(rte_atomic64_t *v)
-{
- v->cnt = 0;
-}
-
-static inline int64_t
-rte_atomic64_read(rte_atomic64_t *v)
-{
- return v->cnt;
-}
-
-static inline void
-rte_atomic64_set(rte_atomic64_t *v, int64_t new_value)
-{
- v->cnt = new_value;
-}
-
-static inline void
-rte_atomic64_add(rte_atomic64_t *v, int64_t inc)
-{
- asm volatile(
- MPLOCKED
- "addq %[inc], %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : [inc] "ir" (inc), /* input */
- "m" (v->cnt)
- );
-}
-
-static inline void
-rte_atomic64_sub(rte_atomic64_t *v, int64_t dec)
-{
- asm volatile(
- MPLOCKED
- "subq %[dec], %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : [dec] "ir" (dec), /* input */
- "m" (v->cnt)
- );
-}
-
-static inline void
-rte_atomic64_inc(rte_atomic64_t *v)
-{
- asm volatile(
- MPLOCKED
- "incq %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : "m" (v->cnt) /* input */
- );
-}
-
-static inline void
-rte_atomic64_dec(rte_atomic64_t *v)
-{
- asm volatile(
- MPLOCKED
- "decq %[cnt]"
- : [cnt] "=m" (v->cnt) /* output */
- : "m" (v->cnt) /* input */
- );
-}
-
-static inline int64_t
-rte_atomic64_add_return(rte_atomic64_t *v, int64_t inc)
-{
- int64_t prev = inc;
-
- asm volatile(
- MPLOCKED
- "xaddq %[prev], %[cnt]"
- : [prev] "+r" (prev), /* output */
- [cnt] "=m" (v->cnt)
- : "m" (v->cnt) /* input */
- );
- return prev + inc;
-}
-
-static inline int64_t
-rte_atomic64_sub_return(rte_atomic64_t *v, int64_t dec)
-{
- return rte_atomic64_add_return(v, -dec);
-}
-
-static inline int rte_atomic64_inc_and_test(rte_atomic64_t *v)
-{
- uint8_t ret;
-
- asm volatile(
- MPLOCKED
- "incq %[cnt] ; "
- "sete %[ret]"
- : [cnt] "+m" (v->cnt), /* output */
- [ret] "=qm" (ret)
- );
-
- return ret != 0;
-}
-
-static inline int rte_atomic64_dec_and_test(rte_atomic64_t *v)
-{
- uint8_t ret;
-
- asm volatile(
- MPLOCKED
- "decq %[cnt] ; "
- "sete %[ret]"
- : [cnt] "+m" (v->cnt), /* output */
- [ret] "=qm" (ret)
- );
- return ret != 0;
-}
-
-static inline int rte_atomic64_test_and_set(rte_atomic64_t *v)
-{
- return rte_atomic64_cmpset((volatile uint64_t *)&v->cnt, 0, 1);
-}
-
-static inline void rte_atomic64_clear(rte_atomic64_t *v)
-{
- v->cnt = 0;
-}
-#endif
/*------------------------ 128 bit atomic operations -------------------------*/
diff --git a/lib/eal/x86/include/rte_byteorder.h b/lib/eal/x86/include/rte_byteorder.h
index bcf4a02225..5a9e5f0762 100644
--- a/lib/eal/x86/include/rte_byteorder.h
+++ b/lib/eal/x86/include/rte_byteorder.h
@@ -18,55 +18,6 @@ extern "C" {
#define RTE_BYTE_ORDER RTE_LITTLE_ENDIAN
#endif
-#ifndef RTE_FORCE_INTRINSICS
-/*
- * An architecture-optimized byte swap for a 16-bit value.
- *
- * Do not use this function directly. The preferred function is rte_bswap16().
- */
-static inline uint16_t rte_arch_bswap16(uint16_t _x)
-{
- uint16_t x = _x;
- asm volatile ("xchgb %b[x1],%h[x2]"
- : [x1] "=Q" (x)
- : [x2] "0" (x)
- );
- return x;
-}
-
-/*
- * An architecture-optimized byte swap for a 32-bit value.
- *
- * Do not use this function directly. The preferred function is rte_bswap32().
- */
-static inline uint32_t rte_arch_bswap32(uint32_t _x)
-{
- uint32_t x = _x;
- asm volatile ("bswap %[x]"
- : [x] "+r" (x)
- );
- return x;
-}
-
-#define rte_bswap16(x) ((uint16_t)(__builtin_constant_p(x) ? \
- rte_constant_bswap16(x) : \
- rte_arch_bswap16(x)))
-
-#define rte_bswap32(x) ((uint32_t)(__builtin_constant_p(x) ? \
- rte_constant_bswap32(x) : \
- rte_arch_bswap32(x)))
-
-#define rte_bswap64(x) ((uint64_t)(__builtin_constant_p(x) ? \
- rte_constant_bswap64(x) : \
- rte_arch_bswap64(x)))
-
-#ifdef RTE_ARCH_I686
-#include "rte_byteorder_32.h"
-#else
-#include "rte_byteorder_64.h"
-#endif
-
-#endif /* !RTE_FORCE_INTRINSICS */
#ifdef __cplusplus
}
diff --git a/lib/eal/x86/include/rte_spinlock.h b/lib/eal/x86/include/rte_spinlock.h
index 273bbdc39c..104da6bd78 100644
--- a/lib/eal/x86/include/rte_spinlock.h
+++ b/lib/eal/x86/include/rte_spinlock.h
@@ -19,55 +19,6 @@ extern "C" {
#define RTE_RTM_MAX_RETRIES (20)
#define RTE_XABORT_LOCK_BUSY (0xff)
-#ifndef RTE_FORCE_INTRINSICS
-static inline void
-rte_spinlock_lock(rte_spinlock_t *sl)
- __rte_no_thread_safety_analysis
-{
- int lock_val = 1;
- asm volatile (
- "1:\n"
- "xchg %[locked], %[lv]\n"
- "test %[lv], %[lv]\n"
- "jz 3f\n"
- "2:\n"
- "pause\n"
- "cmpl $0, %[locked]\n"
- "jnz 2b\n"
- "jmp 1b\n"
- "3:\n"
- : [locked] "=m" (sl->locked), [lv] "=q" (lock_val)
- : "[lv]" (lock_val)
- : "memory");
-}
-
-static inline void
-rte_spinlock_unlock (rte_spinlock_t *sl)
- __rte_no_thread_safety_analysis
-{
- int unlock_val = 0;
- asm volatile (
- "xchg %[locked], %[ulv]\n"
- : [locked] "=m" (sl->locked), [ulv] "=q" (unlock_val)
- : "[ulv]" (unlock_val)
- : "memory");
-}
-
-static inline int
-rte_spinlock_trylock (rte_spinlock_t *sl)
- __rte_no_thread_safety_analysis
-{
- int lockval = 1;
-
- asm volatile (
- "xchg %[locked], %[lockval]"
- : [locked] "=m" (sl->locked), [lockval] "=q" (lockval)
- : "[lockval]" (lockval)
- : "memory");
-
- return lockval == 0;
-}
-#endif
extern uint8_t rte_rtm_supported;
--
2.53.0
^ permalink raw reply related
* [RFC 6/7] net/nbl: remove unused rte_atomic16 field
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Dimon Zhao, Leon Yu, Sam Chen
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
The tx_current_queue was defined as rte_atomic16_t which
is deprecated. Remove it since it was never used.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
drivers/net/nbl/nbl_hw/nbl_resource.h | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/net/nbl/nbl_hw/nbl_resource.h b/drivers/net/nbl/nbl_hw/nbl_resource.h
index bf5a9461f5..f2182ba6bc 100644
--- a/drivers/net/nbl/nbl_hw/nbl_resource.h
+++ b/drivers/net/nbl/nbl_hw/nbl_resource.h
@@ -225,7 +225,6 @@ struct nbl_res_info {
u16 base_qid;
u16 lcore_max;
u16 *pf_qid_to_lcore_id;
- rte_atomic16_t tx_current_queue;
};
struct nbl_resource_mgt {
--
2.53.0
^ permalink raw reply related
* [RFC 5/7] net/bonding: use stdatomic
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Chas Williams, Min Hu (Connor)
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
The old rte_atomic16 functions are deprecated.
Replace with rte_stdatomic for managing warning flag.
Can also use fetch_or and exchange to avoid CAS here.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
drivers/net/bonding/eth_bond_8023ad_private.h | 4 ++--
drivers/net/bonding/rte_eth_bond_8023ad.c | 18 ++++--------------
2 files changed, 6 insertions(+), 16 deletions(-)
diff --git a/drivers/net/bonding/eth_bond_8023ad_private.h b/drivers/net/bonding/eth_bond_8023ad_private.h
index ab7d15f81a..1756c9307d 100644
--- a/drivers/net/bonding/eth_bond_8023ad_private.h
+++ b/drivers/net/bonding/eth_bond_8023ad_private.h
@@ -9,7 +9,7 @@
#include <rte_ether.h>
#include <rte_byteorder.h>
-#include <rte_atomic.h>
+#include <rte_stdatomic.h>
#include <rte_flow.h>
#include "rte_eth_bond_8023ad.h"
@@ -143,7 +143,7 @@ struct port {
volatile uint64_t rx_marker_timer;
uint64_t warning_timer;
- volatile uint16_t warnings_to_show;
+ RTE_ATOMIC(uint16_t) warnings_to_show;
/** Memory pool used to allocate slow queues */
struct rte_mempool *slow_pool;
diff --git a/drivers/net/bonding/rte_eth_bond_8023ad.c b/drivers/net/bonding/rte_eth_bond_8023ad.c
index ba88f6d261..641aae1a67 100644
--- a/drivers/net/bonding/rte_eth_bond_8023ad.c
+++ b/drivers/net/bonding/rte_eth_bond_8023ad.c
@@ -171,27 +171,17 @@ timer_is_running(uint64_t *timer)
static void
set_warning_flags(struct port *port, uint16_t flags)
{
- int retval;
- uint16_t old;
- uint16_t new_flag = 0;
-
- do {
- old = port->warnings_to_show;
- new_flag = old | flags;
- retval = rte_atomic16_cmpset(&port->warnings_to_show, old, new_flag);
- } while (unlikely(retval == 0));
+ rte_atomic_fetch_or_explicit(&port->warnings_to_show, flags, rte_memory_order_relaxed);
}
static void
show_warnings(uint16_t member_id)
{
struct port *port = &bond_mode_8023ad_ports[member_id];
- uint8_t warnings;
-
- do {
- warnings = port->warnings_to_show;
- } while (rte_atomic16_cmpset(&port->warnings_to_show, warnings, 0) == 0);
+ uint16_t warnings;
+ warnings = rte_atomic_exchange_explicit(&port->warnings_to_show, 0,
+ rte_memory_order_relaxed);
if (!warnings)
return;
--
2.53.0
^ permalink raw reply related
* [RFC 4/7] net/zxdh: work around GCC bitfield uninit false positive
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Junlong Wang, Ming Ran
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
GCC's -Wmaybe-uninitialized analysis cannot follow struct
initialization through bitfield reads. The warning is currently
masked by inline assembly elsewhere limiting analysis depth; it
surfaces once the EAL atomic and spinlock primitives switch to
compiler intrinsics.
Replace the struct initializer with an explicit memset() so the
full-width initialization is visible to the analyzer.
Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85301
Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110743
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
drivers/net/zxdh/zxdh_msg.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/net/zxdh/zxdh_msg.c b/drivers/net/zxdh/zxdh_msg.c
index 4b01daf37a..8f88181a3f 100644
--- a/drivers/net/zxdh/zxdh_msg.c
+++ b/drivers/net/zxdh/zxdh_msg.c
@@ -728,13 +728,15 @@ zxdh_bar_chan_sync_msg_reps_get(uint64_t subchan_addr,
int
zxdh_bar_chan_sync_msg_send(struct zxdh_pci_bar_msg *in, struct zxdh_msg_recviver_mem *result)
{
- struct zxdh_bar_msg_header msg_header = {0};
+ struct zxdh_bar_msg_header msg_header;
uint16_t seq_id = 0;
uint64_t subchan_addr = 0;
uint32_t time_out_cnt = 0;
uint16_t valid = 0;
int ret = 0;
+ memset(&msg_header, 0, sizeof(msg_header));
+
ret = zxdh_bar_chan_send_para_check(in, result);
if (ret != ZXDH_BAR_MSG_OK)
goto exit;
--
2.53.0
^ permalink raw reply related
* [RFC 3/7] ring: use C11 atomic operations for MP/SP head/tail
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Konstantin Ananyev, Wathsala Vithanage
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
Last caller of rte_atomic32_cmpset() in lib/, blocking deprecation
of the rte_atomicNN_*() family.
Replace cmpset with rte_atomic_compare_exchange_weak_explicit(),
and convert head/tail loads/stores from implicit seq_cst to explicit
acquire/release. Matches the HTS/RTS pattern.
Acquire-load of d->head orders the subsequent load of s->tail (was
rte_smp_rmb()). Acquire-load of s->tail pairs with the release-store
of the counterpart tail in __rte_ring_update_tail(), which subsumes
the previous wmb/rmb barriers.
Weak CAS avoids arm64's hidden inner retry; the outer do-while already
loops. CAS orderings relaxed: no data published by the reservation.
The now-unused 'enqueue' parameter of __rte_ring_update_tail() is
removed; both call sites updated.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
lib/ring/rte_ring_generic_pvt.h | 64 +++++++++++++++++++++++----------
1 file changed, 45 insertions(+), 19 deletions(-)
diff --git a/lib/ring/rte_ring_generic_pvt.h b/lib/ring/rte_ring_generic_pvt.h
index affd2d5ba7..9497f6737b 100644
--- a/lib/ring/rte_ring_generic_pvt.h
+++ b/lib/ring/rte_ring_generic_pvt.h
@@ -23,21 +23,25 @@
*/
static __rte_always_inline void
__rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
- uint32_t new_val, uint32_t single, uint32_t enqueue)
+ uint32_t new_val, uint32_t single,
+ uint32_t enqueue __rte_unused)
{
- if (enqueue)
- rte_smp_wmb();
- else
- rte_smp_rmb();
/*
* If there are other enqueues/dequeues in progress that preceded us,
* we need to wait for them to complete
*/
if (!single)
- rte_wait_until_equal_32((volatile uint32_t *)(uintptr_t)&ht->tail, old_val,
- rte_memory_order_relaxed);
+ rte_wait_until_equal_32((volatile uint32_t *)(uintptr_t)&ht->tail,
+ old_val, rte_memory_order_relaxed);
- ht->tail = new_val;
+ /*
+ * Release ordering on the tail store ensures that the slot reads
+ * (dequeue) or writes (enqueue) performed by this thread are visible
+ * to the other side before the new tail value is observed.
+ * Pairs with the acquire load of the counterpart's tail in
+ * __rte_ring_headtail_move_head().
+ */
+ rte_atomic_store_explicit(&ht->tail, new_val, rte_memory_order_release);
}
/**
@@ -76,25 +80,35 @@ __rte_ring_headtail_move_head(struct rte_ring_headtail *d,
{
unsigned int max = n;
int success;
+ uint32_t tail;
do {
/* Reset n to the initial burst count */
n = max;
- *old_head = d->head;
+ /*
+ * Acquire load: orders this load before the load of s->tail
+ * below (replaces rte_smp_rmb() in the previous version) and
+ * re-establishes ordering after a failed CAS on retry.
+ */
+ *old_head = rte_atomic_load_explicit(&d->head,
+ rte_memory_order_acquire);
- /* add rmb barrier to avoid load/load reorder in weak
- * memory model. It is noop on x86
+ /*
+ * Acquire load on the counterpart's tail pairs with the
+ * release store in __rte_ring_update_tail() on the other
+ * side, ensuring slot operations performed there are visible
+ * before the caller accesses the reserved slots.
*/
- rte_smp_rmb();
+ tail = rte_atomic_load_explicit(&s->tail, rte_memory_order_acquire);
/*
* The subtraction is done between two unsigned 32bits value
* (the result is always modulo 32 bits even if we have
- * *old_head > s->tail). So 'entries' is always between 0
+ * *old_head > tail). So 'entries' is always between 0
* and capacity (which is < size).
*/
- *entries = (capacity + s->tail - *old_head);
+ *entries = (capacity + tail - *old_head);
/* check that we have enough room in ring */
if (unlikely(n > *entries))
@@ -106,12 +120,24 @@ __rte_ring_headtail_move_head(struct rte_ring_headtail *d,
*new_head = *old_head + n;
if (is_st) {
- d->head = *new_head;
+ rte_atomic_store_explicit(&d->head, *new_head, rte_memory_order_relaxed);
success = 1;
- } else
- success = rte_atomic32_cmpset(
- (uint32_t *)(uintptr_t)&d->head,
- *old_head, *new_head);
+ } else {
+ /*
+ * Weak CAS: the outer do-while handles spurious
+ * failures, so we avoid the strong variant's
+ * internal retry (which on arm64 wraps the LL/SC
+ * pair in a hidden inner loop).
+ *
+ * Relaxed on both success and failure: this CAS
+ * does not publish data. Slot data visibility is
+ * provided by the acquire loads above and the
+ * release store of tail in __rte_ring_update_tail().
+ */
+ success = rte_atomic_compare_exchange_weak_explicit(
+ &d->head, old_head, *new_head,
+ rte_memory_order_relaxed, rte_memory_order_relaxed);
+ }
} while (unlikely(success == 0));
return n;
}
--
2.53.0
^ permalink raw reply related
* [RFC 2/7] eal: reimplement rte_smp_*mb with rte_atomic_thread_fence
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev
Cc: Stephen Hemminger, Wathsala Vithanage, Bibo Mao,
David Christensen, Sun Yuechi, Bruce Richardson,
Konstantin Ananyev
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
The rte_smp_mb(), rte_smp_wmb() and rte_smp_rmb() functions were
flagged as deprecated by commit 3ec965b6de12 ("doc: update atomic
operation deprecation") in 2021 but nothing came of it.
Reimplement them as inline wrappers over rte_atomic_thread_fence()
and drop the deprecation notice.
The API is preserved; only the implementation changes.
Generated code is unchanged on x86 (seq_cst keeps the lock-addl
trick, release/acquire collapse to a compiler barrier under TSO).
On arm64, release/acquire emit dmb ish instead of dmb ishst/ishld;
the difference is below measurement noise.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/rel_notes/deprecation.rst | 8 --
lib/eal/arm/include/rte_atomic_32.h | 6 --
lib/eal/arm/include/rte_atomic_64.h | 6 --
lib/eal/include/generic/rte_atomic.h | 106 +++++++++++--------------
lib/eal/loongarch/include/rte_atomic.h | 6 --
lib/eal/ppc/include/rte_atomic.h | 6 --
lib/eal/riscv/include/rte_atomic.h | 6 --
lib/eal/x86/include/rte_atomic.h | 33 +++-----
8 files changed, 57 insertions(+), 120 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 346c517623..03b763b472 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -47,14 +47,6 @@ Deprecation Notices
operations must be used for patches that need to be merged in 20.08 onwards.
This change will not introduce any performance degradation.
-* rte_smp_*mb: These APIs provide full barrier functionality. However, many
- use cases do not require full barriers. To support such use cases, DPDK has
- adopted atomic operations from
- https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html. These
- operations and a new wrapper ``rte_atomic_thread_fence`` instead of
- ``__atomic_thread_fence`` must be used for patches that need to be merged in
- 20.08 onwards. This change will not introduce any performance degradation.
-
* lib: will fix extending some enum/define breaking the ABI. There are multiple
samples in DPDK that enum/define terminated with a ``.*MAX.*`` value which is
used by iterators, and arrays holding these values are sized with this
diff --git a/lib/eal/arm/include/rte_atomic_32.h b/lib/eal/arm/include/rte_atomic_32.h
index 0b9a0dfa30..3809ddefb7 100644
--- a/lib/eal/arm/include/rte_atomic_32.h
+++ b/lib/eal/arm/include/rte_atomic_32.h
@@ -21,12 +21,6 @@ extern "C" {
#define rte_rmb() __sync_synchronize()
-#define rte_smp_mb() rte_mb()
-
-#define rte_smp_wmb() rte_wmb()
-
-#define rte_smp_rmb() rte_rmb()
-
#define rte_io_mb() rte_mb()
#define rte_io_wmb() rte_wmb()
diff --git a/lib/eal/arm/include/rte_atomic_64.h b/lib/eal/arm/include/rte_atomic_64.h
index 181bb60929..c9b41f6212 100644
--- a/lib/eal/arm/include/rte_atomic_64.h
+++ b/lib/eal/arm/include/rte_atomic_64.h
@@ -24,12 +24,6 @@ extern "C" {
#define rte_rmb() asm volatile("dmb oshld" : : : "memory")
-#define rte_smp_mb() asm volatile("dmb ish" : : : "memory")
-
-#define rte_smp_wmb() asm volatile("dmb ishst" : : : "memory")
-
-#define rte_smp_rmb() asm volatile("dmb ishld" : : : "memory")
-
#define rte_io_mb() rte_mb()
#define rte_io_wmb() rte_wmb()
diff --git a/lib/eal/include/generic/rte_atomic.h b/lib/eal/include/generic/rte_atomic.h
index 0a4f3f8528..4e9d230f85 100644
--- a/lib/eal/include/generic/rte_atomic.h
+++ b/lib/eal/include/generic/rte_atomic.h
@@ -49,69 +49,8 @@ static inline void rte_wmb(void);
* occur before the LOAD operations generated after.
*/
static inline void rte_rmb(void);
-///@}
-
-/** @name SMP Memory Barrier
- */
-///@{
-/**
- * General memory barrier between lcores
- *
- * Guarantees that the LOAD and STORE operations that precede the
- * rte_smp_mb() call are globally visible across the lcores
- * before the LOAD and STORE operations that follows it.
- *
- * @note
- * This function is deprecated.
- * It provides similar synchronization primitive as atomic fence,
- * but has different syntax and memory ordering semantic. Hence
- * deprecated for the simplicity of memory ordering semantics in use.
- *
- * rte_atomic_thread_fence(rte_memory_order_acq_rel) should be used instead.
- */
-static inline void rte_smp_mb(void);
-/**
- * Write memory barrier between lcores
- *
- * Guarantees that the STORE operations that precede the
- * rte_smp_wmb() call are globally visible across the lcores
- * before the STORE operations that follows it.
- *
- * @note
- * This function is deprecated.
- * It provides similar synchronization primitive as atomic fence,
- * but has different syntax and memory ordering semantic. Hence
- * deprecated for the simplicity of memory ordering semantics in use.
- *
- * rte_atomic_thread_fence(rte_memory_order_release) should be used instead.
- * The fence also guarantees LOAD operations that precede the call
- * are globally visible across the lcores before the STORE operations
- * that follows it.
- */
-static inline void rte_smp_wmb(void);
-
-/**
- * Read memory barrier between lcores
- *
- * Guarantees that the LOAD operations that precede the
- * rte_smp_rmb() call are globally visible across the lcores
- * before the LOAD operations that follows it.
- *
- * @note
- * This function is deprecated.
- * It provides similar synchronization primitive as atomic fence,
- * but has different syntax and memory ordering semantic. Hence
- * deprecated for the simplicity of memory ordering semantics in use.
- *
- * rte_atomic_thread_fence(rte_memory_order_acquire) should be used instead.
- * The fence also guarantees LOAD operations that precede the call
- * are globally visible across the lcores before the STORE operations
- * that follows it.
- */
-static inline void rte_smp_rmb(void);
///@}
-
/** @name I/O Memory Barrier
*/
///@{
@@ -164,6 +103,51 @@ static inline void rte_io_rmb(void);
*/
static inline void rte_atomic_thread_fence(rte_memory_order memorder);
+
+/** @name SMP Memory Barrier
+ */
+///@{
+/**
+ * General memory barrier between lcores
+ *
+ * Guarantees that the LOAD and STORE operations that precede the
+ * rte_smp_mb() call are globally visible across the lcores
+ * before the LOAD and STORE operations that follows it.
+ */
+static __rte_always_inline void
+rte_smp_mb(void)
+{
+ rte_atomic_thread_fence(rte_memory_order_seq_cst);
+}
+
+/**
+ * Write memory barrier between lcores
+ *
+ * Guarantees that the STORE operations that precede the
+ * rte_smp_wmb() call are globally visible across the lcores
+ * before the STORE operations that follows it.
+ */
+static __rte_always_inline void
+rte_smp_wmb(void)
+{
+ rte_atomic_thread_fence(rte_memory_order_release);
+}
+
+/**
+ * Read memory barrier between lcores
+ *
+ * Guarantees that the LOAD operations that precede the
+ * rte_smp_rmb() call are globally visible across the lcores
+ * before the LOAD operations that follows it.
+ */
+static __rte_always_inline void
+rte_smp_rmb(void)
+{
+ rte_atomic_thread_fence(rte_memory_order_acquire);
+}
+
+///@}
+
/*------------------------- 16 bit atomic operations -------------------------*/
#ifndef RTE_TOOLCHAIN_MSVC
diff --git a/lib/eal/loongarch/include/rte_atomic.h b/lib/eal/loongarch/include/rte_atomic.h
index c8066a4612..49e0c67020 100644
--- a/lib/eal/loongarch/include/rte_atomic.h
+++ b/lib/eal/loongarch/include/rte_atomic.h
@@ -22,12 +22,6 @@ extern "C" {
#define rte_rmb() rte_mb()
-#define rte_smp_mb() rte_mb()
-
-#define rte_smp_wmb() rte_mb()
-
-#define rte_smp_rmb() rte_mb()
-
#define rte_io_mb() rte_mb()
#define rte_io_wmb() rte_mb()
diff --git a/lib/eal/ppc/include/rte_atomic.h b/lib/eal/ppc/include/rte_atomic.h
index 10acc238f9..1da5afccbf 100644
--- a/lib/eal/ppc/include/rte_atomic.h
+++ b/lib/eal/ppc/include/rte_atomic.h
@@ -24,12 +24,6 @@ extern "C" {
#define rte_rmb() asm volatile("sync" : : : "memory")
-#define rte_smp_mb() rte_mb()
-
-#define rte_smp_wmb() rte_wmb()
-
-#define rte_smp_rmb() rte_rmb()
-
#define rte_io_mb() rte_mb()
#define rte_io_wmb() rte_wmb()
diff --git a/lib/eal/riscv/include/rte_atomic.h b/lib/eal/riscv/include/rte_atomic.h
index 66346ad474..dd10ad5127 100644
--- a/lib/eal/riscv/include/rte_atomic.h
+++ b/lib/eal/riscv/include/rte_atomic.h
@@ -27,12 +27,6 @@ extern "C" {
#define rte_rmb() asm volatile("fence r, r" : : : "memory")
-#define rte_smp_mb() rte_mb()
-
-#define rte_smp_wmb() rte_wmb()
-
-#define rte_smp_rmb() rte_rmb()
-
#define rte_io_mb() asm volatile("fence iorw, iorw" : : : "memory")
#define rte_io_wmb() asm volatile("fence orw, ow" : : : "memory")
diff --git a/lib/eal/x86/include/rte_atomic.h b/lib/eal/x86/include/rte_atomic.h
index e071e4234e..a850b0257c 100644
--- a/lib/eal/x86/include/rte_atomic.h
+++ b/lib/eal/x86/include/rte_atomic.h
@@ -23,10 +23,6 @@
#define rte_rmb() _mm_lfence()
-#define rte_smp_wmb() rte_compiler_barrier()
-
-#define rte_smp_rmb() rte_compiler_barrier()
-
#ifdef __cplusplus
extern "C" {
#endif
@@ -63,20 +59,6 @@ extern "C" {
* So below we use that technique for rte_smp_mb() implementation.
*/
-static __rte_always_inline void
-rte_smp_mb(void)
-{
-#ifdef RTE_TOOLCHAIN_MSVC
- _mm_mfence();
-#else
-#ifdef RTE_ARCH_I686
- asm volatile("lock addl $0, -128(%%esp); " ::: "memory");
-#else
- asm volatile("lock addl $0, -128(%%rsp); " ::: "memory");
-#endif
-#endif
-}
-
#define rte_io_mb() rte_mb()
#define rte_io_wmb() rte_compiler_barrier()
@@ -93,10 +75,19 @@ rte_smp_mb(void)
static __rte_always_inline void
rte_atomic_thread_fence(rte_memory_order memorder)
{
- if (memorder == rte_memory_order_seq_cst)
- rte_smp_mb();
- else
+ if (memorder == rte_memory_order_seq_cst) {
+#ifdef RTE_TOOLCHAIN_MSVC
+ _mm_mfence();
+#else
+#ifdef RTE_ARCH_I686
+ asm volatile("lock addl $0, -128(%%esp); " ::: "memory");
+#else
+ asm volatile("lock addl $0, -128(%%rsp); " ::: "memory");
+#endif
+#endif
+ } else {
__rte_atomic_thread_fence(memorder);
+ }
}
#ifdef __cplusplus
--
2.53.0
^ permalink raw reply related
* [RFC 1/7] doc: update versions in deprecation file
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
This document was mentioned 23.11 release and needs update
for upcoming 26.11.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/rel_notes/deprecation.rst | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 35c9b4e06c..346c517623 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -7,8 +7,8 @@ ABI and API Deprecation
See the guidelines document for details of the :doc:`ABI policy
<../contributing/abi_policy>`.
-With DPDK 23.11, there will be a new major ABI version: 24.
-This means that during the development of 23.11,
+With DPDK 26.11, there will be a new major ABI version: 27.
+This means that during the development of 26.11,
new items may be added to structs or enums,
even if those additions involve an ABI compatibility breakage.
--
2.53.0
^ permalink raw reply related
* [RFC 0/7] prepare deprecation of rte_atomicNN_*() family
From: Stephen Hemminger @ 2026-05-21 4:17 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
The goal is to land every deprecation currently listed in the release
notes by the 26.11 ABI bump. Working back from there: any function to
be removed in 26.11 needs to be marked __rte_deprecated by 26.07, and
all in-tree users converted off it before the marker patch goes in so
CI stays clean.
This series is the preparatory work for the rte_atomicNN_*() family
under that plan. It does not yet add the __rte_deprecated marker;
that's a separate follow-up once the remaining in-tree users are
converted. Other items on the deprecation list (VXLAN_GPE,
pipeline/table/port legacy API, the MAX enum fix, regexdev, pdump,
TM locks) will follow as their own series on the same timeline.
Patch 3 is the load-bearing change: the last lib/ caller of
rte_atomic32_cmpset() is converted, clearing the way.
Patch 7 drops RTE_FORCE_INTRINSICS entirely. With the option always on,
the asm implementations of atomics, spinlock and byteorder become dead
code. ~900 lines deleted; the patch most worth review attention.
This makes it easier to flag the rte_atomicNN as deprecated
since they are all in one place.
Patch 2 retires the rte_smp_*mb deprecation notice (open since 2021)
by reimplementing those APIs as wrappers over rte_atomic_thread_fence,
preserving the API for readability. Patches 5 and 6 convert and clean
up two driver users (bonding, nbl).
Patch 4 is a preparatory workaround for a pre-existing GCC bitfield
-Wmaybe-uninitialized false positive in net/zxdh, surfaced by the
improved compiler visibility after patch 7. Placed ahead of patch 7
to keep every commit bisectable.
Follow on patches will mechanically convert drivers.
If driver writer fixes it themselves; all the beter.
Stephen Hemminger (7):
doc: update versions in deprecation file
eal: reimplement rte_smp_*mb with rte_atomic_thread_fence
ring: use C11 atomic operations for MP/SP head/tail
net/zxdh: work around GCC bitfield uninit false positive
net/bonding: use stdatomic
net/nbl: remove unused rte_atomic16 field
config: use RTE_FORCE_INTRINSICS on all platforms
config/arm/meson.build | 1 -
config/loongarch/meson.build | 1 -
config/meson.build | 3 -
config/riscv/meson.build | 1 -
doc/guides/rel_notes/deprecation.rst | 12 +-
doc/guides/rel_notes/release_26_07.rst | 5 +
drivers/net/bonding/eth_bond_8023ad_private.h | 4 +-
drivers/net/bonding/rte_eth_bond_8023ad.c | 18 +-
drivers/net/nbl/nbl_hw/nbl_resource.h | 1 -
drivers/net/zxdh/zxdh_msg.c | 4 +-
lib/eal/arm/include/rte_atomic_32.h | 9 -
lib/eal/arm/include/rte_atomic_64.h | 9 -
lib/eal/arm/include/rte_byteorder.h | 3 -
lib/eal/arm/include/rte_spinlock.h | 3 -
lib/eal/include/generic/rte_atomic.h | 164 ++++----------
lib/eal/include/generic/rte_byteorder.h | 2 -
lib/eal/include/generic/rte_spinlock.h | 10 -
lib/eal/loongarch/include/rte_atomic.h | 9 -
lib/eal/loongarch/include/rte_spinlock.h | 3 -
lib/eal/ppc/include/rte_atomic.h | 179 ---------------
lib/eal/ppc/include/rte_byteorder.h | 13 --
lib/eal/ppc/include/rte_spinlock.h | 26 ---
lib/eal/riscv/include/rte_atomic.h | 9 -
lib/eal/riscv/include/rte_spinlock.h | 3 -
lib/eal/x86/include/rte_atomic.h | 205 +-----------------
lib/eal/x86/include/rte_atomic_32.h | 188 ----------------
lib/eal/x86/include/rte_atomic_64.h | 157 --------------
lib/eal/x86/include/rte_byteorder.h | 49 -----
lib/eal/x86/include/rte_spinlock.h | 49 -----
lib/ring/rte_ring_generic_pvt.h | 64 ++++--
30 files changed, 118 insertions(+), 1086 deletions(-)
--
2.53.0
^ permalink raw reply
* [PATCH] net/ena: use C11 atomic operations in platform header
From: Stephen Hemminger @ 2026-05-21 4:07 UTC (permalink / raw)
To: dev
Cc: Stephen Hemminger, Shai Brandes, Evgeny Schemeilin, Ron Beider,
Amit Bernstein, Wajeeh Atrash
Convert the legacy rte_atomic32_t and rte_atomic32_{inc,dec,set,read}
macros to C11 stdatomic equivalents. This clears another user of the
rte_atomicNN_*() family ahead of its deprecation.
Memory ordering is kept at seq_cst, matching the implicit ordering of
the legacy API. The ena_com access patterns can be audited and orderings
tightened in a follow-up.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
drivers/net/ena/base/ena_plat_dpdk.h | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ena/base/ena_plat_dpdk.h b/drivers/net/ena/base/ena_plat_dpdk.h
index c84420de22..83b354d9da 100644
--- a/drivers/net/ena/base/ena_plat_dpdk.h
+++ b/drivers/net/ena/base/ena_plat_dpdk.h
@@ -40,7 +40,7 @@ typedef uint64_t dma_addr_t;
#endif
#define ENA_PRIu64 PRIu64
-#define ena_atomic32_t rte_atomic32_t
+typedef RTE_ATOMIC(int32_t) ena_atomic32_t;
#define ena_mem_handle_t const struct rte_memzone *
#define SZ_256 (256U)
@@ -267,10 +267,14 @@ ena_mem_alloc_coherent(struct rte_eth_dev_data *data, size_t size,
#define ENA_REG_READ32(bus, reg) \
__extension__ ({ (void)(bus); rte_read32_relaxed((reg)); })
-#define ATOMIC32_INC(i32_ptr) rte_atomic32_inc(i32_ptr)
-#define ATOMIC32_DEC(i32_ptr) rte_atomic32_dec(i32_ptr)
-#define ATOMIC32_SET(i32_ptr, val) rte_atomic32_set(i32_ptr, val)
-#define ATOMIC32_READ(i32_ptr) rte_atomic32_read(i32_ptr)
+#define ATOMIC32_INC(i32_ptr) \
+ rte_atomic_fetch_add_explicit((i32_ptr), 1, rte_memory_order_seq_cst)
+#define ATOMIC32_DEC(i32_ptr) \
+ rte_atomic_fetch_sub_explicit((i32_ptr), 1, rte_memory_order_seq_cst)
+#define ATOMIC32_SET(i32_ptr, val) \
+ rte_atomic_store_explicit((i32_ptr), (val), rte_memory_order_seq_cst)
+#define ATOMIC32_READ(i32_ptr) \
+ rte_atomic_load_explicit((i32_ptr), rte_memory_order_seq_cst)
#define msleep(x) rte_delay_us(x * 1000)
#define udelay(x) rte_delay_us(x)
--
2.53.0
^ permalink raw reply related
* [PATCH v4] ring: fix zero-copy burst API documentation
From: Zhiguang Jin @ 2026-05-21 3:00 UTC (permalink / raw)
To: Thomas Monjalon, Konstantin Ananyev, Wathsala Vithanage,
Dharmik Thakkar, Honnappa Nagarahalli
Cc: dev, Zhiguang Jin, stable
In-Reply-To: <20260519133853.889-1-jinzhiguang@kylinos.cn>
These are burst APIs relying on RTE_RING_QUEUE_VARIABLE behavior, they
operate on a best-effort basis and return the actual number of
objects processed (between 0 and n).
Update description to match implementation.
Fixes: 47bec9a5ca9f ("ring: add zero copy API")
Cc: stable@dpdk.org
Signed-off-by: Zhiguang Jin <jinzhiguang@kylinos.cn>
Acked-by: Konstantin Ananyev <konstantin.ananyev@huawei.com>
---
Notes:
v4:
- The name in the .mailmap file has been corrected and placed in the correct location.
v3:
- Resend properly threaded to the v1 patch. (No code/text changes)
v2:
- Update description to match actual behavior per Maintainer's suggestion.
.mailmap | 1 +
lib/ring/rte_ring_peek_zc.h | 8 ++++----
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/.mailmap b/.mailmap
index 4d26d9c286..3172cb08e3 100644
--- a/.mailmap
+++ b/.mailmap
@@ -1934,6 +1934,7 @@ Zhichao Zeng <zhichaox.zeng@intel.com>
Zhigang Hu <zhigang.hu@intel.com>
Zhigang Lu <zlu@ezchip.com>
Zhiguang He <hezhiguang3@huawei.com>
+Zhiguang Jin <jinzhiguang@kylinos.cn>
Zhihong Peng <zhihongx.peng@intel.com>
Zhihong Wang <wangzhihong.wzh@bytedance.com> <zhihong.wang@intel.com>
Zhike Wang <wangzhike@jd.com> <wangzk320@163.com>
diff --git a/lib/ring/rte_ring_peek_zc.h b/lib/ring/rte_ring_peek_zc.h
index 3254fe0481..43d6a53075 100644
--- a/lib/ring/rte_ring_peek_zc.h
+++ b/lib/ring/rte_ring_peek_zc.h
@@ -235,7 +235,7 @@ rte_ring_enqueue_zc_bulk_start(struct rte_ring *r, unsigned int n,
* If non-NULL, returns the amount of space in the ring after the
* reservation operation has finished.
* @return
- * The number of objects that can be enqueued, either 0 or n
+ * The actual number of objects that can be enqueued.
*/
static __rte_always_inline unsigned int
rte_ring_enqueue_zc_burst_elem_start(struct rte_ring *r, unsigned int esize,
@@ -265,7 +265,7 @@ rte_ring_enqueue_zc_burst_elem_start(struct rte_ring *r, unsigned int esize,
* If non-NULL, returns the amount of space in the ring after the
* reservation operation has finished.
* @return
- * The number of objects that can be enqueued, either 0 or n.
+ * The actual number of objects that can be enqueued.
*/
static __rte_always_inline unsigned int
rte_ring_enqueue_zc_burst_start(struct rte_ring *r, unsigned int n,
@@ -442,7 +442,7 @@ rte_ring_dequeue_zc_bulk_start(struct rte_ring *r, unsigned int n,
* If non-NULL, returns the number of remaining ring entries after the
* dequeue has finished.
* @return
- * The number of objects that can be dequeued, either 0 or n.
+ * The actual number of objects that can be dequeued.
*/
static __rte_always_inline unsigned int
rte_ring_dequeue_zc_burst_elem_start(struct rte_ring *r, unsigned int esize,
@@ -471,7 +471,7 @@ rte_ring_dequeue_zc_burst_elem_start(struct rte_ring *r, unsigned int esize,
* If non-NULL, returns the number of remaining ring entries after the
* dequeue has finished.
* @return
- * The number of objects that can be dequeued, either 0 or n.
+ * The actual number of objects that can be dequeued.
*/
static __rte_always_inline unsigned int
rte_ring_dequeue_zc_burst_start(struct rte_ring *r, unsigned int n,
--
2.53.0
^ permalink raw reply related
* Re: [PATCH v3] ring: fix zero-copy burst API documentation
From: jinzhiguang @ 2026-05-21 2:54 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Konstantin Ananyev, Wathsala Vithanage, Dharmik Thakkar,
Honnappa Nagarahalli, dev, stable
In-Reply-To: <D6Rbi5-MSBSppW6LoqeJTg@monjalon.net>
On 2026/5/20 16:59:42, Thomas Monjalon wrote:
> 19/05/2026 15:38, jinzhiguang:
>> --- a/.mailmap
>> +++ b/.mailmap
>> @@ -756,6 +756,7 @@ Jing Chen <jing.d.chen@intel.com>
>> Jingguo Fu <jingguox.fu@intel.com>
>> Jingjing Wu <jingjing.wu@intel.com>
>> Jingzhao Ni <jingzhao.ni@arm.com>
>> +jinzhiguang <jinzhiguang@kylinos.cn>
> Maybe you want to insert a space? What is your first name and family name?
> Please use capital letters where appropriate.
Thank you for the review! I will update the .mailmap in the v4 version
of the patch.
^ permalink raw reply
* Re: [PATCH v2 19/23] dma/idxd: remove specific bus type
From: Bruce Richardson @ 2026-05-20 17:20 UTC (permalink / raw)
To: David Marchand; +Cc: dev, thomas, stephen, Kevin Laatz
In-Reply-To: <20260506155201.2709810-20-david.marchand@redhat.com>
On Wed, May 06, 2026 at 05:51:51PM +0200, David Marchand wrote:
> There is nothing that requires a specific bus type.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
This patch exposes an underlying issue with the DSA driver. When applied,
attempting to probe the workqueues gives the output:
"EAL: Device wq0.0 is already probed"
For each device. This is because the device is not zeroed in the scan
function, leaving a non-NULL driver pointer. See inline below.
/Bruce
> ---
> drivers/dma/idxd/idxd_bus.c | 38 ++++++++++++++-----------------------
> 1 file changed, 14 insertions(+), 24 deletions(-)
>
> diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c
> index 93bde69ff9..daf5f48ac1 100644
> --- a/drivers/dma/idxd/idxd_bus.c
> +++ b/drivers/dma/idxd/idxd_bus.c
> @@ -41,34 +41,24 @@ struct rte_dsa_device {
> };
>
> /* forward prototypes */
> -struct dsa_bus;
> static int dsa_scan(void);
> static bool dsa_match(const struct rte_driver *drv, const struct rte_device *dev);
> static int dsa_probe_device(struct rte_driver *drv, struct rte_device *dev);
> static enum rte_iova_mode dsa_get_iommu_class(void);
> static int dsa_addr_parse(const char *name, void *addr);
>
> -/**
> - * Structure describing the DSA bus
> - */
> -struct dsa_bus {
> - struct rte_bus bus; /**< Inherit the generic class */
> - struct rte_driver driver; /**< Driver struct for devices to point to */
> +struct rte_bus dsa_bus = {
> + .scan = dsa_scan,
> + .probe = rte_bus_generic_probe,
> + .match = dsa_match,
> + .probe_device = dsa_probe_device,
> + .find_device = rte_bus_generic_find_device,
> + .get_iommu_class = dsa_get_iommu_class,
> + .parse = dsa_addr_parse,
> };
>
> -struct dsa_bus dsa_bus = {
> - .bus = {
> - .scan = dsa_scan,
> - .probe = rte_bus_generic_probe,
> - .match = dsa_match,
> - .probe_device = dsa_probe_device,
> - .find_device = rte_bus_generic_find_device,
> - .get_iommu_class = dsa_get_iommu_class,
> - .parse = dsa_addr_parse,
> - },
> - .driver = {
> - .name = "dmadev_idxd",
> - },
> +struct rte_driver dsa_driver = {
> + .name = "dmadev_idxd",
> };
>
> static inline const char *
> @@ -267,7 +257,7 @@ static bool dsa_match(const struct rte_driver *drv, const struct rte_device *dev
> {
> const struct rte_dsa_device *dsa_dev = RTE_BUS_DEVICE(dev, *dsa_dev);
>
> - if (drv == &dsa_bus.driver) {
> + if (drv == &dsa_driver) {
> char type[64], name[64];
>
> if (read_wq_string(dsa_dev, "type", type, sizeof(type)) >= 0 &&
> @@ -319,7 +309,7 @@ dsa_scan(void)
> continue;
> }
> strlcpy(dev->wq_name, wq->d_name, sizeof(dev->wq_name));
> - rte_bus_add_device(&dsa_bus.bus, &dev->device);
> + rte_bus_add_device(&dsa_bus, &dev->device);
The issue with the driver is here. Before the strlcpy, and the previous
dsa_addr_parse() call, we just need to add:
memset(dev, 0, sizeof(*dev));
Since malloc does not zero the memory. [Alternatively, we can change malloc
to a zeroing equivalent].
Do you want me to do a separate patch to fix this, or roll the fix change
into this patch?
> devcount++;
>
> read_device_int(dev, "numa_node", &numa_node);
> @@ -357,8 +347,8 @@ dsa_addr_parse(const char *name, void *addr)
> return 0;
> }
>
> -RTE_REGISTER_BUS(dsa, dsa_bus.bus);
> +RTE_REGISTER_BUS(dsa, dsa_bus);
> RTE_INIT(dsa_bus_init)
> {
> - rte_bus_add_driver(&dsa_bus.bus, &dsa_bus.driver);
> + rte_bus_add_driver(&dsa_bus, &dsa_driver);
> }
> --
> 2.53.0
>
^ permalink raw reply
* [PATCH] eal: fix data race in hugepage prefault
From: Stephen Hemminger @ 2026-05-20 17:08 UTC (permalink / raw)
To: dev
Cc: Stephen Hemminger, stable, Michal Sieron, Thomas Monjalon,
Anatoly Burakov, Bruce Richardson
In-Reply-To: <20260520125756.530808-1-michal.sieron@nokia.com>
The prefault step in alloc_seg() reads a value from the hugepage and
writes it back unchanged to force the kernel to commit the backing
page. The read and write were not atomic, which races with concurrent
access to the same physical page from a secondary process attaching
to the hugetlbfs-backed mapping during rte_eal_init().
Replace the non-atomic load+store with a single atomic fetch-or of
zero. This touches the page with an atomic read-modify-write without
changing its contents, eliminating the race while preserving the
original intent of forcing a write fault.
Fixes: 0f1631be24bd ("mem: fix page fault trigger")
Cc: stable@dpdk.org
Reported-by: Michal Sieron <michal.sieron@nokia.com>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
.mailmap | 1 +
lib/eal/linux/eal_memalloc.c | 7 ++++---
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/.mailmap b/.mailmap
index 4d26d9c286..07c49eb32f 100644
--- a/.mailmap
+++ b/.mailmap
@@ -1086,6 +1086,7 @@ Michal Mazurek <maz@semihalf.com>
Michal Michalik <michal.michalik@intel.com>
Michal Nowak <michal2.nowak@intel.com>
Michal Schmidt <mschmidt@redhat.com>
+Michal Sieron <michal.sieron@nokia.com>
Michal Swiatkowski <michal.swiatkowski@intel.com>
Michal Wilczynski <michal.wilczynski@intel.com>
Michał Mirosław <michal.miroslaw@atendesoftware.pl> <mirq-linux@rere.qmqm.pl>
diff --git a/lib/eal/linux/eal_memalloc.c b/lib/eal/linux/eal_memalloc.c
index a39bc31c7b..e73a0c11a6 100644
--- a/lib/eal/linux/eal_memalloc.c
+++ b/lib/eal/linux/eal_memalloc.c
@@ -25,6 +25,7 @@
#include <linux/falloc.h>
#include <linux/mman.h> /* for hugetlb-related mmap flags */
+#include <rte_atomic.h>
#include <rte_common.h>
#include <rte_log.h>
#include <rte_eal.h>
@@ -597,10 +598,10 @@ alloc_seg(struct rte_memseg *ms, void *addr, int socket_id,
/* we need to trigger a write to the page to enforce page fault and
* ensure that page is accessible to us, but we can't overwrite value
- * that is already there, so read the old value, and write itback.
- * kernel populates the page with zeroes initially.
+ * that is already there.
+ * Use an atomic OR with zero to touch the page without changing its contents.
*/
- *(volatile int *)addr = *(volatile int *)addr;
+ (void)rte_atomic_fetch_or_explicit((int *)addr, 0, rte_memory_order_relaxed);
iova = rte_mem_virt2iova(addr);
if (iova == RTE_BAD_PHYS_ADDR) {
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox