* RE: [PATCH v5] mempool: improve cache behaviour and performance
From: Morten Brørup @ 2026-05-26 10:37 UTC (permalink / raw)
To: Bruce Richardson
Cc: dev, Andrew Rybchenko, Jingjing Wu, Praveen Shetty,
Hemant Agrawal, Sachin Saxena
In-Reply-To: <ahVqYpVXMn59Tl2d@bricha3-mobl1.ger.corp.intel.com>
> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Tuesday, 26 May 2026 11.40
>
> On Tue, May 26, 2026 at 10:41:44AM +0200, Morten Brørup wrote:
> > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > Sent: Friday, 22 May 2026 18.12
> > >
> > > On Sun, Apr 19, 2026 at 09:55:26AM +0000, Morten Brørup wrote:
> > > > This patch refactors the mempool cache to eliminate some
> unexpected
> > > > behaviour and reduce the mempool cache miss rate.
> > > >
> > >
> > > Agree in principle with most of these changes. As we dicussed at
> the
> > > DPDK
> > > summit conference, only issue I really have is with the threshold
> > > limits
> > > here - allocating and freeing only half the cache at a time seems
> > > overly
> > > conservative. Thinking about use-cases:
> > >
> > > 1 for apps where alloc + free (generally Rx+Tx) is on the same
> core(s),
> > > then we should run (almost) entirely out of cache.
> >
> > I strongly disagree about any goal to run the cache low.
> > The primary goal is to minimize the cache miss (refill and replenish)
> rate.
> >
> > > 2 for apps where we have alloc and free on different cores, then we
> > > have
> > > some caches always being filled and others always being emptied
> >
> > Agree.
> >
> > >
> > > For case #1, we only need worry about the thresholds for the odd
> case
> > > where we have a burst that causes us to overflow our cache (and we
> > > can't increase our cache size to cope and avoid that).
> > > Otherwise the thresholds don't matter.
> >
> > It seems like you assume the application only does something like
> this:
> > Rx -> Rewrite -> Tx
> >
> > In that case, the per-lcore cache only needs capacity for one burst,
> yes.
> > With my patch, the cache can be rightsized by requesting a cache size
> of 2 * burst size.
> > (Then the fill level will be either size/2 or empty, i.e. one burst
> or zero.
> > This also happens to meet your suggested goal about low fill level,
> which I disagree with.)
> >
> > However, I don't think that is a realistic use case.
> > Many apps do something like this:
> >
> > |-> Rewrite ->|
> > Rx ->| |-> Tx
> > |-> Hold |
> > Release ->|
> >
>
> I agree, that's why our cache size needs to be bigger than our burst
> size.
>
> > They often hold back packets before they are transmitted.
> > For a simple router, when the destination IP address is not in the
> neighbor table, packets to that IP address are queued until ARP/ND has
> been resolved, and then they are dequeued and transmitted.
> > Or apps performing shaping or pacing, where packets are held back in
> queues, and dequeued at the appropriate time.
> > For such apps, the waves are much bigger (than the simple Rx-
> >Rewrite->Tx use case).
> >
> > With a random enqueue/dequeue pattern, replenishing/draining the
> cache to size/2 minimizes the probability of reaching one of its edges
> (empty or full), triggering a "cache miss" (refill/replenish).
> >
> > > However, for case #2, the thresholds are constantly involved as
> > > we
> > > always are going to backing store. In this case, we really want to
> have
> > > the
> > > allocs *always* fill the cache completely, and the frees completely
> > > empty
> > > the cache.
> >
> > Agree.
> >
> > >
> > > Because of this, while we want to avoid cases where we fill the
> cache
> > > completely only to have a further free causing it to be flushed,
> > > because of
> > > case #2 we cannot be overly conservative in how much we free/empty.
> > > Ideally, we want to fill to full less a single burst, and empty
> leaving
> > > only a single burst in the cache. Unfortunately, we don't know what
> > > those
> > > burst limits are, so we have to try and guess the best behaviour
> from
> > > everything else.
> >
> > I agree about not wanting to be overly conservative.
> > But in the use cases I have described for #1, I don't think a target
> fill level of size/2 is overly conservative.
> >
> > I also acknowledge that this patch doubles the mempool cache miss
> rate for #2.
> > E.g. with a cache size of 512 and burst size of 64, the per-burst
> miss rate will be 64 / (1/2 * 512) = 1/4, compared to 64 / 512 = 1/8
> with a full replenish/drain algorithm.
> >
> > In theory, we could make it build time configurable to optimize
> mempools for #2.
> > But mempools are also used for other objects than mbufs, so that
> would have unwanted side effects for non-mbuf mempools.
> >
> > If we went for an algorithm targeting replenish/drain at 25 % from
> the edges, the per-burst miss rate for #2 would be: 64 / (3/4 * 512) =
> 1/6.
> >
> > How about addressing #2 in the release notes:
> > We describe that the cache refill/drain algorithm has been changed to
> only refill/drain to 50 % of the cache size, so pipelined applications
> performing Rx (mempool get) and Tx (mempool put) on separate cores
> should configure their mbuf pools with double the cache size of what
> they previously were to achieve similar performance.
> >
> > >
> > > All that said, commits with specific suggestions inline.
> > >
> > > /Bruce
> > >
> > > <snip>
> > >
> > > > diff --git a/lib/mempool/rte_mempool.h
> b/lib/mempool/rte_mempool.h
> > > > index 2e54fc4466..432c43ab15 100644
> > > > --- a/lib/mempool/rte_mempool.h
> > > > +++ b/lib/mempool/rte_mempool.h
> > > > @@ -89,7 +89,7 @@ struct __rte_cache_aligned
> rte_mempool_debug_stats
> > > {
> > > > */
> > > > struct __rte_cache_aligned rte_mempool_cache {
> > > > uint32_t size; /**< Size of the cache */
> > > > - uint32_t flushthresh; /**< Threshold before we flush excess
> > > elements */
> > > > + uint32_t flushthresh; /**< Obsolete; for API/ABI
> compatibility
> > > purposes only */
> > > > uint32_t len; /**< Current cache count */
> > > > #ifdef RTE_LIBRTE_MEMPOOL_STATS
> > > > uint32_t unused;
> > > > @@ -107,8 +107,10 @@ struct __rte_cache_aligned rte_mempool_cache
> {
> > > > /**
> > > > * Cache objects
> > > > *
> > > > - * Cache is allocated to this size to allow it to overflow
> in
> > > certain
> > > > - * cases to avoid needless emptying of cache.
> > > > + * Note:
> > > > + * Cache is allocated at double size for API/ABI
> compatibility
> > > purposes only.
> > > > + * When reducing its size at an API/ABI breaking release,
> > > > + * remember to add a cache guard after it.
> > > > */
> > > > alignas(RTE_CACHE_LINE_SIZE) void
> > > *objs[RTE_MEMPOOL_CACHE_MAX_SIZE * 2];
> > > > };
> > > > @@ -1046,12 +1048,17 @@ rte_mempool_free(struct rte_mempool *mp);
> > > > * @param cache_size
> > > > * If cache_size is non-zero, the rte_mempool library will try
> to
> > > > * limit the accesses to the common lockless pool, by
> maintaining
> > > a
> > > > - * per-lcore object cache. This argument must be lower or
> equal to
> > > > - * RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.
> > > > + * per-lcore object cache. This argument must be an even
> number,
> > > > + * lower or equal to RTE_MEMPOOL_CACHE_MAX_SIZE and n.
> > > > * The access to the per-lcore table is of course
> > > > * faster than the multi-producer/consumer pool. The cache can
> be
> > > > * disabled if the cache_size argument is set to 0; it can be
> > > useful to
> > > > * avoid losing objects in cache.
> > > > + * Note:
> > > > + * Mempool put/get requests of more than cache_size / 2
> objects
> > > may be
> > > > + * partially or fully served directly by the multi-
> > > producer/consumer
> > > > + * pool, to avoid the overhead of copying the objects twice
> > > (instead of
> > > > + * once) when using the cache as a bounce buffer.
> > > > * @param private_data_size
> > > > * The size of the private data appended after the mempool
> > > > * structure. This is useful for storing some private data
> after
> > > the
> > > > @@ -1390,24 +1397,32 @@ rte_mempool_do_generic_put(struct
> rte_mempool
> > > *mp, void * const *obj_table,
> > > > RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_bulk, 1);
> > > > RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_objs, n);
> > > >
> > > > - __rte_assume(cache->flushthresh <=
> RTE_MEMPOOL_CACHE_MAX_SIZE *
> > > 2);
> > > > - __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE * 2);
> > > > - __rte_assume(cache->len <= cache->flushthresh);
> > > > - if (likely(cache->len + n <= cache->flushthresh)) {
> > > > + __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> > > > + __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE
> / 2);
> > > > + __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> > > > + __rte_assume(cache->len <= cache->size);
> > > > + if (likely(cache->len + n <= cache->size)) {
> > > > /* Sufficient room in the cache for the objects. */
> > > > cache_objs = &cache->objs[cache->len];
> > > > cache->len += n;
> > > > - } else if (n <= cache->flushthresh) {
> > > > + } else if (n <= cache->size / 2) {
> > > > /*
> > > > - * The cache is big enough for the objects, but - as
> > > detected by
> > > > - * the comparison above - has insufficient room for
> them.
> > > > - * Flush the cache to make room for the objects.
> > > > + * The number of objects is within the cache bounce
> buffer
> > > limit,
> > > > + * but - as detected by the comparison above - the
> cache
> > > has
> > > > + * insufficient room for them.
> > > > + * Flush the cache to the backend to make room for
> the
> > > objects;
> > > > + * flush (size / 2) objects from the bottom of the
> cache,
> > > where
> > > > + * objects are less hot, and move down the remaining
> > > objects, which
> > > > + * are more hot, from the upper half of the cache.
> > > > */
> > > > - cache_objs = &cache->objs[0];
> > > > - rte_mempool_ops_enqueue_bulk(mp, cache_objs, cache-
> >len);
> > > > - cache->len = n;
> > > > + __rte_assume(cache->len > cache->size / 2);
> > > > + rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0],
> cache-
> > > >size / 2);
> > > > + rte_memcpy(&cache->objs[0], &cache->objs[cache->size
> / 2],
> > > > + sizeof(void *) * (cache->len - cache-
> >size /
> > > 2));
> > > > + cache_objs = &cache->objs[cache->len - cache->size /
> 2];
> > > > + cache->len = cache->len - cache->size / 2 + n;
> > >
> > > The flushing of only half the cache I'm not so certain about. I
> agree
> > > that
> > > we want to not flush to empty, but I also think that we want to do
> more
> > > than a half-flush, especially since we do an enqueue to the cache
> > > immediately afterwards. Consider the case where we have a cache
> size of
> > > 128, and we do an enqueue of 32, with the cache currently full. In
> that
> > > case we only flush 64, reducing the cache to 64, but then
> immediately
> > > bringing it back up to 96.
> >
> > I thought in depth about whether the flush/replenish sizes should
> consider the request size or not. (E.g. if I should replenish size/2 or
> size/2+request.)
> > I decided for not considering the request size, for two reasons:
> > a) It roughly doesn't matter, especially when considering a sequence
> of random get/put requests.
> > b) The size of the backend transactions become fixed, which has
> performance benefits: With my patch, they are always size/2, so if the
> cache size is 2^N, the backend transactions are 2^N and CPU cache
> aligned.
> >
> > > For cases where we have pipelines with all
> > > alloc
> > > on one core and all free on another, this half-flush would be
> > > inefficient.
> > >
> > > Instead, I would look to have a lower target threshold post-flush,
> and
> > > I
> > > would suggest 25% - taking into account the newly freed buffers.
> >
> > It's not good for #1.
> > I agree that it is better for #2. But I don't think #2 is the likely
> use case.
> >
> > After our discussion at the summit, I did start working a patch
> targeting fill levels at 25% from the cache edges, but I don't think
> it's better; so I'd rather stick with a target fill level of 50%.
> >
> > > For example:
> > >
> > > /* if n > our target of 1/4 full, flush everything,
> > > * else flush so that we end up with 1/4 full after n added.
> > > */
> > > flush_count = n > cache->size/4 ? cache->len :
> > > (cache->len + n) - cache->size/4;
> > >
> > >
> > > > } else {
> > > > - /* The request itself is too big for the cache. */
> > > > + /* The request itself is too big. */
> > > > goto driver_enqueue_stats_incremented;
> > >
> > > I think original comment is better. The request itself is not too
> big
> > > for
> > > the whole mempool, just for the cache.
> >
> > Ack.
> >
> > >
> > > > }
> > > >
> > > > @@ -1524,7 +1539,7 @@ rte_mempool_do_generic_get(struct
> rte_mempool
> > > *mp, void **obj_table,
> > > > /* The cache is a stack, so copy will be in reverse order.
> */
> > > > cache_objs = &cache->objs[cache->len];
> > > >
> > > > - __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE * 2);
> > > > + __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> > > > if (likely(n <= cache->len)) {
> > > > /* The entire request can be satisfied from the
> cache. */
> > > > RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk,
> 1);
> > > > @@ -1548,13 +1563,13 @@ rte_mempool_do_generic_get(struct
> rte_mempool
> > > *mp, void **obj_table,
> > > > for (index = 0; index < len; index++)
> > > > *obj_table++ = *--cache_objs;
> > > >
> > > > - /* Dequeue below would overflow mem allocated for cache? */
> > > > - if (unlikely(remaining > RTE_MEMPOOL_CACHE_MAX_SIZE))
> > > > + /* Dequeue below would exceed the cache bounce buffer
> limit? */
> > > > + __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE
> / 2);
> > > > + if (unlikely(remaining > cache->size / 2))
> > > > goto driver_dequeue;
> > > >
> > > > - /* Fill the cache from the backend; fetch size + remaining
> > > objects. */
> > > > - ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs,
> > > > - cache->size + remaining);
> > > > + /* Fill the cache from the backend; fetch (size / 2)
> objects. */
> > > > + ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, cache-
> >size /
> > > 2);
> > >
> > > Again, the cache->size / 2 doesn't seem right here. We at most
> half-
> > > fill
> > > the cache and then take some objects from that, meaning that have
> just
> > > done
> > > a re-fill of cache but end the function with it less than half
> full.
> > > Since
> > > we take from this value, I'd suggest just filling the cache
> completely.
> >
> > The issues at the edges of the cache are symmetrical.
> > If we replenish the cache to full, and the next transaction is a put,
> the cache needs to be drained.
> > That's why I replenish to size/2.
> >
>
> Not necessarily. After you fill the cache here (to 50%) you then take
> elements from it. If we filled to 100%, then we'd only hit a flush if
> we
> freed back more elements than we just allocated. Now, that's reasonably
> likely which is why I'm ok to not filling to 100%. However, doing a
> refill
> as part of the op, and then leaving the cache less than half full after
> the
> op finishes is just wasteful IMHO!
I'm targeting half full, and disregard if it is pre-op or post-op.
This makes the backend transactions cache aligned in both addressing and size, which opens an opportunity to performance optimize the mempool drivers for this.
>
> One other point. The obvious worst case scenario we want to avoid is
> constant fill/empty/fill/empty sequences. However, so long as we have
> an
> asymmetry in how we do fills and empty thresholds, a repeating sequence
> should never occur, and even pairs of fill/empty should be very rare.
> Consider the case, for example, where we fill to 100% but only drain to
> 25%. For this, we can ignore scenario #2, since we should have pretty
> good
> cache usage. For scenario #1, of allocs/frees on a single core,
> randomly
> interleaved, our worst case is:
> 1) alloc with cache empty, in which case we fill to 100%, then take the
> alloc
> 2) have a free of a burst greater than that which we just allocated,
> causing an immediate flush again.
>
> That's pretty poor behaviour, but the thing is that after the flush we
> now
> have mempool at 25% + freed burst - so expected between 25-50% of cache
> utilization. That's the sweet spot we want to target - after each
> operation, the mempool cache should ideally be at 50%. Whether the next
> op
> is an alloc or a free, our cache can handle it, and likely a couple of
> each
> in sequence. Therefore, our possible fill/empty combinations are likely
> to
> be rare occurances.
>
> [In all this, I am making the assumption that burst size is well less
> than
> cache size. Also, similar logic would be applicable for the inverse
> scenario, e.g. flush to empty (and fill burst) and fill to 75%]
I'm not so sure about this assumption.
With a cache size of 512 and a bursts of 64, the cache only holds 8 bursts.
50% is 4 bursts, and 25% is only 2 bursts.
Using a replenish/drain level in the middle requires 5 bursts in either direction to pass the edge (and trigger replenish/flush).
Using a replenish/drain level 25% from the edge requires only 3 bursts in the wrong direction to pass the edge (and trigger replenish/flush). Much higher probability with random get/put.
>
> Now, all said, I tend to agree that we want to leave space for a decent
> size burst after a fill. That is why I think that filling to 75% is
> reasonable. After an alloc that triggers a fill, I don't want the cache
> less than 50% full, but not completely full so there is room for a free
> without a flush, and similarly for a free that triggers a flush, the
> cache
> should not be empty, but also should not be more than half full.
>
> One suggestion - we could always add a simple tunable that specifies
> the
> margin, or reserved entries for alloc and free. We can then guide in
> the
> docs that the value should be e.g. "zero for apps where alloc and free
> take
> place on different cores. 20%-50% of cache is recommended where alloc
> and
> free take place on the same core"
Yes, a simple tunable is a really good idea.
At this point, I think we should optimize for use case #1, and go for the 50% fill level.
Then we can add a tunable to optimize for use case #2 later. I will try to come up with a draft for such a follow-up patch within the next few days.
The 50% fill level in this patch is not as bad for use case #2 (roughly doubling the burst miss rate from 1/8 to 1/4), compared to how bad the original algorithm is for use case #1 (very high miss probability - only two ops in the wrong direction - after drain/replenish).
-Morten
^ permalink raw reply
* [v4 0/1] net/hinic3: Fix VXLAN TSO issue
From: Feifei Wang @ 2026-05-26 12:18 UTC (permalink / raw)
To: dev
In-Reply-To: <20260520065817.931-2-wff_light@vip.163.com>
v1: The RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO flag is added to support the VXLAN TSO function
v2: Modify the commit information issue and supplement the commit information
v3: Revise review comments. First, deterine whether the hardware supports it, then add the flag bit
v4: Fix the compilation error caused by leading spaces
Feifei Wang (1):
net/hinic3: Fix VXLAN TSO issue
drivers/net/hinic3/hinic3_ethdev.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
--
2.47.0.windows.2
^ permalink raw reply
* [v4 1/1] net/hinic3: Fix VXLAN TSO issue
From: Feifei Wang @ 2026-05-26 12:18 UTC (permalink / raw)
To: dev; +Cc: Feifei Wang
In-Reply-To: <20260526121820.1093-1-wff_light@vip.163.com>
From: Feifei Wang <wangfeifei40@huawei.com>
VXLAN TSO lacks a flag bit, causing the processing function
to determine that the hardware does not support it, leading
to improper handling.
Signed-off-by: Feifei Wang <wangfeifei40@huawei.com>
---
drivers/net/hinic3/hinic3_ethdev.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
index f4eb788..4776bc1 100644
--- a/drivers/net/hinic3/hinic3_ethdev.c
+++ b/drivers/net/hinic3/hinic3_ethdev.c
@@ -652,8 +652,12 @@ hinic3_dev_configure(struct rte_eth_dev *dev)
static void
hinic3_dev_tnl_tso_support(struct rte_eth_dev_info *info, struct hinic3_nic_dev *nic_dev)
{
+ if (HINIC3_SUPPORT_VXLAN_OFFLOAD(nic_dev))
+ info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO;
+
if (HINIC3_SUPPORT_GENEVE_OFFLOAD(nic_dev))
info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
+
if (HINIC3_SUPPORT_IPXIP_OFFLOAD(nic_dev))
info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO;
}
@@ -698,7 +702,6 @@ hinic3_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
RTE_ETH_TX_OFFLOAD_TCP_TSO | RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
- if (nic_dev->feature_cap & NIC_F_HTN_CMDQ)
hinic3_dev_tnl_tso_support(info, nic_dev);
info->hash_key_size = HINIC3_RSS_KEY_SIZE;
--
2.47.0.windows.2
^ permalink raw reply related
* Re: [v4 1/1] net/hinic3: Fix VXLAN TSO issue
From: Stephen Hemminger @ 2026-05-26 13:15 UTC (permalink / raw)
To: Feifei Wang; +Cc: dev, Feifei Wang
In-Reply-To: <20260526121820.1093-2-wff_light@vip.163.com>
On Tue, 26 May 2026 20:18:18 +0800
Feifei Wang <wff_light@vip.163.com> wrote:
> From: Feifei Wang <wangfeifei40@huawei.com>
>
> VXLAN TSO lacks a flag bit, causing the processing function
> to determine that the hardware does not support it, leading
> to improper handling.
>
> Signed-off-by: Feifei Wang <wangfeifei40@huawei.com>
If this is a bug fix, it needs Fixes: tag and should
have Cc: stable@dpdk.org
> ---
> drivers/net/hinic3/hinic3_ethdev.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/hinic3/hinic3_ethdev.c b/drivers/net/hinic3/hinic3_ethdev.c
> index f4eb788..4776bc1 100644
> --- a/drivers/net/hinic3/hinic3_ethdev.c
> +++ b/drivers/net/hinic3/hinic3_ethdev.c
> @@ -652,8 +652,12 @@ hinic3_dev_configure(struct rte_eth_dev *dev)
> static void
> hinic3_dev_tnl_tso_support(struct rte_eth_dev_info *info, struct hinic3_nic_dev *nic_dev)
> {
> + if (HINIC3_SUPPORT_VXLAN_OFFLOAD(nic_dev))
> + info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO;
> +
> if (HINIC3_SUPPORT_GENEVE_OFFLOAD(nic_dev))
> info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO;
> +
> if (HINIC3_SUPPORT_IPXIP_OFFLOAD(nic_dev))
> info->tx_offload_capa |= RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO;
> }
> @@ -698,7 +702,6 @@ hinic3_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
> RTE_ETH_TX_OFFLOAD_SCTP_CKSUM |
> RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM |
> RTE_ETH_TX_OFFLOAD_TCP_TSO | RTE_ETH_TX_OFFLOAD_MULTI_SEGS;
> - if (nic_dev->feature_cap & NIC_F_HTN_CMDQ)
> hinic3_dev_tnl_tso_support(info, nic_dev);
If you remove the if here, then you need to fix indentation on
the line below.
^ permalink raw reply
* Re: [RFC v2 2/3] lib: add fastmem library
From: Stephen Hemminger @ 2026-05-26 13:23 UTC (permalink / raw)
To: Mattias Rönnblom
Cc: dev, Morten Brørup, Konstantin Ananyev,
Mattias Rönnblom, Yogaraj Baskaravel
In-Reply-To: <20260526085743.64396-3-hofors@lysator.liu.se>
On Tue, 26 May 2026 10:57:42 +0200
Mattias Rönnblom <hofors@lysator.liu.se> wrote:
> +__rte_experimental
> +void *
> +rte_fastmem_alloc(size_t size, size_t align, unsigned int flags)
> + __rte_alloc_size(1) __rte_alloc_align(2);
Should also add attribute __rte_malloc which tells compiler
that pointer returned cannot alias other memory
And add __rte_dealloc(rte_fastmem_free, 1)
which tells compiler that the returned pointer should only go
back to fastmem (not free, rte_free, etc).
^ permalink raw reply
* Re: [PATCH v1 00/23] add net/sxe2 support for flow control
From: Stephen Hemminger @ 2026-05-26 13:29 UTC (permalink / raw)
To: liujie5; +Cc: dev
In-Reply-To: <20260524093259.397506-1-liujie5@linkdatatechnology.com>
On Sun, 24 May 2026 17:32:36 +0800
liujie5@linkdatatechnology.com wrote:
> From: Jie Liu <liujie5@linkdatatechnology.com>
>
> v1:
> -- add support for flow control
> -- add support for flow control status interrupt notification
> -- add support for ipsec inline protocol offload
>
> Jie Liu (23):
> net/sxe2: support AVX512 vectorized path for Rx and Tx
> net/sxe2: add AVX2 vector data path for Rx and Tx
> drivers: add supported packet types get callback
> net/sxe2: support L2 filtering and MAC config
> drivers: support RSS feature
> net/sxe2: support TM hierarchy and shaping
> net/sxe2: support IPsec inline protocol offload
> net/sxe2: support statistics and multi-process
> drivers: interrupt handling
> net/sxe2: add NEON vec Rx/Tx burst functions
> net/sxe2: add support for VF representors
> net/sxe2: add support for custom UDP tunnel ports
> net/sxe2: support firmware version reading
> net/sxe2: implement get monitor address
> common/sxe2: add shared SFP module definitions
> net/sxe2: support SFP module info and EEPROM access
> net/sxe2: implement private dump info
> net/sxe2: add mbuf validation in Tx debug mode
> net/sxe2: add testpmd commands for private features
> net/sxe2: add private devargs parsing
> net/sxe2: support flow control status interrupt notification
> net/sxe2: update sxe2 feature matrix docs
> common/sxe2: add memseg walk callback
>
> doc/guides/nics/features/sxe2.ini | 66 +
> drivers/common/sxe2/sxe2_common.c | 156 ++
> drivers/common/sxe2/sxe2_common.h | 4 +
> drivers/common/sxe2/sxe2_flow_public.h | 633 +++++++
> drivers/common/sxe2/sxe2_ioctl_chnl.c | 179 +-
> drivers/common/sxe2/sxe2_ioctl_chnl_func.h | 18 +
> drivers/common/sxe2/sxe2_msg.h | 117 ++
> drivers/common/sxe2/sxe2_ptype.h | 1793 ++++++++++++++++++
> drivers/net/sxe2/meson.build | 56 +-
> drivers/net/sxe2/sxe2_cmd_chnl.c | 1588 +++++++++++++++-
> drivers/net/sxe2/sxe2_cmd_chnl.h | 139 ++
> drivers/net/sxe2/sxe2_drv_cmd.h | 439 ++++-
> drivers/net/sxe2/sxe2_dump.c | 304 +++
> drivers/net/sxe2/sxe2_dump.h | 12 +
> drivers/net/sxe2/sxe2_ethdev.c | 1526 ++++++++++++++-
> drivers/net/sxe2/sxe2_ethdev.h | 113 +-
> drivers/net/sxe2/sxe2_ethdev_repr.c | 610 ++++++
> drivers/net/sxe2/sxe2_ethdev_repr.h | 32 +
> drivers/net/sxe2/sxe2_filter.c | 897 +++++++++
> drivers/net/sxe2/sxe2_filter.h | 100 +
> drivers/net/sxe2/sxe2_flow.c | 1391 ++++++++++++++
> drivers/net/sxe2/sxe2_flow.h | 30 +
> drivers/net/sxe2/sxe2_flow_define.h | 144 ++
> drivers/net/sxe2/sxe2_flow_parse_action.c | 1182 ++++++++++++
> drivers/net/sxe2/sxe2_flow_parse_action.h | 23 +
> drivers/net/sxe2/sxe2_flow_parse_engine.c | 106 ++
> drivers/net/sxe2/sxe2_flow_parse_engine.h | 13 +
> drivers/net/sxe2/sxe2_flow_parse_pattern.c | 1935 ++++++++++++++++++++
> drivers/net/sxe2/sxe2_flow_parse_pattern.h | 46 +
> drivers/net/sxe2/sxe2_ipsec.c | 1565 ++++++++++++++++
> drivers/net/sxe2/sxe2_ipsec.h | 254 +++
> drivers/net/sxe2/sxe2_irq.c | 1024 +++++++++++
> drivers/net/sxe2/sxe2_irq.h | 25 +
> drivers/net/sxe2/sxe2_mac.c | 535 ++++++
> drivers/net/sxe2/sxe2_mac.h | 84 +
> drivers/net/sxe2/sxe2_mp.c | 413 +++++
> drivers/net/sxe2/sxe2_mp.h | 73 +
> drivers/net/sxe2/sxe2_queue.c | 17 +-
> drivers/net/sxe2/sxe2_rss.c | 584 ++++++
> drivers/net/sxe2/sxe2_rss.h | 81 +
> drivers/net/sxe2/sxe2_rx.c | 38 +
> drivers/net/sxe2/sxe2_rx.h | 2 +
> drivers/net/sxe2/sxe2_security.c | 335 ++++
> drivers/net/sxe2/sxe2_security.h | 77 +
> drivers/net/sxe2/sxe2_stats.c | 591 ++++++
> drivers/net/sxe2/sxe2_stats.h | 39 +
> drivers/net/sxe2/sxe2_switchdev.c | 332 ++++
> drivers/net/sxe2/sxe2_switchdev.h | 33 +
> drivers/net/sxe2/sxe2_testpmd.c | 733 ++++++++
> drivers/net/sxe2/sxe2_testpmd_lib.c | 969 ++++++++++
> drivers/net/sxe2/sxe2_testpmd_lib.h | 142 ++
> drivers/net/sxe2/sxe2_tm.c | 1169 ++++++++++++
> drivers/net/sxe2/sxe2_tm.h | 78 +
> drivers/net/sxe2/sxe2_tx.c | 7 +
> drivers/net/sxe2/sxe2_txrx.c | 174 +-
> drivers/net/sxe2/sxe2_txrx.h | 4 +
> drivers/net/sxe2/sxe2_txrx_check_mbuf.c | 595 ++++++
> drivers/net/sxe2/sxe2_txrx_check_mbuf.h | 38 +
> drivers/net/sxe2/sxe2_txrx_poll.c | 243 ++-
> drivers/net/sxe2/sxe2_txrx_vec.c | 46 +-
> drivers/net/sxe2/sxe2_txrx_vec.h | 38 +-
> drivers/net/sxe2/sxe2_txrx_vec_avx2.c | 777 ++++++++
> drivers/net/sxe2/sxe2_txrx_vec_avx512.c | 897 +++++++++
> drivers/net/sxe2/sxe2_txrx_vec_common.h | 1 -
> drivers/net/sxe2/sxe2_txrx_vec_neon.c | 707 +++++++
> drivers/net/sxe2/sxe2_vsi.c | 146 ++
> drivers/net/sxe2/sxe2_vsi.h | 12 +-
> drivers/net/sxe2/sxe2vf_regs.h | 82 +
> 68 files changed, 26531 insertions(+), 81 deletions(-)
> create mode 100644 drivers/common/sxe2/sxe2_flow_public.h
> create mode 100644 drivers/common/sxe2/sxe2_msg.h
> create mode 100644 drivers/common/sxe2/sxe2_ptype.h
> create mode 100644 drivers/net/sxe2/sxe2_dump.c
> create mode 100644 drivers/net/sxe2/sxe2_dump.h
> create mode 100644 drivers/net/sxe2/sxe2_ethdev_repr.c
> create mode 100644 drivers/net/sxe2/sxe2_ethdev_repr.h
> create mode 100644 drivers/net/sxe2/sxe2_filter.c
> create mode 100644 drivers/net/sxe2/sxe2_filter.h
> create mode 100644 drivers/net/sxe2/sxe2_flow.c
> create mode 100644 drivers/net/sxe2/sxe2_flow.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_define.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_action.c
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_action.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_engine.c
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_engine.h
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_pattern.c
> create mode 100644 drivers/net/sxe2/sxe2_flow_parse_pattern.h
> create mode 100644 drivers/net/sxe2/sxe2_ipsec.c
> create mode 100644 drivers/net/sxe2/sxe2_ipsec.h
> create mode 100644 drivers/net/sxe2/sxe2_irq.c
> create mode 100644 drivers/net/sxe2/sxe2_mac.c
> create mode 100644 drivers/net/sxe2/sxe2_mac.h
> create mode 100644 drivers/net/sxe2/sxe2_mp.c
> create mode 100644 drivers/net/sxe2/sxe2_mp.h
> create mode 100644 drivers/net/sxe2/sxe2_rss.c
> create mode 100644 drivers/net/sxe2/sxe2_rss.h
> create mode 100644 drivers/net/sxe2/sxe2_security.c
> create mode 100644 drivers/net/sxe2/sxe2_security.h
> create mode 100644 drivers/net/sxe2/sxe2_stats.c
> create mode 100644 drivers/net/sxe2/sxe2_stats.h
> create mode 100644 drivers/net/sxe2/sxe2_switchdev.c
> create mode 100644 drivers/net/sxe2/sxe2_switchdev.h
> create mode 100644 drivers/net/sxe2/sxe2_testpmd.c
> create mode 100644 drivers/net/sxe2/sxe2_testpmd_lib.c
> create mode 100644 drivers/net/sxe2/sxe2_testpmd_lib.h
> create mode 100644 drivers/net/sxe2/sxe2_tm.c
> create mode 100644 drivers/net/sxe2/sxe2_tm.h
> create mode 100644 drivers/net/sxe2/sxe2_txrx_check_mbuf.c
> create mode 100644 drivers/net/sxe2/sxe2_txrx_check_mbuf.h
> create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_avx2.c
> create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_avx512.c
> create mode 100644 drivers/net/sxe2/sxe2_txrx_vec_neon.c
> create mode 100644 drivers/net/sxe2/sxe2vf_regs.h
>
Lots of AI review feedback issues (see below).
One other thing which I have recently started looking at is use of __rte_always_inline.
Since much of DPDK has just used __rte_always_inline without justification,
the resulting code is usually slower! Therefore in any new submission,
any use of always inline requires actual benchmark data that it helps.
There are a few case like when there it can help, but very few.
Review of "[PATCH v1 00/23] net/sxe2: PMD updates" series
20 of 23 patches are in the bundle (01-20); 21-23 are missing.
Note: the series targets main, not an LTS branch.
Patch 01/23 (net/sxe2: support AVX512 vectorized path for Rx and Tx)
====================================================================
Error: sxe2_tx_mode_func_set() now contains two back-to-back
'if (tx_mode_flags & SXE2_TX_MODE_VEC_SET_MASK)' blocks. The first
sets dev->tx_pkt_burst to sxe2_tx_pkts_vec_sse/sse_simple; the
second immediately overwrites those assignments (and sets
tx_pkt_prepare = NULL before the inner if/else decides what to
set). The first block is dead. Either it was meant to be deleted
when the AVX512-aware block was added, or the new code should
have replaced rather than duplicated the old one.
Error: After this patch the first if-block above is no longer wrapped
in #ifdef RTE_ARCH_X86 (the original '-#ifdef RTE_ARCH_X86' line
was removed). On non-x86 builds the references to
sxe2_tx_pkts_vec_sse / sxe2_tx_pkts_vec_sse_simple are
undefined and the file will not compile.
Error: In sxe2_tx_queue_mbufs_release_vec(), the new non-AVX512 'else'
branch has 'buffer = txq->buffer_ring;' written twice in a row:
+ buffer = txq->buffer_ring;
+ buffer = txq->buffer_ring;
This is a dead store and indicates careless editing.
Warning: In the same function, after the new
#ifdef CC_AVX512_SUPPORT / else / #endif block, the original
trailing 'for (; i < txq->next_use; ++i) { ... }' loop is still
there. By the time control reaches it, the inner branches have
already advanced i to txq->next_use, so the loop never runs.
This is dead code; remove it.
Warning: In sxe2_tx_bufs_free_vec_avx512(), the 'normal' path
(sxe2_txrx_vec_avx512.c around the rs_thresh loop) is
misindented and has mismatched braces visually:
} else {
rte_mempool_put_bulk(mbuf_free_arr[0]->pool, ...);
mbuf_free_arr[0] = mbuf; /* wrong indent */
free_num = 1;
}
}
The code is syntactically correct but unreadable. Reindent.
Info: 'tx_mode_flags |= SXE2_TX_MODE_VEC_SSE;' (double space) and the
Yoda '(0 == (tx_mode_flags & ...))' style are inconsistent with
the rest of the function which uses 'x == 0'.
Patch 02/23 (net/sxe2: add AVX2 vector data path for Rx and Tx)
===============================================================
Warning: The commit message body ends with
Signed-off-by: ...
net/sxe2: support AVX512 vectorized path for Rx and Tx
The trailing line looks like a stray subject from patch 01
pasted in. Remove it.
Warning: The new AVX2 meson block adds '-mavx2' to c_args without
gating on cc.has_argument('-mavx2') or __AVX2__. The AVX512
block above it checks both compiler and CPU support. Apply
the same pattern to AVX2 for consistency.
Note: this patch does not delete the duplicate-block / non-x86 bugs
introduced in patch 01; it extends them with an AVX2 branch in
the second block while the first SSE-only block is still
executing first and being overwritten.
Patch 03/23 (drivers: add supported packet types get callback)
==============================================================
Error: drivers/net/sxe2/sxe2_ethdev.c new
sxe2_buffer_split_supported_hdr_ptypes_get() declares its
'size_t *no_of_elements' parameter as __rte_unused and never
writes to it. The ethdev caller
(rte_eth_buffer_split_get_supported_hdr_ptypes in
lib/ethdev/rte_ethdev.c) reads no_of_elements after the
callback returns and uses it as a loop bound. The caller's
no_of_elements is an uninitialized stack variable, so this
will iterate a garbage number of times and walk past the
ptypes[] array. Set *no_of_elements = RTE_DIM(ptypes); before
returning.
Error: drivers/common/sxe2/sxe2_ptype.h opens with
#ifndef _SXE2_PTYPE_H_
#define _SXE2_PTYPE_H_
and ends with
#endif /* _RTE_PTYPE_TUNNEL_GRENAT_H_ */
The closing comment is from a different header. Compilation
isn't affected but it makes the file look generated/copied.
Fix the comment.
Warning: The file is a 1793-line static inline function
(sxe2_init_ptype_list) plus a smaller static inline in a
header installed under drivers/common/sxe2/. Every TU that
includes it gets its own copy of the array and code. Move the
body to a .c file and expose only a prototype.
Warning: sxe2_mtu_set() in sxe2_mac.c declares 'uint16_t mtu
__rte_unused' and does nothing with the value. It only checks
dev_started and returns 0. It does not configure hardware,
does not validate against scatter Rx state, and does not
re-select Rx/Tx burst functions. ethdev will write
dev->data->mtu = mtu on return, so the driver silently accepts
any value the caller passes that passes ethdev's range check.
If this is a stub, mark it with a TODO; if not, finish it.
Info: sxe2_mac.h begins with a blank line before the SPDX comment.
Patch 04/23 (net/sxe2: support L2 filtering and MAC config)
===========================================================
No findings.
Patch 05/23 (drivers: support RSS feature)
==========================================
Info: '(uint8_t)rte_rand()' in sxe2_rss_hash_key_init throws away
56 bits of entropy per byte. Use rte_rand() once per 8 key bytes
or memcpy a uint64_t. Not a correctness bug.
Patch 06/23 (net/sxe2: support TM hierarchy and shaping)
========================================================
Error: Nine occurrences in sxe2_tm.c return positive ENOTSUP instead
of negative -ENOTSUP:
ret = ENOTSUP;
goto l_end;
Surrounding code uses -EINVAL and -ENOMEM (negative), so this
is inconsistent and wrong. A positive return is treated as
success by callers that check 'if (ret)'/'if (ret < 0)'.
Change all nine to '-ENOTSUP'.
Patch 07/23 (net/sxe2: support IPsec inline protocol offload)
=============================================================
Info: sxe2_ipsec_id_alloc() does a linear bitmap scan for every SA
allocation. For max_tx_sa/max_rx_sa in the thousands this is
slow but functionally correct. Consider rte_bitmap_scan or a
free-list.
Info: '0XFFFF' uppercase X is unusual; rest of the tree uses '0xFFFF'.
Patch 08/23 (net/sxe2: support statistics and multi-process)
============================================================
Info: 'send_reply = false;' assigns false (bool) to an int32_t. Either
change the type of send_reply to bool or use 0/1.
Patch 09/23 (drivers: interrupt handling)
=========================================
Warning: sxe2_event_intr_post() uses
rte_malloc(NULL, sizeof(struct sxe2_event_element), 0) for a
per-event control structure. This consumes hugepage memory
for an object that does not need DMA, is not shared with
secondary processes, and is freed on the same thread.
Use plain malloc().
Warning: The 'write(handler->fd[1], ¬ify_byte, 1)' return is
masked with RTE_SET_USED(nw). On EAGAIN/EINTR/short write the
element is queued but never delivered, and the consumer side
is never woken. Either loop on EINTR/EAGAIN or check the
return and unqueue/free the element on failure.
Patch 10/23 (net/sxe2: add NEON vec Rx/Tx burst functions)
==========================================================
Not reviewed in detail this round.
Patch 11/23 (net/sxe2: add support for VF representors)
=======================================================
Error: sxe2_flow_check_rss_action_attr() declares
'int32_t ret = ENOTSUP;' (positive). It is then overwritten in
every reachable path (-EINVAL in the default case, 0 at the
end), so the initial value is a dead store - but the bigger
problem is the function returns 0 after calling
rte_flow_error_set() for rss->level > 2, rss->key_len != 0, or
rss->queue_num != 0. The errors are reported via the error
object but ret is not set to -EINVAL/-ENOTSUP, so the caller
sees success. Each of those if-branches should set ret to a
negative errno and goto l_end (or return immediately).
Additionally, rte_flow_error_set() is called with positive
ENOTSUP as the error code. rte_flow API expects negative errno
(-rte_errno style). Use -ENOTSUP.
(Same dead-store pattern around 'ret = ENOTSUP;' exists at
least once more in this patch - audit all occurrences.)
Patches 12-20
=============
Not reviewed in detail this round.
General observations across the series
======================================
- Several patches mix positive and negative errno returns; this is a
recurring class of bug. Search the whole series for
'ret = E[A-Z]' (no minus) and audit each hit.
- Multiple files use 'rte_malloc' for per-call control structures
that do not need hugepage memory. Use malloc() unless the buffer
is for DMA, shared between primary/secondary, or NUMA-pinned.
- A few places use 'rte_zmalloc' (no _socket variant) for what looks
like queue-related state. Where the buffer is touched by the data
path or by secondary processes, use rte_zmalloc_socket().
- The Signed-off-by chain in some patches has stray lines after the
s-o-b (see patch 02). Re-run 'git format-patch' or hand-edit.
^ permalink raw reply
* Re: [PATCH v3 00/25] Consolidate bus driver infrastructure
From: David Marchand @ 2026-05-26 13:59 UTC (permalink / raw)
To: dev; +Cc: thomas, stephen, bruce.richardson
In-Reply-To: <CAJFAV8wAt1hyHrL0N1AxQ1kBjKoQNtuo89m_H7LHZJmtGea_gQ@mail.gmail.com>
On Tue, 26 May 2026 at 11:03, David Marchand <david.marchand@redhat.com> wrote:
>
> On Tue, 26 May 2026 at 10:42, David Marchand <david.marchand@redhat.com> wrote:
> >
> > This is a continuation of the work I started on the bus infrastructure,
> > but this time, a lot of the changes were done by a AI "friend".
> > It is still an unfinished topic as the current series focuses on probing
> > only. The detaching/cleanup aspect is postponed to another release/time.
> >
> > My AI "friend" really *sucked* at git and at separating unrelated changes,
> > so it required quite a lot of massage/polishing afterwards.
> > But it seems good enough now for upstream submission.
> >
> > I would like to see this series merged in 26.07, so that we have enough
> > time to stabilize it before the next LTS.
> > And seeing how it affects drivers, it is probably better to merge it
> > the sooner possible (so Thomas does not have to solve too many conflicts
> > when pulling next-* subtrees after, especially wrt the last patch).
> >
> >
> > This series refactors the DPDK bus infrastructure to consolidate common
> > operations and reduce code duplication across all bus drivers.
> > Currently, each bus implements its own specific device/driver lists,
> > probe logic, and lookup functions.
> > This series moves these common patterns into the EAL bus layer,
> > providing generic helpers that all buses can use.
> >
> > The refactoring removes approximately 1,400 lines of duplicated code across
> > the codebase while maintaining full functional equivalence.
> >
> > Key changes:
> > - Factorize device and driver lists into struct rte_bus
> > - Implement generic probe, device/driver lookup, and iteration helpers in EAL
> > - Introduce conversion macros (RTE_BUS_DEVICE, RTE_BUS_DRIVER, RTE_CLASS_TO_BUS_DEVICE)
> > to safely convert between generic and bus-specific types
> > - Remove bus-specific device/driver types from most driver code
> > - Move probe logic from individual buses to rte_bus_generic_probe()
> > - Separate NXP-specific metadata from generic bus structures
> >
> > Benefits:
> > - Significant code reduction (~1,400 lines removed)
> > - Consistent behavior across all bus types
> > - Simplified bus driver implementation
> > - Easier maintenance and future enhancements
> >
> > The series is structured as a progressive refactoring:
> > - Remove redundant checks and helpers (patches 1-5)
> > - Add conversion macros and factorize lists (patches 6-8)
> > - Consolidate device/driver lookup and iteration (patches 9-11)
> > - Refactor probe logic (patches 12-15)
> > - Remove bus-specific types from drivers (patches 16-23)
> >
> > Note on ABI:
> > This series breaks the ABI for drivers (changes to rte_pci_device,
> > rte_pci_driver, and similar structures for other buses). However, the DPDK
> > ABI policy does not provide guarantees for driver-level interfaces.
> >
> >
> > --
> > David Marchand
> >
> > Changes since v2:
> > - fixed dma/idxd probing as reported by Bruce,
> > - moved api_ver setting (from match to scan) in bus/uacce,
> > - fixed transient bug in the vdev code (in the middle of the series).
> > tl;dr the high level difference for bus/vdev between v2 and v3 == 0,
> > - fixed doxygen,
> > - enhanced API description,
>
> There was some hiccup from Red Hat SMTP when sending the series..
> hopefully my resending of the second half of the patches is good
> enough.
> At least, patchwork looks happy.
The sxe drivers got merged and pulled in main, so I'll prepare a v4
and send it probably tomorrow.
Comments still welcome.
--
David Marchand
^ permalink raw reply
* [PATCH v6] mempool: improve cache behaviour and performance
From: Morten Brørup @ 2026-05-26 14:00 UTC (permalink / raw)
To: dev, Andrew Rybchenko, Bruce Richardson, Jingjing Wu,
Praveen Shetty, Hemant Agrawal, Sachin Saxena
Cc: Morten Brørup
In-Reply-To: <20260408141315.904381-1-mb@smartsharesystems.com>
This patch refactors the mempool cache to eliminate some unexpected
behaviour and reduce the mempool cache miss rate.
1.
The actual cache size was 1.5 times the cache size specified at run-time
mempool creation.
This was obviously not expected by application developers.
2.
In get operations, the check for when to use the cache as bounce buffer
did not respect the run-time configured cache size,
but compared to the build time maximum possible cache size
(RTE_MEMPOOL_CACHE_MAX_SIZE, default 512).
E.g. with a configured cache size of 32 objects, getting 256 objects
would first fetch 32 + 256 = 288 objects into the cache,
and then move the 256 objects from the cache to the destination memory,
instead of fetching the 256 objects directly to the destination memory.
This had a performance cost.
However, this is unlikely to occur in real applications, so it is not
important in itself.
3.
When putting objects into a mempool, and the mempool cache did not have
free space for so many objects,
the cache was flushed completely, and the new objects were then put into
the cache.
I.e. the cache drain level was zero.
This (complete cache flush) meant that a subsequent get operation (with
the same number of objects) completely emptied the cache,
so another subsequent get operation required replenishing the cache.
Similarly,
When getting objects from a mempool, and the mempool cache did not hold so
many objects,
the cache was replenished to cache->size + remaining objects,
and then (the remaining part of) the requested objects were fetched via
the cache,
which left the cache filled (to cache->size) at completion.
I.e. the cache refill level was cache->size (plus some, depending on
request size).
(1) was improved by generally comparing to cache->size instead of
cache->flushthresh, when considering the capacity of the cache.
The cache->flushthresh field is kept for API/ABI compatibility purposes,
and initialized to cache->size instead of cache->size * 1.5.
(2) was improved by generally comparing to cache->size / 2 instead of
RTE_MEMPOOL_CACHE_MAX_SIZE, when checking the bounce buffer limit.
(3) was improved by flushing and replenishing the cache by half its size,
so a flush/refill can be followed randomly by get or put requests.
This also reduced the number of objects in each flush/refill operation.
As a consequence of these changes, the size of the array holding the
objects in the cache (cache->objs[]) no longer needs to be
2 * RTE_MEMPOOL_CACHE_MAX_SIZE, and can be reduced to
RTE_MEMPOOL_CACHE_MAX_SIZE at an API/ABI breaking release.
Performance data:
With a real WAN Optimization application, where the number of allocated
packets varies (as they are held in e.g. shaper queues), the mempool
cache miss rate dropped from ca. 1/20 objects to ca. 1/48 objects.
This was deployed in production at an ISP, and using an effective cache
size of 384 objects.
Bugzilla ID: 1027
Fixes: ea5dd2744b90 ("mempool: cache optimisations")
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
Depends-on: patch-163181 ("net/intel: do not bypass mbuf lib for mbuf fast-free")
---
v6:
* Moved driver changes out as separate patches, for easier review. (Bruce)
Tests using the Intel idpf PMD in AVX512 mode may fail with this patch.
* Reverted a small code comment change. The original was better. (Bruce)
* Reverted rte_mempool_create() description requiring the cache_size to be
an even number. There is no such requirement.
v5:
* Flush the cache from the bottom, where objects are colder, and move down
the remaining objects, which are hotter.
* In the Intel idpf PMD, move up the hot objects in the cache and refill
with cold objects at the bottom.
v4:
* Added Bugzilla ID.
* Added Fixes tag. For reference only.
* Moved fast-free related update of Intel common driver out as a separate
patch, and depend on that patch.
* Omitted unrelated changes to the Intel idpf AVX512 driver, specifically
fixing an indentation and adding mbuf instrumentation.
* Omitted unrelated changes to the mempool library, specifically adding
__rte_restrict and changing a couple of comments to proper sentences.
* Please checkpatches by swapping operators in a couple of comparisons.
v3:
* Fixed my copy-paste bug in idpf_splitq_rearm().
v2:
* Fixed issue found by abidiff:
Reverted cache objects array size reduction. Added a note instead.
* Added missing mbuf instrumentation to the Intel idpf AVX512 driver.
* Updated idpf_splitq_rearm() like idpf_singleq_rearm().
* Added a few more __rte_assume(). (Inspired by AI review)
* Updated NXP dpaa and dpaa2 mempool drivers to not set mempool cache
flush threshold.
* Added release notes.
* Added deprecation notes.
---
doc/guides/rel_notes/deprecation.rst | 7 +++
doc/guides/rel_notes/release_26_07.rst | 11 +++++
lib/mempool/rte_mempool.c | 14 +-----
lib/mempool/rte_mempool.h | 66 ++++++++++++++++----------
4 files changed, 61 insertions(+), 37 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 35c9b4e06c..40760fffbb 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -154,3 +154,10 @@ Deprecation Notices
* bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in
``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.
Those API functions are used internally by DPDK core and netvsc PMD.
+
+* mempool: The ``flushthresh`` field in ``struct rte_mempool_cache``
+ is obsolete, and will be removed in DPDK 26.11.
+
+* mempool: The object array in ``struct rte_mempool_cache`` is oversize by
+ factor two, and will be reduced to ``RTE_MEMPOOL_CACHE_MAX_SIZE`` in
+ DPDK 26.11.
diff --git a/doc/guides/rel_notes/release_26_07.rst b/doc/guides/rel_notes/release_26_07.rst
index 6f43d9b61c..3f793f504a 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -63,6 +63,17 @@ New Features
``rte_eal_init`` and the application is responsible for probing each device,
* ``--auto-probing`` enables the initial bus probing, which is the current default behavior.
+* **Changed effective size of mempool cache.**
+
+ * The effective size of a mempool cache was changed to match the specified size at mempool creation; the effective size was previously 50 % larger than requested.
+ * The ``flushthresh`` field of the ``struct rte_mempool_cache`` became obsolete, but was kept for API/ABI compatibility purposes.
+ * The effective size of the ``objs`` array in the ``struct rte_mempool_cache`` was reduced to ``RTE_MEMPOOL_CACHE_MAX_SIZE``, but its size was kept for API/ABI compatibility purposes.
+
+* **Improved mempool cache flush/refill algorithm.**
+
+ The mempool cache flush/refill algorithm was improved, to reduce the mempool cache miss rate for most application types.
+ Applications where each lcore only puts or gets to a mempool, e.g. pipelined applications where ethdev Rx and Tx run on separate lcores, should adapt to the new algorithm by doubling their configured mempool cache size, to avoid doubling their mempool cache miss rate.
+
* **Updated PCAP ethernet driver.**
* Added support for VLAN insertion and stripping.
diff --git a/lib/mempool/rte_mempool.c b/lib/mempool/rte_mempool.c
index 3042d94c14..805b52cc58 100644
--- a/lib/mempool/rte_mempool.c
+++ b/lib/mempool/rte_mempool.c
@@ -52,11 +52,6 @@ static void
mempool_event_callback_invoke(enum rte_mempool_event event,
struct rte_mempool *mp);
-/* Note: avoid using floating point since that compiler
- * may not think that is constant.
- */
-#define CALC_CACHE_FLUSHTHRESH(c) (((c) * 3) / 2)
-
#if defined(RTE_ARCH_X86)
/*
* return the greatest common divisor between a and b (fast algorithm)
@@ -757,13 +752,8 @@ rte_mempool_free(struct rte_mempool *mp)
static void
mempool_cache_init(struct rte_mempool_cache *cache, uint32_t size)
{
- /* Check that cache have enough space for flush threshold */
- RTE_BUILD_BUG_ON(CALC_CACHE_FLUSHTHRESH(RTE_MEMPOOL_CACHE_MAX_SIZE) >
- RTE_SIZEOF_FIELD(struct rte_mempool_cache, objs) /
- RTE_SIZEOF_FIELD(struct rte_mempool_cache, objs[0]));
-
cache->size = size;
- cache->flushthresh = CALC_CACHE_FLUSHTHRESH(size);
+ cache->flushthresh = size; /* Obsolete; for API/ABI compatibility purposes only */
cache->len = 0;
}
@@ -850,7 +840,7 @@ rte_mempool_create_empty(const char *name, unsigned n, unsigned elt_size,
/* asked cache too big */
if (cache_size > RTE_MEMPOOL_CACHE_MAX_SIZE ||
- CALC_CACHE_FLUSHTHRESH(cache_size) > n) {
+ cache_size > n) {
rte_errno = EINVAL;
return NULL;
}
diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h
index 2e54fc4466..cd0f229b59 100644
--- a/lib/mempool/rte_mempool.h
+++ b/lib/mempool/rte_mempool.h
@@ -89,7 +89,7 @@ struct __rte_cache_aligned rte_mempool_debug_stats {
*/
struct __rte_cache_aligned rte_mempool_cache {
uint32_t size; /**< Size of the cache */
- uint32_t flushthresh; /**< Threshold before we flush excess elements */
+ uint32_t flushthresh; /**< Obsolete; for API/ABI compatibility purposes only */
uint32_t len; /**< Current cache count */
#ifdef RTE_LIBRTE_MEMPOOL_STATS
uint32_t unused;
@@ -107,8 +107,10 @@ struct __rte_cache_aligned rte_mempool_cache {
/**
* Cache objects
*
- * Cache is allocated to this size to allow it to overflow in certain
- * cases to avoid needless emptying of cache.
+ * Note:
+ * Cache is allocated at double size for API/ABI compatibility purposes only.
+ * When reducing its size at an API/ABI breaking release,
+ * remember to add a cache guard after it.
*/
alignas(RTE_CACHE_LINE_SIZE) void *objs[RTE_MEMPOOL_CACHE_MAX_SIZE * 2];
};
@@ -1047,11 +1049,16 @@ rte_mempool_free(struct rte_mempool *mp);
* If cache_size is non-zero, the rte_mempool library will try to
* limit the accesses to the common lockless pool, by maintaining a
* per-lcore object cache. This argument must be lower or equal to
- * RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.
+ * RTE_MEMPOOL_CACHE_MAX_SIZE and n.
* The access to the per-lcore table is of course
* faster than the multi-producer/consumer pool. The cache can be
* disabled if the cache_size argument is set to 0; it can be useful to
* avoid losing objects in cache.
+ * Note:
+ * Mempool put/get requests of more than cache_size / 2 objects may be
+ * partially or fully served directly by the multi-producer/consumer
+ * pool, to avoid the overhead of copying the objects twice (instead of
+ * once) when using the cache as a bounce buffer.
* @param private_data_size
* The size of the private data appended after the mempool
* structure. This is useful for storing some private data after the
@@ -1390,22 +1397,30 @@ rte_mempool_do_generic_put(struct rte_mempool *mp, void * const *obj_table,
RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_bulk, 1);
RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_objs, n);
- __rte_assume(cache->flushthresh <= RTE_MEMPOOL_CACHE_MAX_SIZE * 2);
- __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE * 2);
- __rte_assume(cache->len <= cache->flushthresh);
- if (likely(cache->len + n <= cache->flushthresh)) {
+ __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
+ __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
+ __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
+ __rte_assume(cache->len <= cache->size);
+ if (likely(cache->len + n <= cache->size)) {
/* Sufficient room in the cache for the objects. */
cache_objs = &cache->objs[cache->len];
cache->len += n;
- } else if (n <= cache->flushthresh) {
+ } else if (n <= cache->size / 2) {
/*
- * The cache is big enough for the objects, but - as detected by
- * the comparison above - has insufficient room for them.
- * Flush the cache to make room for the objects.
+ * The number of objects is within the cache bounce buffer limit,
+ * but - as detected by the comparison above - the cache has
+ * insufficient room for them.
+ * Flush the cache to the backend to make room for the objects;
+ * flush (size / 2) objects from the bottom of the cache, where
+ * objects are less hot, and move down the remaining objects, which
+ * are more hot, from the upper half of the cache.
*/
- cache_objs = &cache->objs[0];
- rte_mempool_ops_enqueue_bulk(mp, cache_objs, cache->len);
- cache->len = n;
+ __rte_assume(cache->len > cache->size / 2);
+ rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0], cache->size / 2);
+ rte_memcpy(&cache->objs[0], &cache->objs[cache->size / 2],
+ sizeof(void *) * (cache->len - cache->size / 2));
+ cache_objs = &cache->objs[cache->len - cache->size / 2];
+ cache->len = cache->len - cache->size / 2 + n;
} else {
/* The request itself is too big for the cache. */
goto driver_enqueue_stats_incremented;
@@ -1524,7 +1539,7 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
/* The cache is a stack, so copy will be in reverse order. */
cache_objs = &cache->objs[cache->len];
- __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE * 2);
+ __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
if (likely(n <= cache->len)) {
/* The entire request can be satisfied from the cache. */
RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk, 1);
@@ -1548,13 +1563,13 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
for (index = 0; index < len; index++)
*obj_table++ = *--cache_objs;
- /* Dequeue below would overflow mem allocated for cache? */
- if (unlikely(remaining > RTE_MEMPOOL_CACHE_MAX_SIZE))
+ /* Dequeue below would exceed the cache bounce buffer limit? */
+ __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
+ if (unlikely(remaining > cache->size / 2))
goto driver_dequeue;
- /* Fill the cache from the backend; fetch size + remaining objects. */
- ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs,
- cache->size + remaining);
+ /* Fill the cache from the backend; fetch (size / 2) objects. */
+ ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, cache->size / 2);
if (unlikely(ret < 0)) {
/*
* We are buffer constrained, and not able to fetch all that.
@@ -1568,10 +1583,11 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk, 1);
RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_objs, n);
- __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
- __rte_assume(remaining <= RTE_MEMPOOL_CACHE_MAX_SIZE);
- cache_objs = &cache->objs[cache->size + remaining];
- cache->len = cache->size;
+ __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
+ __rte_assume(remaining <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
+ __rte_assume(remaining <= cache->size / 2);
+ cache_objs = &cache->objs[cache->size / 2];
+ cache->len = cache->size / 2 - remaining;
for (index = 0; index < remaining; index++)
*obj_table++ = *--cache_objs;
--
2.43.0
^ permalink raw reply related
* Re: [PATCH v19 00/11]net/sxe2: fix logic errors and address feedback
From: Thomas Monjalon @ 2026-05-26 14:13 UTC (permalink / raw)
To: Jie Liu; +Cc: dev, stephen, David Marchand
In-Reply-To: <9XBXAO5XSSSHFRFv8zy4yQ@monjalon.net>
21/05/2026 17:16, Thomas Monjalon:
> Hello,
>
> 20/05/2026 04:17, liujie5@linkdatatechnology.com:
> > common/sxe2: add sxe2 basic structures
>
> Are you planning to add a crypto or compress driver?
> This is usually the reason to have a common library.
> If you don't intend to share some code between different driver,
> then you should not have a common library.
We are curious about your plans for sxe2.
Please, could you reply?
Will you add another driver class to sxe2?
^ permalink raw reply
* Re: [PATCH v2 2/6] eal/memory: remove per-list segment and memory limits
From: Bruce Richardson @ 2026-05-26 14:57 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev
In-Reply-To: <c5fa90b0700411d63652a5fae547996699da4b0b.1773417833.git.anatoly.burakov@intel.com>
On Fri, Mar 13, 2026 at 04:06:33PM +0000, Anatoly Burakov wrote:
> Initially, the dynamic memory mode has used multiple segment lists for
> backing of different memory types, with the motivation being that it should
> be easier for secondary processes to map many smaller segments than fewer
> but larger ones, but in practice this does not seem to make any difference
> for 64-bit platforms, as there's usually plenty of address space.
>
I think it's worth clarifying here that each list corresponds to a VA
address space block. The term segment is also a little confusing here,
since the EAL memseg used in segment lists is now a page IIRC, but the
"smaller segments" for secondaries are blocks of VA space, right?
> To reduce the amount of complexity in how memory segment lists are handled,
> collapse the multi-list logic to always use single segment list.
>
> That does not mean that all memory types will always get one segment - in
> some cases (e.g. 32-bit) we may not be able to allocate enough contiguous
> VA spaces to fit entire memory type into one list, in which case the number
> of memseg lists for that type will be more than one. It is more about
> lifting the upper limit on how many segment lists can a type have. If we
> end up blowing up our number of segment lists so much that we exceed a
> very generous default maximum memseg lists number then the user has bigger
> problems to address.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Some comments inline below. Nothing major, just some style suggestions.
> ---
> config/rte_config.h | 2 -
> .../prog_guide/env_abstraction_layer.rst | 4 -
> lib/eal/common/eal_common_dynmem.c | 110 +++++-------------
> lib/eal/common/eal_common_memory.c | 6 +-
> lib/eal/common/eal_filesystem.h | 13 +++
> lib/eal/common/eal_private.h | 6 +-
> lib/eal/freebsd/eal_memory.c | 75 ++++--------
> lib/eal/linux/eal_memalloc.c | 4 +-
> lib/eal/linux/eal_memory.c | 88 ++++++--------
> 9 files changed, 107 insertions(+), 201 deletions(-)
>
> diff --git a/config/rte_config.h b/config/rte_config.h
> index a2609fa403..0447cdf2ad 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -43,8 +43,6 @@
> #define RTE_MAX_HEAPS 32
> #define RTE_MAX_LCORE_VAR 131072
> #define RTE_MAX_MEMSEG_LISTS 128
> -#define RTE_MAX_MEMSEG_PER_LIST 8192
> -#define RTE_MAX_MEM_MB_PER_LIST 32768
> #define RTE_MAX_MEMSEG_PER_TYPE 32768
> #define RTE_MAX_MEM_MB_PER_TYPE 65536
> #define RTE_MAX_TAILQ 32
> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
> index d716895c1d..04368a3950 100644
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> @@ -204,10 +204,6 @@ of virtual memory being preallocated at startup by editing the following config
> variables:
>
> * ``RTE_MAX_MEMSEG_LISTS`` controls how many segment lists can DPDK have
> -* ``RTE_MAX_MEM_MB_PER_LIST`` controls how much megabytes of memory each
> - segment list can address
> -* ``RTE_MAX_MEMSEG_PER_LIST`` controls how many segments each segment list
> - can have
> * ``RTE_MAX_MEMSEG_PER_TYPE`` controls how many segments each memory type
> can have (where "type" is defined as "page size + NUMA node" combination)
> * ``RTE_MAX_MEM_MB_PER_TYPE`` controls how much megabytes of memory each
> diff --git a/lib/eal/common/eal_common_dynmem.c b/lib/eal/common/eal_common_dynmem.c
> index 8f51d6dd4a..ef0270cc30 100644
> --- a/lib/eal/common/eal_common_dynmem.c
> +++ b/lib/eal/common/eal_common_dynmem.c
> @@ -24,11 +24,10 @@ eal_dynmem_memseg_lists_init(void)
> struct memtype {
> uint64_t page_sz;
> int socket_id;
> - } *memtypes = NULL;
> + } memtypes[RTE_MAX_MEMSEG_LISTS] = {0};
> int i, hpi_idx, msl_idx, ret = -1; /* fail unless told to succeed */
> struct rte_memseg_list *msl;
> uint64_t max_mem, max_mem_per_type;
> - unsigned int max_seglists_per_type;
> unsigned int n_memtypes, cur_type;
> struct internal_config *internal_conf =
> eal_get_internal_configuration();
> @@ -45,8 +44,7 @@ eal_dynmem_memseg_lists_init(void)
> *
> * deciding amount of memory going towards each memory type is a
> * balancing act between maximum segments per type, maximum memory per
> - * type, and number of detected NUMA nodes. the goal is to make sure
> - * each memory type gets at least one memseg list.
> + * type, and number of detected NUMA nodes.
> *
> * the total amount of memory is limited by RTE_MAX_MEM_MB value.
> *
> @@ -57,26 +55,18 @@ eal_dynmem_memseg_lists_init(void)
> * smaller page sizes, it can take hundreds of thousands of segments to
> * reach the above specified per-type memory limits.
> *
> - * additionally, each type may have multiple memseg lists associated
> - * with it, each limited by either RTE_MAX_MEM_MB_PER_LIST for bigger
> - * page sizes, or RTE_MAX_MEMSEG_PER_LIST segments for smaller ones.
> - *
> - * the number of memseg lists per type is decided based on the above
> - * limits, and also taking number of detected NUMA nodes, to make sure
> - * that we don't run out of memseg lists before we populate all NUMA
> - * nodes with memory.
> - *
> - * we do this in three stages. first, we collect the number of types.
> - * then, we figure out memory constraints and populate the list of
> - * would-be memseg lists. then, we go ahead and allocate the memseg
> - * lists.
> + * each memory type is allotted a single memseg list. the size of that
> + * list is calculated here to respect the per-type memory and segment
> + * limits that apply.
> */
>
> - /* create space for mem types */
> + /* maximum number of memtypes we're ever going to get */
> n_memtypes = internal_conf->num_hugepage_sizes * rte_socket_count();
> - memtypes = calloc(n_memtypes, sizeof(*memtypes));
> - if (memtypes == NULL) {
> - EAL_LOG(ERR, "Cannot allocate space for memory types");
> +
> + /* can we fit all memtypes into the memseg lists? */
> + if (n_memtypes > RTE_MAX_MEMSEG_LISTS) {
> + EAL_LOG(ERR, "Too many memory types detected: %u. Please increase "
> + "RTE_MAX_MEMSEG_LISTS in configuration.", n_memtypes);
> return -1;
Don't split log messages across lines. Go over the 100 char limit if
necessary to avoid splits.
> }
>
> @@ -113,91 +103,49 @@ eal_dynmem_memseg_lists_init(void)
> max_mem = (uint64_t)RTE_MAX_MEM_MB << 20;
> max_mem_per_type = RTE_MIN((uint64_t)RTE_MAX_MEM_MB_PER_TYPE << 20,
> max_mem / n_memtypes);
> - /*
> - * limit maximum number of segment lists per type to ensure there's
> - * space for memseg lists for all NUMA nodes with all page sizes
> - */
> - max_seglists_per_type = RTE_MAX_MEMSEG_LISTS / n_memtypes;
> -
> - if (max_seglists_per_type == 0) {
> - EAL_LOG(ERR, "Cannot accommodate all memory types, please increase RTE_MAX_MEMSEG_LISTS");
> - goto out;
> - }
>
> /* go through all mem types and create segment lists */
> msl_idx = 0;
> for (cur_type = 0; cur_type < n_memtypes; cur_type++) {
> - unsigned int cur_seglist, n_seglists, n_segs;
> - unsigned int max_segs_per_type, max_segs_per_list;
> + unsigned int n_segs;
> struct memtype *type = &memtypes[cur_type];
> - uint64_t max_mem_per_list, pagesz;
> + uint64_t pagesz;
> int socket_id;
>
> pagesz = type->page_sz;
> socket_id = type->socket_id;
>
> /*
> - * we need to create segment lists for this type. we must take
> + * we need to create a segment list for this type. we must take
> * into account the following things:
> *
> - * 1. total amount of memory we can use for this memory type
> - * 2. total amount of memory per memseg list allowed
> + * 1. total amount of memory to use for this memory type
> + * 2. total amount of memory allowed per type
> * 3. number of segments needed to fit the amount of memory
> * 4. number of segments allowed per type
> - * 5. number of segments allowed per memseg list
> - * 6. number of memseg lists we are allowed to take up
> */
> + n_segs = max_mem_per_type / pagesz;
> + n_segs = RTE_MIN(n_segs, (unsigned int)RTE_MAX_MEMSEG_PER_TYPE);
>
> - /* calculate how much segments we will need in total */
> - max_segs_per_type = max_mem_per_type / pagesz;
> - /* limit number of segments to maximum allowed per type */
> - max_segs_per_type = RTE_MIN(max_segs_per_type,
> - (unsigned int)RTE_MAX_MEMSEG_PER_TYPE);
> - /* limit number of segments to maximum allowed per list */
> - max_segs_per_list = RTE_MIN(max_segs_per_type,
> - (unsigned int)RTE_MAX_MEMSEG_PER_LIST);
> + EAL_LOG(DEBUG, "Creating segment list: "
> + "n_segs:%u socket_id:%i hugepage_sz:%" PRIu64,
> + n_segs, socket_id, pagesz);
>
Again, better not to split this message.
> - /* calculate how much memory we can have per segment list */
> - max_mem_per_list = RTE_MIN(max_segs_per_list * pagesz,
> - (uint64_t)RTE_MAX_MEM_MB_PER_LIST << 20);
> + msl = &mcfg->memsegs[msl_idx];
>
> - /* calculate how many segments each segment list will have */
> - n_segs = RTE_MIN(max_segs_per_list, max_mem_per_list / pagesz);
> + if (eal_memseg_list_init(msl, pagesz, n_segs, socket_id,
> + msl_idx, true))
This can actually fit on one line.
> + goto out;
>
> - /* calculate how many segment lists we can have */
> - n_seglists = RTE_MIN(max_segs_per_type / n_segs,
> - max_mem_per_type / max_mem_per_list);
> -
> - /* limit number of segment lists according to our maximum */
> - n_seglists = RTE_MIN(n_seglists, max_seglists_per_type);
> -
> - EAL_LOG(DEBUG, "Creating %i segment lists: "
> - "n_segs:%i socket_id:%i hugepage_sz:%" PRIu64,
> - n_seglists, n_segs, socket_id, pagesz);
> -
> - /* create all segment lists */
> - for (cur_seglist = 0; cur_seglist < n_seglists; cur_seglist++) {
> - if (msl_idx >= RTE_MAX_MEMSEG_LISTS) {
> - EAL_LOG(ERR,
> - "No more space in memseg lists, please increase RTE_MAX_MEMSEG_LISTS");
> - goto out;
> - }
> - msl = &mcfg->memsegs[msl_idx++];
> -
> - if (eal_memseg_list_init(msl, pagesz, n_segs,
> - socket_id, cur_seglist, true))
> - goto out;
> -
> - if (eal_memseg_list_alloc(msl, 0)) {
> - EAL_LOG(ERR, "Cannot allocate VA space for memseg list");
> - goto out;
> - }
> + if (eal_memseg_list_alloc(msl, 0)) {
> + EAL_LOG(ERR, "Cannot allocate VA space for memseg list");
> + goto out;
> }
> + msl_idx++;
> }
> /* we're successful */
> ret = 0;
> out:
> - free(memtypes);
> return ret;
> }
>
> diff --git a/lib/eal/common/eal_common_memory.c b/lib/eal/common/eal_common_memory.c
> index dccf9406c5..b9388021ff 100644
> --- a/lib/eal/common/eal_common_memory.c
> +++ b/lib/eal/common/eal_common_memory.c
> @@ -228,12 +228,12 @@ eal_memseg_list_init_named(struct rte_memseg_list *msl, const char *name,
>
> int
> eal_memseg_list_init(struct rte_memseg_list *msl, uint64_t page_sz,
> - int n_segs, int socket_id, int type_msl_idx, bool heap)
> + int n_segs, int socket_id, int msl_idx, bool heap)
> {
> char name[RTE_FBARRAY_NAME_LEN];
>
> - snprintf(name, sizeof(name), MEMSEG_LIST_FMT, page_sz >> 10, socket_id,
> - type_msl_idx);
> + snprintf(name, sizeof(name), MEMSEG_LIST_FMT,
> + page_sz >> 10, socket_id, msl_idx);
>
Fits on single line.
> return eal_memseg_list_init_named(
> msl, name, page_sz, n_segs, socket_id, heap);
> diff --git a/lib/eal/common/eal_filesystem.h b/lib/eal/common/eal_filesystem.h
> index 6b99d22160..2d22b52e76 100644
> --- a/lib/eal/common/eal_filesystem.h
> +++ b/lib/eal/common/eal_filesystem.h
> @@ -114,6 +114,19 @@ eal_get_hugefile_path(char *buffer, size_t buflen, const char *hugedir, int f_id
> return buffer;
> }
>
> +#define HUGEFILE_FMT_LIST_SEG "%s/%smap_%u_%u"
> +static inline __rte_warn_unused_result const char *
> +eal_get_hugefile_list_seg_path(char *buffer, size_t buflen,
> + const char *hugedir, unsigned int list_idx, unsigned int seg_idx)
> +{
> + if (snprintf(buffer, buflen, HUGEFILE_FMT_LIST_SEG,
> + hugedir, eal_get_hugefile_prefix(), list_idx, seg_idx)
> + >= (int)buflen)
Can fit in two lines not three, e.g. move up the ">= (int)buflen".
> + return NULL;
> + else
> + return buffer;
While I know it's a copy of the previous function in the file, the "else"
is really unnecessary.
> +}
> +
> /** define the default filename prefix for the %s values above */
> #define HUGEFILE_PREFIX_DEFAULT "rte"
>
> diff --git a/lib/eal/common/eal_private.h b/lib/eal/common/eal_private.h
> index e032dd10c9..70f7b46699 100644
> --- a/lib/eal/common/eal_private.h
> +++ b/lib/eal/common/eal_private.h
> @@ -299,14 +299,14 @@ eal_memseg_list_init_named(struct rte_memseg_list *msl, const char *name,
> * Initialize memory segment list and create its backing storage
> * with a name corresponding to MSL parameters.
> *
> - * @param type_msl_idx
> - * Index of the MSL among other MSLs of the same socket and page size.
> + * @param msl_idx
> + * Index of the MSL in memsegs array.
> *
> * @see eal_memseg_list_init_named for remaining parameters description.
> */
> int
> eal_memseg_list_init(struct rte_memseg_list *msl, uint64_t page_sz,
> - int n_segs, int socket_id, int type_msl_idx, bool heap);
> + int n_segs, int socket_id, int msl_idx, bool heap);
>
> /**
> * Reserve VA space for a memory segment list
> diff --git a/lib/eal/freebsd/eal_memory.c b/lib/eal/freebsd/eal_memory.c
> index cd608db9f9..3eb5d193ec 100644
> --- a/lib/eal/freebsd/eal_memory.c
> +++ b/lib/eal/freebsd/eal_memory.c
> @@ -190,8 +190,8 @@ rte_eal_hugepage_init(void)
> break;
> }
> if (msl_idx == RTE_MAX_MEMSEG_LISTS) {
> - EAL_LOG(ERR, "Could not find space for memseg. Please increase RTE_MAX_MEMSEG_PER_LIST "
> - "RTE_MAX_MEMSEG_PER_TYPE and/or RTE_MAX_MEM_MB_PER_TYPE in configuration.");
> + EAL_LOG(ERR,
> + "Could not find suitable space for memseg in existing memseg lists");
> return -1;
> }
> arr = &msl->memseg_arr;
> @@ -320,23 +320,6 @@ rte_eal_using_phys_addrs(void)
> return 0;
> }
>
> -static uint64_t
> -get_mem_amount(uint64_t page_sz, uint64_t max_mem)
> -{
> - uint64_t area_sz, max_pages;
> -
> - /* limit to RTE_MAX_MEMSEG_PER_LIST pages or RTE_MAX_MEM_MB_PER_LIST */
> - max_pages = RTE_MAX_MEMSEG_PER_LIST;
> - max_mem = RTE_MIN((uint64_t)RTE_MAX_MEM_MB_PER_LIST << 20, max_mem);
> -
> - area_sz = RTE_MIN(page_sz * max_pages, max_mem);
> -
> - /* make sure the list isn't smaller than the page size */
> - area_sz = RTE_MAX(area_sz, page_sz);
> -
> - return RTE_ALIGN(area_sz, page_sz);
> -}
> -
> static int
> memseg_list_alloc(struct rte_memseg_list *msl)
> {
> @@ -380,9 +363,10 @@ memseg_primary_init(void)
> hpi_idx++) {
> uint64_t max_type_mem, total_type_mem = 0;
> uint64_t avail_mem;
> - int type_msl_idx, max_segs, avail_segs, total_segs = 0;
> + unsigned int avail_segs;
> struct hugepage_info *hpi;
> uint64_t hugepage_sz;
> + unsigned int n_segs;
>
> hpi = &internal_conf->hugepage_info[hpi_idx];
> hugepage_sz = hpi->hugepage_sz;
> @@ -396,7 +380,6 @@ memseg_primary_init(void)
> /* first, calculate theoretical limits according to config */
> max_type_mem = RTE_MIN(max_mem - total_mem,
> (uint64_t)RTE_MAX_MEM_MB_PER_TYPE << 20);
> - max_segs = RTE_MAX_MEMSEG_PER_TYPE;
>
> /* now, limit all of that to whatever will actually be
> * available to us, because without dynamic allocation support,
> @@ -412,42 +395,30 @@ memseg_primary_init(void)
> avail_mem = avail_segs * hugepage_sz;
>
> max_type_mem = RTE_MIN(avail_mem, max_type_mem);
> - max_segs = RTE_MIN(avail_segs, max_segs);
> -
> - type_msl_idx = 0;
> - while (total_type_mem < max_type_mem &&
> - total_segs < max_segs) {
> - uint64_t cur_max_mem, cur_mem;
> - unsigned int n_segs;
> -
> - if (msl_idx >= RTE_MAX_MEMSEG_LISTS) {
> - EAL_LOG(ERR,
> - "No more space in memseg lists, please increase RTE_MAX_MEMSEG_LISTS");
> - return -1;
> - }
> -
> - msl = &mcfg->memsegs[msl_idx++];
> -
> - cur_max_mem = max_type_mem - total_type_mem;
> -
> - cur_mem = get_mem_amount(hugepage_sz,
> - cur_max_mem);
> - n_segs = cur_mem / hugepage_sz;
> + n_segs = max_type_mem / hugepage_sz;
> + if (n_segs == 0)
> + continue;
> +
> + if (msl_idx >= RTE_MAX_MEMSEG_LISTS) {
> + EAL_LOG(ERR,
> + "No more space in memseg lists, please increase RTE_MAX_MEMSEG_LISTS");
Not sure it's worth splitting this across two lines. You only save 3 or 4
chars. :-)
> + return -1;
> + }
>
> - if (eal_memseg_list_init(msl, hugepage_sz, n_segs,
> - 0, type_msl_idx, false))
> - return -1;
> + msl = &mcfg->memsegs[msl_idx];
>
> - total_segs += msl->memseg_arr.len;
> - total_type_mem = total_segs * hugepage_sz;
> - type_msl_idx++;
> + if (eal_memseg_list_init(msl, hugepage_sz, n_segs,
> + 0, msl_idx, false))
Fits on one line.
> + return -1;
>
> - if (memseg_list_alloc(msl)) {
> - EAL_LOG(ERR, "Cannot allocate VA space for memseg list");
> - return -1;
> - }
> + total_type_mem = n_segs * hugepage_sz;
> + if (memseg_list_alloc(msl)) {
> + EAL_LOG(ERR, "Cannot allocate VA space for memseg list");
> + return -1;
> }
> +
> total_mem += total_type_mem;
> + msl_idx++;
> }
> return 0;
> }
> diff --git a/lib/eal/linux/eal_memalloc.c b/lib/eal/linux/eal_memalloc.c
> index a39bc31c7b..2227b1c52b 100644
> --- a/lib/eal/linux/eal_memalloc.c
> +++ b/lib/eal/linux/eal_memalloc.c
> @@ -282,8 +282,8 @@ get_seg_fd(char *path, int buflen, struct hugepage_info *hi,
> huge_path = eal_get_hugefile_path(path, buflen, hi->hugedir, list_idx);
> } else {
> out_fd = &fd_list[list_idx].fds[seg_idx];
> - huge_path = eal_get_hugefile_path(path, buflen, hi->hugedir,
> - list_idx * RTE_MAX_MEMSEG_PER_LIST + seg_idx);
> + huge_path = eal_get_hugefile_list_seg_path(path, buflen,
> + hi->hugedir, list_idx, seg_idx);
> }
> if (huge_path == NULL) {
> EAL_LOG(DEBUG, "%s(): hugefile path truncated: '%s'",
> diff --git a/lib/eal/linux/eal_memory.c b/lib/eal/linux/eal_memory.c
> index bf783e3c76..691d8eb3cc 100644
> --- a/lib/eal/linux/eal_memory.c
> +++ b/lib/eal/linux/eal_memory.c
> @@ -740,8 +740,8 @@ remap_segment(struct hugepage_file *hugepages, int seg_start, int seg_end)
> break;
> }
> if (msl_idx == RTE_MAX_MEMSEG_LISTS) {
> - EAL_LOG(ERR, "Could not find space for memseg. Please increase RTE_MAX_MEMSEG_PER_LIST "
> - "RTE_MAX_MEMSEG_PER_TYPE and/or RTE_MAX_MEM_MB_PER_TYPE in configuration.");
> + EAL_LOG(ERR,
> + "Could not find suitable space for memseg in existing memseg lists");
> return -1;
> }
>
> @@ -822,23 +822,6 @@ remap_segment(struct hugepage_file *hugepages, int seg_start, int seg_end)
> return seg_len;
> }
>
> -static uint64_t
> -get_mem_amount(uint64_t page_sz, uint64_t max_mem)
> -{
> - uint64_t area_sz, max_pages;
> -
> - /* limit to RTE_MAX_MEMSEG_PER_LIST pages or RTE_MAX_MEM_MB_PER_LIST */
> - max_pages = RTE_MAX_MEMSEG_PER_LIST;
> - max_mem = RTE_MIN((uint64_t)RTE_MAX_MEM_MB_PER_LIST << 20, max_mem);
> -
> - area_sz = RTE_MIN(page_sz * max_pages, max_mem);
> -
> - /* make sure the list isn't smaller than the page size */
> - area_sz = RTE_MAX(area_sz, page_sz);
> -
> - return RTE_ALIGN(area_sz, page_sz);
> -}
> -
> static int
> memseg_list_free(struct rte_memseg_list *msl)
> {
> @@ -1831,7 +1814,6 @@ memseg_primary_init_32(void)
> uint64_t max_pagesz_mem, cur_pagesz_mem = 0;
> uint64_t hugepage_sz;
> struct hugepage_info *hpi;
> - int type_msl_idx, max_segs, total_segs = 0;
>
> hpi = &internal_conf->hugepage_info[hpi_idx];
> hugepage_sz = hpi->hugepage_sz;
> @@ -1840,62 +1822,60 @@ memseg_primary_init_32(void)
> if (hpi->num_pages[socket_id] == 0)
> continue;
>
> - max_segs = RTE_MAX_MEMSEG_PER_TYPE;
> max_pagesz_mem = max_socket_mem - cur_socket_mem;
>
> /* make it multiple of page size */
> max_pagesz_mem = RTE_ALIGN_FLOOR(max_pagesz_mem,
> hugepage_sz);
>
> + if (max_pagesz_mem == 0)
> + continue;
> +
> EAL_LOG(DEBUG, "Attempting to preallocate "
> "%" PRIu64 "M on socket %i",
> max_pagesz_mem >> 20, socket_id);
>
> - type_msl_idx = 0;
> - while (cur_pagesz_mem < max_pagesz_mem &&
> - total_segs < max_segs) {
> - uint64_t cur_mem;
> + while (cur_pagesz_mem < max_pagesz_mem) {
> + uint64_t rem_mem;
> unsigned int n_segs;
>
> - if (msl_idx >= RTE_MAX_MEMSEG_LISTS) {
> - EAL_LOG(ERR,
> - "No more space in memseg lists, please increase RTE_MAX_MEMSEG_LISTS");
> - return -1;
> - }
> + rem_mem = max_pagesz_mem - cur_pagesz_mem;
> + n_segs = rem_mem / hugepage_sz;
>
> - msl = &mcfg->memsegs[msl_idx];
> + while (n_segs > 0) {
> + if (msl_idx >= RTE_MAX_MEMSEG_LISTS) {
> + EAL_LOG(ERR,
> + "No more space in memseg lists, please increase RTE_MAX_MEMSEG_LISTS");
> + return -1;
> + }
>
> - cur_mem = get_mem_amount(hugepage_sz,
> - max_pagesz_mem);
> - n_segs = cur_mem / hugepage_sz;
> + msl = &mcfg->memsegs[msl_idx];
>
> - if (eal_memseg_list_init(msl, hugepage_sz,
> - n_segs, socket_id, type_msl_idx,
> - true)) {
> - /* failing to allocate a memseg list is
> - * a serious error.
> - */
> - EAL_LOG(ERR, "Cannot allocate memseg list");
> - return -1;
> - }
> + if (eal_memseg_list_init(msl, hugepage_sz,
> + n_segs, socket_id, msl_idx, true) < 0) {
> + /* failing to allocate a memseg list is a serious error. */
> + EAL_LOG(ERR, "Cannot allocate memseg list");
> + return -1;
> + }
> +
> + if (eal_memseg_list_alloc(msl, 0) == 0)
> + break;
>
> - if (eal_memseg_list_alloc(msl, 0)) {
> - /* if we couldn't allocate VA space, we
> - * can try with smaller page sizes.
> - */
> - EAL_LOG(ERR, "Cannot allocate VA space for memseg list, retrying with different page size");
> - /* deallocate memseg list */
> if (memseg_list_free(msl))
> return -1;
> - break;
> +
> + EAL_LOG(DEBUG,
> + "Cannot allocate VA space for memseg list, retrying with smaller chunk");
> + n_segs /= 2;
> }
>
> - total_segs += msl->memseg_arr.len;
> - cur_pagesz_mem = total_segs * hugepage_sz;
> - type_msl_idx++;
> + if (n_segs == 0)
> + break;
> +
> + cur_pagesz_mem += (uint64_t)n_segs * hugepage_sz;
> + cur_socket_mem += (uint64_t)n_segs * hugepage_sz;
> msl_idx++;
> }
> - cur_socket_mem += cur_pagesz_mem;
> }
> if (cur_socket_mem == 0) {
> EAL_LOG(ERR, "Cannot allocate VA space on socket %u",
> --
> 2.47.3
>
^ permalink raw reply
* Re: [PATCH v2 3/6] eal/memory: allocate all VA space in one go
From: Bruce Richardson @ 2026-05-26 15:25 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev
In-Reply-To: <23100128428096f0179a8c64904449a95311bca6.1773417833.git.anatoly.burakov@intel.com>
On Fri, Mar 13, 2026 at 04:06:34PM +0000, Anatoly Burakov wrote:
> Instead of allocating VA space per memseg list in dynmem mode, allocate it
> all in one go, and then assign memseg lists portions of that space. In a
> similar way, for dynmem initialization in secondary processes, also attach
> all VA space in one go. Legacy/32-bit paths are untouched.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
As with previous patch, stylistically there are a few places where lines
are split unncessarily.
^ permalink raw reply
* DPDK Release Status Meeting 2026-05-26
From: Mcnamara, John @ 2026-05-26 15:47 UTC (permalink / raw)
To: dev@dpdk.org; +Cc: Thomas Monjalon, David Marchand
[-- Attachment #1: Type: text/plain, Size: 2979 bytes --]
Release status meeting minutes 2026-05-26
=========================================
Agenda:
- Release Dates
- Subtrees
- Roadmaps
- LTS
- Defects
- Opens
Participants:
- ARM [No]
- Broadcom
- Debian [No]
- Intel
- Marvell
- Nvidia
- Red Hat [No]
- Stephen Hemminger
Release Dates
-------------
The following are the proposed working dates for 27.03:
| Date | Milestone | Description |
|-----------------|------------------|---------------------------------|
| 30 April 2026 | RFC/v1 patches | Proposal deadline |
| 04 June 2026 | 26.07-rc1 | API freeze |
| 25 June 2026 | 26.07-rc2 | PMD features freeze |
| 02 July 2026 | 26.07-rc3 | Builtin apps features freeze |
| 9 July 2026 | 26.07-rc4 | Documentation ready |
| 16 July 2026 | 26.07.0 | Release |
See https://core.dpdk.org/roadmap/
Subtrees
--------
- next-net
- 69 patches pending for merge.
- 12 patches in review/waiting.
- next-net-intel
- 100 patches in queue.
- 12 patches in backlog + large patch for rte_flow rework.
- next-net-mlx
- No update.
- next-broadcom
- Nothing pushed yet.
- ~20 patches in backlog.
- next-net-mvl
- No update.
- next-eventdev
- No update.
- next-baseband
- No update.
- next-virtio
- Series for ASID under review.
- next-crypto
- Nothing pushed yet.
- ~10 patches in backlog.
- next-dts
- No update.
- main
- New version of BUS drivers refactoring series.
- Looking at EAL patches. Large patchset on atomic deprecations.
- Merging going on for RC1.
- RC1 deadline 4 June 2026
Other
-----
- None.
LTS
---
See also: https://core.dpdk.org/roadmap/#stable
LTS versions ongoing/released:
- 25.11.1 - Released + regression fix on 25.11.2.
- 24.11.5 - Released + regression fix on 24.11.6.
- 23.11.7 - Released.
Older releases:
- 20.11.10 - Will only be updated with CVE and critical fixes.
- 19.11.14 - Will only be updated with CVE and critical fixes.
- Distros
- Debian 13 contains DPDK v24.11
- Ubuntu 25.04 contains DPDK v24.11
- Ubuntu 24.04 LTS contains DPDK v23.11
- RHEL 9 contains DPDK 24.11
Defects
-------
- Bugzilla links, 'Bugs', added for hosted projects
- https://www.dpdk.org/hosted-projects/
DPDK Release Status Meetings
----------------------------
The DPDK Release Status Meeting is intended for DPDK Committers to discuss the
status of the main tree and sub-trees, and for project managers to track
progress or milestone dates.
The meeting occurs on every Tuesday at 14:30 DST over Jitsi on https://meet.jit.si/DPDK
You don't need an invite to join the meeting but if you want a calendar reminder just
send an email to "John McNamara <john.mcnamara@intel.com>" for the invite.
[-- Attachment #2: Type: text/html, Size: 20283 bytes --]
^ permalink raw reply
* RE: [PATCH v6] mempool: improve cache behaviour and performance
From: Morten Brørup @ 2026-05-26 16:00 UTC (permalink / raw)
To: dev, Andrew Rybchenko, Bruce Richardson, Jingjing Wu,
Praveen Shetty, Hemant Agrawal, Sachin Saxena
In-Reply-To: <20260526140000.175092-1-mb@smartsharesystems.com>
> From: Morten Brørup [mailto:mb@smartsharesystems.com]
> Sent: Tuesday, 26 May 2026 16.00
>
> This patch refactors the mempool cache to eliminate some unexpected
> behaviour and reduce the mempool cache miss rate.
>
> 1.
> The actual cache size was 1.5 times the cache size specified at run-
> time
> mempool creation.
> This was obviously not expected by application developers.
>
> 2.
> In get operations, the check for when to use the cache as bounce buffer
> did not respect the run-time configured cache size,
> but compared to the build time maximum possible cache size
> (RTE_MEMPOOL_CACHE_MAX_SIZE, default 512).
> E.g. with a configured cache size of 32 objects, getting 256 objects
> would first fetch 32 + 256 = 288 objects into the cache,
> and then move the 256 objects from the cache to the destination memory,
> instead of fetching the 256 objects directly to the destination memory.
> This had a performance cost.
> However, this is unlikely to occur in real applications, so it is not
> important in itself.
>
> 3.
> When putting objects into a mempool, and the mempool cache did not have
> free space for so many objects,
> the cache was flushed completely, and the new objects were then put
> into
> the cache.
> I.e. the cache drain level was zero.
> This (complete cache flush) meant that a subsequent get operation (with
> the same number of objects) completely emptied the cache,
> so another subsequent get operation required replenishing the cache.
>
> Similarly,
> When getting objects from a mempool, and the mempool cache did not hold
> so
> many objects,
> the cache was replenished to cache->size + remaining objects,
> and then (the remaining part of) the requested objects were fetched via
> the cache,
> which left the cache filled (to cache->size) at completion.
> I.e. the cache refill level was cache->size (plus some, depending on
> request size).
>
> (1) was improved by generally comparing to cache->size instead of
> cache->flushthresh, when considering the capacity of the cache.
> The cache->flushthresh field is kept for API/ABI compatibility
> purposes,
> and initialized to cache->size instead of cache->size * 1.5.
>
> (2) was improved by generally comparing to cache->size / 2 instead of
> RTE_MEMPOOL_CACHE_MAX_SIZE, when checking the bounce buffer limit.
>
> (3) was improved by flushing and replenishing the cache by half its
> size,
> so a flush/refill can be followed randomly by get or put requests.
> This also reduced the number of objects in each flush/refill operation.
>
> As a consequence of these changes, the size of the array holding the
> objects in the cache (cache->objs[]) no longer needs to be
> 2 * RTE_MEMPOOL_CACHE_MAX_SIZE, and can be reduced to
> RTE_MEMPOOL_CACHE_MAX_SIZE at an API/ABI breaking release.
>
> Performance data:
> With a real WAN Optimization application, where the number of allocated
> packets varies (as they are held in e.g. shaper queues), the mempool
> cache miss rate dropped from ca. 1/20 objects to ca. 1/48 objects.
> This was deployed in production at an ISP, and using an effective cache
> size of 384 objects.
>
> Bugzilla ID: 1027
> Fixes: ea5dd2744b90 ("mempool: cache optimisations")
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Forgot carrying an Ack over from v5:
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> ---
> Depends-on: patch-163181 ("net/intel: do not bypass mbuf lib for mbuf
> fast-free")
This dependency seems to cause CI apply failures.
The dependency is based on an older snapshot of main,
and this patch is based on a new snapshot of main.
> ---
> v6:
> * Moved driver changes out as separate patches, for easier review.
> (Bruce)
> Tests using the Intel idpf PMD in AVX512 mode may fail with this
> patch.
> * Reverted a small code comment change. The original was better.
> (Bruce)
> * Reverted rte_mempool_create() description requiring the cache_size to
> be
> an even number. There is no such requirement.
> v5:
> * Flush the cache from the bottom, where objects are colder, and move
> down
> the remaining objects, which are hotter.
> * In the Intel idpf PMD, move up the hot objects in the cache and
> refill
> with cold objects at the bottom.
> v4:
> * Added Bugzilla ID.
> * Added Fixes tag. For reference only.
> * Moved fast-free related update of Intel common driver out as a
> separate
> patch, and depend on that patch.
> * Omitted unrelated changes to the Intel idpf AVX512 driver,
> specifically
> fixing an indentation and adding mbuf instrumentation.
> * Omitted unrelated changes to the mempool library, specifically adding
> __rte_restrict and changing a couple of comments to proper sentences.
> * Please checkpatches by swapping operators in a couple of comparisons.
> v3:
> * Fixed my copy-paste bug in idpf_splitq_rearm().
> v2:
> * Fixed issue found by abidiff:
> Reverted cache objects array size reduction. Added a note instead.
> * Added missing mbuf instrumentation to the Intel idpf AVX512 driver.
> * Updated idpf_splitq_rearm() like idpf_singleq_rearm().
> * Added a few more __rte_assume(). (Inspired by AI review)
> * Updated NXP dpaa and dpaa2 mempool drivers to not set mempool cache
> flush threshold.
> * Added release notes.
> * Added deprecation notes.
> ---
> doc/guides/rel_notes/deprecation.rst | 7 +++
> doc/guides/rel_notes/release_26_07.rst | 11 +++++
> lib/mempool/rte_mempool.c | 14 +-----
> lib/mempool/rte_mempool.h | 66 ++++++++++++++++----------
> 4 files changed, 61 insertions(+), 37 deletions(-)
>
> diff --git a/doc/guides/rel_notes/deprecation.rst
> b/doc/guides/rel_notes/deprecation.rst
> index 35c9b4e06c..40760fffbb 100644
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> @@ -154,3 +154,10 @@ Deprecation Notices
> * bus/vmbus: Starting DPDK 25.11, all the vmbus API defined in
> ``drivers/bus/vmbus/rte_bus_vmbus.h`` will become internal to DPDK.
> Those API functions are used internally by DPDK core and netvsc PMD.
> +
> +* mempool: The ``flushthresh`` field in ``struct rte_mempool_cache``
> + is obsolete, and will be removed in DPDK 26.11.
> +
> +* mempool: The object array in ``struct rte_mempool_cache`` is
> oversize by
> + factor two, and will be reduced to ``RTE_MEMPOOL_CACHE_MAX_SIZE`` in
> + DPDK 26.11.
> diff --git a/doc/guides/rel_notes/release_26_07.rst
> b/doc/guides/rel_notes/release_26_07.rst
> index 6f43d9b61c..3f793f504a 100644
> --- a/doc/guides/rel_notes/release_26_07.rst
> +++ b/doc/guides/rel_notes/release_26_07.rst
> @@ -63,6 +63,17 @@ New Features
> ``rte_eal_init`` and the application is responsible for probing
> each device,
> * ``--auto-probing`` enables the initial bus probing, which is the
> current default behavior.
>
> +* **Changed effective size of mempool cache.**
> +
> + * The effective size of a mempool cache was changed to match the
> specified size at mempool creation; the effective size was previously
> 50 % larger than requested.
> + * The ``flushthresh`` field of the ``struct rte_mempool_cache``
> became obsolete, but was kept for API/ABI compatibility purposes.
> + * The effective size of the ``objs`` array in the ``struct
> rte_mempool_cache`` was reduced to ``RTE_MEMPOOL_CACHE_MAX_SIZE``, but
> its size was kept for API/ABI compatibility purposes.
> +
> +* **Improved mempool cache flush/refill algorithm.**
> +
> + The mempool cache flush/refill algorithm was improved, to reduce the
> mempool cache miss rate for most application types.
> + Applications where each lcore only puts or gets to a mempool, e.g.
> pipelined applications where ethdev Rx and Tx run on separate lcores,
> should adapt to the new algorithm by doubling their configured mempool
> cache size, to avoid doubling their mempool cache miss rate.
> +
> * **Updated PCAP ethernet driver.**
>
> * Added support for VLAN insertion and stripping.
> diff --git a/lib/mempool/rte_mempool.c b/lib/mempool/rte_mempool.c
> index 3042d94c14..805b52cc58 100644
> --- a/lib/mempool/rte_mempool.c
> +++ b/lib/mempool/rte_mempool.c
> @@ -52,11 +52,6 @@ static void
> mempool_event_callback_invoke(enum rte_mempool_event event,
> struct rte_mempool *mp);
>
> -/* Note: avoid using floating point since that compiler
> - * may not think that is constant.
> - */
> -#define CALC_CACHE_FLUSHTHRESH(c) (((c) * 3) / 2)
> -
> #if defined(RTE_ARCH_X86)
> /*
> * return the greatest common divisor between a and b (fast algorithm)
> @@ -757,13 +752,8 @@ rte_mempool_free(struct rte_mempool *mp)
> static void
> mempool_cache_init(struct rte_mempool_cache *cache, uint32_t size)
> {
> - /* Check that cache have enough space for flush threshold */
> -
> RTE_BUILD_BUG_ON(CALC_CACHE_FLUSHTHRESH(RTE_MEMPOOL_CACHE_MAX_SIZ
> E) >
> - RTE_SIZEOF_FIELD(struct rte_mempool_cache, objs) /
> - RTE_SIZEOF_FIELD(struct rte_mempool_cache,
> objs[0]));
> -
> cache->size = size;
> - cache->flushthresh = CALC_CACHE_FLUSHTHRESH(size);
> + cache->flushthresh = size; /* Obsolete; for API/ABI compatibility
> purposes only */
> cache->len = 0;
> }
>
> @@ -850,7 +840,7 @@ rte_mempool_create_empty(const char *name, unsigned
> n, unsigned elt_size,
>
> /* asked cache too big */
> if (cache_size > RTE_MEMPOOL_CACHE_MAX_SIZE ||
> - CALC_CACHE_FLUSHTHRESH(cache_size) > n) {
> + cache_size > n) {
> rte_errno = EINVAL;
> return NULL;
> }
> diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h
> index 2e54fc4466..cd0f229b59 100644
> --- a/lib/mempool/rte_mempool.h
> +++ b/lib/mempool/rte_mempool.h
> @@ -89,7 +89,7 @@ struct __rte_cache_aligned rte_mempool_debug_stats {
> */
> struct __rte_cache_aligned rte_mempool_cache {
> uint32_t size; /**< Size of the cache */
> - uint32_t flushthresh; /**< Threshold before we flush excess
> elements */
> + uint32_t flushthresh; /**< Obsolete; for API/ABI compatibility
> purposes only */
> uint32_t len; /**< Current cache count */
> #ifdef RTE_LIBRTE_MEMPOOL_STATS
> uint32_t unused;
> @@ -107,8 +107,10 @@ struct __rte_cache_aligned rte_mempool_cache {
> /**
> * Cache objects
> *
> - * Cache is allocated to this size to allow it to overflow in
> certain
> - * cases to avoid needless emptying of cache.
> + * Note:
> + * Cache is allocated at double size for API/ABI compatibility
> purposes only.
> + * When reducing its size at an API/ABI breaking release,
> + * remember to add a cache guard after it.
> */
> alignas(RTE_CACHE_LINE_SIZE) void
> *objs[RTE_MEMPOOL_CACHE_MAX_SIZE * 2];
> };
> @@ -1047,11 +1049,16 @@ rte_mempool_free(struct rte_mempool *mp);
> * If cache_size is non-zero, the rte_mempool library will try to
> * limit the accesses to the common lockless pool, by maintaining a
> * per-lcore object cache. This argument must be lower or equal to
> - * RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.
> + * RTE_MEMPOOL_CACHE_MAX_SIZE and n.
> * The access to the per-lcore table is of course
> * faster than the multi-producer/consumer pool. The cache can be
> * disabled if the cache_size argument is set to 0; it can be useful
> to
> * avoid losing objects in cache.
> + * Note:
> + * Mempool put/get requests of more than cache_size / 2 objects may
> be
> + * partially or fully served directly by the multi-producer/consumer
> + * pool, to avoid the overhead of copying the objects twice (instead
> of
> + * once) when using the cache as a bounce buffer.
> * @param private_data_size
> * The size of the private data appended after the mempool
> * structure. This is useful for storing some private data after the
> @@ -1390,22 +1397,30 @@ rte_mempool_do_generic_put(struct rte_mempool
> *mp, void * const *obj_table,
> RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_bulk, 1);
> RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_objs, n);
>
> - __rte_assume(cache->flushthresh <= RTE_MEMPOOL_CACHE_MAX_SIZE *
> 2);
> - __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE * 2);
> - __rte_assume(cache->len <= cache->flushthresh);
> - if (likely(cache->len + n <= cache->flushthresh)) {
> + __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> + __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
> + __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> + __rte_assume(cache->len <= cache->size);
> + if (likely(cache->len + n <= cache->size)) {
> /* Sufficient room in the cache for the objects. */
> cache_objs = &cache->objs[cache->len];
> cache->len += n;
> - } else if (n <= cache->flushthresh) {
> + } else if (n <= cache->size / 2) {
> /*
> - * The cache is big enough for the objects, but - as
> detected by
> - * the comparison above - has insufficient room for them.
> - * Flush the cache to make room for the objects.
> + * The number of objects is within the cache bounce buffer
> limit,
> + * but - as detected by the comparison above - the cache
> has
> + * insufficient room for them.
> + * Flush the cache to the backend to make room for the
> objects;
> + * flush (size / 2) objects from the bottom of the cache,
> where
> + * objects are less hot, and move down the remaining
> objects, which
> + * are more hot, from the upper half of the cache.
> */
> - cache_objs = &cache->objs[0];
> - rte_mempool_ops_enqueue_bulk(mp, cache_objs, cache->len);
> - cache->len = n;
> + __rte_assume(cache->len > cache->size / 2);
> + rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0], cache-
> >size / 2);
> + rte_memcpy(&cache->objs[0], &cache->objs[cache->size / 2],
> + sizeof(void *) * (cache->len - cache->size /
> 2));
> + cache_objs = &cache->objs[cache->len - cache->size / 2];
> + cache->len = cache->len - cache->size / 2 + n;
> } else {
> /* The request itself is too big for the cache. */
> goto driver_enqueue_stats_incremented;
> @@ -1524,7 +1539,7 @@ rte_mempool_do_generic_get(struct rte_mempool
> *mp, void **obj_table,
> /* The cache is a stack, so copy will be in reverse order. */
> cache_objs = &cache->objs[cache->len];
>
> - __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE * 2);
> + __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> if (likely(n <= cache->len)) {
> /* The entire request can be satisfied from the cache. */
> RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk, 1);
> @@ -1548,13 +1563,13 @@ rte_mempool_do_generic_get(struct rte_mempool
> *mp, void **obj_table,
> for (index = 0; index < len; index++)
> *obj_table++ = *--cache_objs;
>
> - /* Dequeue below would overflow mem allocated for cache? */
> - if (unlikely(remaining > RTE_MEMPOOL_CACHE_MAX_SIZE))
> + /* Dequeue below would exceed the cache bounce buffer limit? */
> + __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
> + if (unlikely(remaining > cache->size / 2))
> goto driver_dequeue;
>
> - /* Fill the cache from the backend; fetch size + remaining
> objects. */
> - ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs,
> - cache->size + remaining);
> + /* Fill the cache from the backend; fetch (size / 2) objects. */
> + ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, cache->size /
> 2);
> if (unlikely(ret < 0)) {
> /*
> * We are buffer constrained, and not able to fetch all
> that.
> @@ -1568,10 +1583,11 @@ rte_mempool_do_generic_get(struct rte_mempool
> *mp, void **obj_table,
> RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk, 1);
> RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_objs, n);
>
> - __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> - __rte_assume(remaining <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> - cache_objs = &cache->objs[cache->size + remaining];
> - cache->len = cache->size;
> + __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
> + __rte_assume(remaining <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
> + __rte_assume(remaining <= cache->size / 2);
> + cache_objs = &cache->objs[cache->size / 2];
> + cache->len = cache->size / 2 - remaining;
> for (index = 0; index < remaining; index++)
> *obj_table++ = *--cache_objs;
>
> --
> 2.43.0
^ permalink raw reply
* Re: [PATCH v2 4/6] eal/memory: get rid of global VA space limits
From: Bruce Richardson @ 2026-05-26 16:02 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev, Wathsala Vithanage
In-Reply-To: <1f0e3755ed80dd3dfa98d35002afde053e0d23b9.1773417833.git.anatoly.burakov@intel.com>
On Fri, Mar 13, 2026 at 04:06:35PM +0000, Anatoly Burakov wrote:
> Currently, all VA space reservations take into account global memory limit.
> The original intent was to limit memory allocations to however many NUMA
> nodes the machine had taking into the account that socket ID's may be
> discontiguous. Since we have had "socket count" API for while and it gives
> us correct NUMA node count, taking discontiguousness into account, we can
> relax the total limits and remove the restrictions, and let VA space usage
> scale with NUMA nodes.
>
> The only place where we actually require a hard limit is in 32-bit code,
> where we cannot allocate more than 2G of VA space.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Asking AI about this patch flags an issue whereby the contigmem driver does
not guarantee the number of buffers is >0, which can cause issues. However,
that's better fixed in contigmem.
One additional comment inline below.
> ---
> config/arm/meson.build | 1 -
> config/meson.build | 5 ----
> .../prog_guide/env_abstraction_layer.rst | 2 --
> lib/eal/common/eal_common_dynmem.c | 13 +++------
> lib/eal/freebsd/eal_memory.c | 28 +++----------------
> lib/eal/linux/eal_memory.c | 10 +++----
> 6 files changed, 13 insertions(+), 46 deletions(-)
>
> diff --git a/config/arm/meson.build b/config/arm/meson.build
> index 523b0fc0ed..3b03f5e31b 100644
> --- a/config/arm/meson.build
> +++ b/config/arm/meson.build
> @@ -69,7 +69,6 @@ part_number_config_arm = {
> 'flags': [
> ['RTE_MACHINE', '"neoverse-n1"'],
> ['RTE_ARM_FEATURE_ATOMICS', true],
> - ['RTE_MAX_MEM_MB', 1048576],
> ['RTE_MAX_LCORE', 256],
> ['RTE_MAX_NUMA_NODES', 8]
> ]
> diff --git a/config/meson.build b/config/meson.build
> index 02e2798cca..f68f1f5f53 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -383,11 +383,6 @@ dpdk_conf.set('RTE_PKTMBUF_HEADROOM', get_option('pkt_mbuf_headroom'))
> dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64)
> dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
> dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
> -if dpdk_conf.get('RTE_ARCH_64')
> - dpdk_conf.set('RTE_MAX_MEM_MB', 524288)
> -else # for 32-bit we need smaller reserved memory areas
> - dpdk_conf.set('RTE_MAX_MEM_MB', 2048)
> -endif
> if get_option('mbuf_refcnt_atomic')
> dpdk_conf.set('RTE_MBUF_REFCNT_ATOMIC', true)
> endif
> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
> index 04368a3950..63e0568afa 100644
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> @@ -208,8 +208,6 @@ variables:
> can have (where "type" is defined as "page size + NUMA node" combination)
Is the RTE_MAX_MEMSEG_PER_TYPE value still necessary?
> * ``RTE_MAX_MEM_MB_PER_TYPE`` controls how much megabytes of memory each
> memory type can address
> -* ``RTE_MAX_MEM_MB`` places a global maximum on the amount of memory
> - DPDK can reserve
>
<snip>
^ permalink raw reply
* Re: [PATCH v2 5/6] eal/memory: store default segment limits in config
From: Bruce Richardson @ 2026-05-26 16:08 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev, Dmitry Kozlyuk
In-Reply-To: <4b4aa7b43e8354f5ae325e634c88e4e649707be5.1773417833.git.anatoly.burakov@intel.com>
On Fri, Mar 13, 2026 at 04:06:36PM +0000, Anatoly Burakov wrote:
> Currently, VA space allocation is regulated by two constants picked up from
> config - max memseg per list, and max memory per list. In preparation for
> these limits being dynamic, add a per-page-size limit value in config,
> populate that value from these defaults at init time, and adjust the code
> to only refer to the mem limits from internal config.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
^ permalink raw reply
* Re: [PATCH v2 6/6] eal/memory: add page size VA limits EAL parameter
From: Bruce Richardson @ 2026-05-26 16:16 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev
In-Reply-To: <968ab78a3f7bd130287e269dfe82a3481baab161.1773417833.git.anatoly.burakov@intel.com>
On Fri, Mar 13, 2026 at 04:06:37PM +0000, Anatoly Burakov wrote:
> Currently, the VA space limits placed on DPDK memory are only informed by
> the default configuration coming from `rte_config.h` file. Add an EAL flag
> to specify per-page size memory limits explicitly, thereby overriding the
> default VA space reservations.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
CI reports some errors with 32-bit builds. Also a couple of small comments
inline below.
Thanks for the series, looks some nice simplification.
> app/test/test.c | 1 +
> app/test/test_eal_flags.c | 126 ++++++++++++++++++
> doc/guides/linux_gsg/linux_eal_parameters.rst | 13 ++
> .../prog_guide/env_abstraction_layer.rst | 27 +++-
> lib/eal/common/eal_common_dynmem.c | 9 ++
> lib/eal/common/eal_common_options.c | 121 +++++++++++++++++
> lib/eal/common/eal_internal_cfg.h | 6 +
> lib/eal/common/eal_option_list.h | 1 +
> 8 files changed, 302 insertions(+), 2 deletions(-)
>
> diff --git a/app/test/test.c b/app/test/test.c
> index 58ef52f312..c610c3588e 100644
> --- a/app/test/test.c
> +++ b/app/test/test.c
> @@ -80,6 +80,7 @@ do_recursive_call(void)
> { "test_memory_flags", no_action },
> { "test_file_prefix", no_action },
> { "test_no_huge_flag", no_action },
> + { "test_pagesz_mem_flags", no_action },
> { "test_panic", test_panic },
> { "test_exit", test_exit },
> #ifdef RTE_LIB_TIMER
> diff --git a/app/test/test_eal_flags.c b/app/test/test_eal_flags.c
> index b3a8d0ae6f..4e1038be75 100644
> --- a/app/test/test_eal_flags.c
> +++ b/app/test/test_eal_flags.c
> @@ -95,6 +95,14 @@ test_misc_flags(void)
> return TEST_SKIPPED;
> }
>
> +static int
> +test_pagesz_mem_flags(void)
> +{
> + printf("pagesz_mem_flags not supported on Windows, skipping test\n");
> + return TEST_SKIPPED;
> +}
> +
> +
> #else
>
> #include <libgen.h>
> @@ -1502,6 +1510,123 @@ populate_socket_mem_param(int num_sockets, const char *mem,
> offset += written;
> }
>
> +/*
> + * Tests for correct handling of --pagesz-mem flag
> + */
> +static int
> +test_pagesz_mem_flags(void)
> +{
> +#ifdef RTE_EXEC_ENV_FREEBSD
> + /* FreeBSD does not support --pagesz-mem */
> + return 0;
> +#else
> + const char *in_memory = "--in-memory";
> +
> + /* invalid: no value */
> + const char * const argv0[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem="};
> +
> + /* invalid: no colon (missing limit) */
> + const char * const argv1[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=2M"};
> +
> + /* invalid: colon present but limit is empty */
> + const char * const argv2[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=2M:"};
> +
> + /* invalid: limit not aligned to page size (3M is not a multiple of 2M) */
> + const char * const argv3[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=2M:3M"};
> +
> + /* invalid: garbage value */
> + const char * const argv4[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=garbage"};
> +
> + /* invalid: garbage value */
> + const char * const argv5[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=2M:garbage"};
> +
> + /* invalid: --pagesz-mem combined with --no-huge */
> + const char * const argv6[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, no_huge, "--pagesz-mem=2M:2M"};
> +
> + /* valid: single well-formed aligned pair */
> + const char * const argv7[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=2M:64M"};
> +
> + /* valid: multiple occurrences */
> + const char * const argv8[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory,
> + "--pagesz-mem=2M:64M", "--pagesz-mem=1K:8K"};
> +
> + /* valid: fake page size set to zero (ignored but syntactically valid) */
> + const char * const argv9[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=1K:0"};
> +
> + /* invalid: page size must be a power of two */
> + const char * const argv10[] = {prgname, eal_debug_logs, no_pci,
> + "--file-prefix=" memtest, in_memory, "--pagesz-mem=3M:6M"};
> +
> + if (launch_proc(argv0) == 0) {
> + printf("Error (line %d) - process run ok with empty --pagesz-mem!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv1) == 0) {
> + printf("Error (line %d) - process run ok with --pagesz-mem missing colon!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv2) == 0) {
> + printf("Error (line %d) - process run ok with --pagesz-mem missing limit!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv3) == 0) {
> + printf("Error (line %d) - process run ok with --pagesz-mem unaligned limit!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv4) == 0) {
> + printf("Error (line %d) - process run ok with --pagesz-mem garbage value!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv5) == 0) {
> + printf("Error (line %d) - process run ok with --pagesz-mem garbage value!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv6) == 0) {
> + printf("Error (line %d) - process run ok with --pagesz-mem and --no-huge!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv7) != 0) {
> + printf("Error (line %d) - process failed with valid --pagesz-mem!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv8) != 0) {
> + printf("Error (line %d) - process failed with multiple valid --pagesz-mem!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv9) != 0) {
> + printf("Error (line %d) - process failed with --pagesz-mem zero limit!\n",
> + __LINE__);
> + return -1;
> + }
> + if (launch_proc(argv10) == 0) {
> + printf("Error (line %d) - process run ok with non-power-of-two pagesz!\n",
> + __LINE__);
> + return -1;
> + }
> +
> + return 0;
> +#endif /* !RTE_EXEC_ENV_FREEBSD */
> +}
> +
> /*
> * Tests for correct handling of -m and --socket-mem flags
> */
> @@ -1683,5 +1808,6 @@ REGISTER_FAST_TEST(eal_flags_b_opt_autotest, NOHUGE_SKIP, ASAN_SKIP, test_invali
> REGISTER_FAST_TEST(eal_flags_vdev_opt_autotest, NOHUGE_SKIP, ASAN_SKIP, test_invalid_vdev_flag);
> REGISTER_FAST_TEST(eal_flags_r_opt_autotest, NOHUGE_SKIP, ASAN_SKIP, test_invalid_r_flag);
> REGISTER_FAST_TEST(eal_flags_mem_autotest, NOHUGE_SKIP, ASAN_SKIP, test_memory_flags);
> +REGISTER_FAST_TEST(eal_flags_pagesz_mem_autotest, NOHUGE_SKIP, ASAN_SKIP, test_pagesz_mem_flags);
> REGISTER_FAST_TEST(eal_flags_file_prefix_autotest, NOHUGE_SKIP, ASAN_SKIP, test_file_prefix);
> REGISTER_FAST_TEST(eal_flags_misc_autotest, NOHUGE_SKIP, ASAN_SKIP, test_misc_flags);
> diff --git a/doc/guides/linux_gsg/linux_eal_parameters.rst b/doc/guides/linux_gsg/linux_eal_parameters.rst
> index 7c5b26ce26..ce38dd128a 100644
> --- a/doc/guides/linux_gsg/linux_eal_parameters.rst
> +++ b/doc/guides/linux_gsg/linux_eal_parameters.rst
> @@ -75,6 +75,19 @@ Memory-related options
> Place a per-NUMA node upper limit on memory use (non-legacy memory mode only).
> 0 will disable the limit for a particular NUMA node.
>
> +* ``--pagesz-mem <page size:limit>``
> +
> + Set memory limit per hugepage size.
> + Each time the option is used, provide a single ``<pagesz>:<limit>`` pair;
> + repeat the option to specify additional page sizes.
> + Both values support K/M/G/T suffixes (for example ``2M:32G``).
> +
> + The memory limit must be a multiple of page size.
> +
> + For example::
> +
> + --pagesz-mem 2M:32G --pagesz-mem 1G:512G
> +
> * ``--single-file-segments``
>
> Create fewer files in hugetlbfs (non-legacy mode only).
> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
> index 63e0568afa..e2adf0a184 100644
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> @@ -204,13 +204,36 @@ of virtual memory being preallocated at startup by editing the following config
> variables:
>
> * ``RTE_MAX_MEMSEG_LISTS`` controls how many segment lists can DPDK have
> -* ``RTE_MAX_MEMSEG_PER_TYPE`` controls how many segments each memory type
> +* ``RTE_MAX_MEMSEG_PER_TYPE`` sets the default number of segments each memory type
> can have (where "type" is defined as "page size + NUMA node" combination)
> -* ``RTE_MAX_MEM_MB_PER_TYPE`` controls how much megabytes of memory each
> +* ``RTE_MAX_MEM_MB_PER_TYPE`` sets the default amount of memory each
> memory type can address
>
> Normally, these options do not need to be changed.
>
> +Runtime Override of Per-Page-Size Memory Limits
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +By default, DPDK uses compile-time configured limits for memory allocation per page size
> +(as set by ``RTE_MAX_MEM_MB_PER_TYPE``).
> +These limits apply uniformly across all NUMA nodes for a given page size.
> +
> +It is possible to override these defaults at runtime using the ``--pagesz-mem`` option,
> +which allows specifying custom memory limits for each page size. This is useful when:
> +
> +* The default limits may be insufficient or excessive for your workload
> +* You want to dedicate more memory to specific page sizes
> +
> +The ``--pagesz-mem`` option accepts exactly one ``<pagesz>:<limit>`` pair per
> +occurrence, where ``pagesz`` is a page size (e.g., ``2M``, ``4M``, ``1G``)
> +and ``limit`` is the maximum memory to reserve for that page size (e.g., ``64G``, ``512M``).
> +Both values support standard binary suffixes (K, M, G, T).
> +Memory limits must be aligned to their corresponding page size.
> +
> +Multiple page sizes can be specified by repeating the option::
> +
> + --pagesz-mem 2M:64G --pagesz-mem 1G:512G
> +
> .. note::
>
> Preallocated virtual memory is not to be confused with preallocated hugepage
> diff --git a/lib/eal/common/eal_common_dynmem.c b/lib/eal/common/eal_common_dynmem.c
> index c33fbdea6d..7096f46ff3 100644
> --- a/lib/eal/common/eal_common_dynmem.c
> +++ b/lib/eal/common/eal_common_dynmem.c
> @@ -127,6 +127,11 @@ eal_dynmem_memseg_lists_init(void)
> mem_va_len += type->mem_sz;
> }
>
> + if (mem_va_len == 0) {
> + EAL_LOG(ERR, "No virtual memory will be reserved");
> + goto out;
> + }
> +
> mem_va_addr = eal_get_virtual_area(NULL, &mem_va_len,
> mem_va_page_sz, 0, 0);
> if (mem_va_addr == NULL) {
> @@ -141,6 +146,10 @@ eal_dynmem_memseg_lists_init(void)
> uint64_t pagesz;
> int socket_id;
>
> + /* skip page sizes with zero memory limit */
> + if (type->n_segs == 0)
> + continue;
> +
> pagesz = type->page_sz;
> socket_id = type->socket_id;
>
> diff --git a/lib/eal/common/eal_common_options.c b/lib/eal/common/eal_common_options.c
> index bbc4427524..0532d27aaa 100644
> --- a/lib/eal/common/eal_common_options.c
> +++ b/lib/eal/common/eal_common_options.c
> @@ -21,6 +21,7 @@
> #endif
>
> #include <rte_string_fns.h>
> +#include <rte_common.h>
> #include <rte_eal.h>
> #include <rte_log.h>
> #include <rte_lcore.h>
> @@ -233,6 +234,20 @@ eal_collate_args(int argc, char **argv)
> EAL_LOG(ERR, "Options allow (-a) and block (-b) can't be used at the same time");
> return -1;
> }
> +#ifdef RTE_EXEC_ENV_FREEBSD
> + if (!TAILQ_EMPTY(&args.pagesz_mem)) {
> + EAL_LOG(ERR, "Option pagesz-mem is not supported on FreeBSD");
> + return -1;
> + }
> +#endif
> + if (!TAILQ_EMPTY(&args.pagesz_mem) && args.no_huge) {
> + EAL_LOG(ERR, "Options pagesz-mem and no-huge can't be used at the same time");
> + return -1;
> + }
> + if (!TAILQ_EMPTY(&args.pagesz_mem) && args.legacy_mem) {
> + EAL_LOG(ERR, "Options pagesz-mem and legacy-mem can't be used at the same time");
> + return -1;
> + }
>
> /* for non-list args, we can just check for zero/null values using macro */
> if (CONFLICTING_OPTIONS(args, coremask, lcores) ||
> @@ -511,7 +526,10 @@ eal_reset_internal_config(struct internal_config *internal_cfg)
> sizeof(internal_cfg->hugepage_info[0]));
> internal_cfg->hugepage_info[i].lock_descriptor = -1;
> internal_cfg->hugepage_mem_sz_limits[i] = 0;
> + internal_cfg->pagesz_mem_overrides[i].pagesz = 0;
> + internal_cfg->pagesz_mem_overrides[i].limit = 0;
> }
> + internal_cfg->num_pagesz_mem_overrides = 0;
> internal_cfg->base_virtaddr = 0;
>
> /* if set to NONE, interrupt mode is determined automatically */
> @@ -1867,6 +1885,96 @@ eal_parse_socket_arg(char *strval, volatile uint64_t *socket_arg)
> return 0;
> }
>
> +static int
> +eal_parse_pagesz_mem(char *strval, struct internal_config *internal_cfg)
> +{
> + char strval_cpy[1024];
> + char *fields[3];
> + char *pagesz_str, *mem_str;
> + int arg_num;
> + int len;
> + unsigned int i;
> + uint64_t pagesz, mem_limit;
> + struct pagesz_mem_override *pmo;
> +
> + len = strnlen(strval, 1024);
> + if (len >= 1024) {
> + EAL_LOG(ERR, "--pagesz-mem parameter is too long");
> + return -1;
> + }
> +
> + rte_strlcpy(strval_cpy, strval, sizeof(strval_cpy));
> +
> + /* parse exactly one pagesz:mem pair per --pagesz-mem option */
> + arg_num = rte_strsplit(strval_cpy, len, fields, RTE_DIM(fields), ':');
> + if (arg_num != 2 || fields[0][0] == '\0' || fields[1][0] == '\0') {
> + EAL_LOG(ERR, "--pagesz-mem parameter format is invalid, expected <pagesz>:<limit>");
> + return -1;
> + }
> + pagesz_str = fields[0];
> + mem_str = fields[1];
> +
> + /* reject accidental multiple pairs in one option */
> + if (strchr(mem_str, ',') != NULL) {
> + EAL_LOG(ERR, "--pagesz-mem accepts one <pagesz>:<limit> pair per option");
> + return -1;
> + }
If multiple options are given, then the rte_strsplit should return >2 when
splitting on ":". I'd suggest checking for the comma first, before even
doing the strlcpy.
> +
> + /* parse page size */
> + errno = 0;
> + pagesz = rte_str_to_size(pagesz_str);
> + if (pagesz == 0 || errno != 0) {
> + EAL_LOG(ERR, "invalid page size in --pagesz-mem: '%s'", pagesz_str);
> + return -1;
> + }
> + if (!rte_is_power_of_2(pagesz)) {
> + EAL_LOG(ERR, "invalid page size in --pagesz-mem: '%s' (must be a power of two)",
> + pagesz_str);
> + return -1;
> + }
> +
> + /* parse memory limit (0 is valid: disables allocation for this page size) */
> + errno = 0;
> + mem_limit = rte_str_to_size(mem_str);
> + if (errno != 0) {
> + EAL_LOG(ERR, "invalid memory limit in --pagesz-mem: '%s'", mem_str);
> + return -1;
> + }
> +
> + /* validate alignment: memory limit must be divisible by page size */
> + if (mem_limit % pagesz != 0) {
> + EAL_LOG(ERR, "--pagesz-mem memory limit must be aligned to page size");
> + return -1;
> + }
> +
> + for (i = 0; i < internal_cfg->num_pagesz_mem_overrides; i++) {
> + pmo = &internal_cfg->pagesz_mem_overrides[i];
> + if (pmo->pagesz != pagesz)
> + continue;
> +
> + EAL_LOG(WARNING,
> + "--pagesz-mem specified multiple times for page size '%s'; later limit '%s' will be used",
> + pagesz_str, mem_str);
> + pmo->limit = mem_limit;
> + return 0;
Rather than just warning, I'd make this a hard error and say you can't
duplicate the hugepage limits on commandline. Saves confusion when
examining a commandline, having to check if a value is overridden later.
> + }
> +
> + /* do we have space? */
> + if (internal_cfg->num_pagesz_mem_overrides >= MAX_HUGEPAGE_SIZES) {
> + EAL_LOG(ERR,
> + "--pagesz-mem: too many page size entries (max %d)",
> + MAX_HUGEPAGE_SIZES);
> + return -1;
> + }
> +
> + pmo = &internal_cfg->pagesz_mem_overrides[internal_cfg->num_pagesz_mem_overrides];
> + pmo->pagesz = pagesz;
> + pmo->limit = mem_limit;
> + internal_cfg->num_pagesz_mem_overrides++;
> +
> + return 0;
> +}
> +
> static int
> eal_parse_vfio_intr(const char *mode)
> {
> @@ -2172,6 +2280,12 @@ eal_parse_args(void)
> }
> int_cfg->force_numa_limits = 1;
> }
> + TAILQ_FOREACH(arg, &args.pagesz_mem, next) {
> + if (eal_parse_pagesz_mem(arg->arg, int_cfg) < 0) {
> + EAL_LOG(ERR, "invalid pagesz-mem parameter: '%s'", arg->arg);
> + return -1;
> + }
> + }
>
> /* tracing settings, not supported on windows */
> #ifdef RTE_EXEC_ENV_WINDOWS
> @@ -2366,6 +2480,7 @@ eal_apply_hugepage_mem_sz_limits(struct internal_config *internal_cfg)
> unsigned int i;
>
> for (i = 0; i < internal_cfg->num_hugepage_sizes; i++) {
> + unsigned int j;
> const uint64_t pagesz = internal_cfg->hugepage_info[i].hugepage_sz;
> uint64_t limit;
>
> @@ -2373,6 +2488,12 @@ eal_apply_hugepage_mem_sz_limits(struct internal_config *internal_cfg)
> limit = RTE_MIN((uint64_t)RTE_MAX_MEM_MB_PER_TYPE << 20,
> (uint64_t)RTE_MAX_MEMSEG_PER_TYPE * pagesz);
>
> + /* override with user value for matching page size */
> + for (j = 0; j < (unsigned int)internal_cfg->num_pagesz_mem_overrides; j++) {
> + if (internal_cfg->pagesz_mem_overrides[j].pagesz == pagesz)
> + limit = internal_cfg->pagesz_mem_overrides[j].limit;
> + }
> +
> internal_cfg->hugepage_mem_sz_limits[i] = limit;
> }
>
> diff --git a/lib/eal/common/eal_internal_cfg.h b/lib/eal/common/eal_internal_cfg.h
> index 0bf192c6e5..8475c87969 100644
> --- a/lib/eal/common/eal_internal_cfg.h
> +++ b/lib/eal/common/eal_internal_cfg.h
> @@ -98,6 +98,12 @@ struct internal_config {
> struct hugepage_info hugepage_info[MAX_HUGEPAGE_SIZES];
> uint64_t hugepage_mem_sz_limits[MAX_HUGEPAGE_SIZES];
> /**< default max memory per hugepage size */
> + /** storage for user-specified pagesz-mem overrides */
> + struct pagesz_mem_override {
> + uint64_t pagesz; /**< page size in bytes */
> + uint64_t limit; /**< memory limit in bytes */
> + } pagesz_mem_overrides[MAX_HUGEPAGE_SIZES];
> + unsigned int num_pagesz_mem_overrides; /**< number of stored overrides */
> enum rte_iova_mode iova_mode ; /**< Set IOVA mode on this system */
> rte_cpuset_t ctrl_cpuset; /**< cpuset for ctrl threads */
> volatile unsigned int init_complete;
> diff --git a/lib/eal/common/eal_option_list.h b/lib/eal/common/eal_option_list.h
> index abee16340b..164a0b3888 100644
> --- a/lib/eal/common/eal_option_list.h
> +++ b/lib/eal/common/eal_option_list.h
> @@ -56,6 +56,7 @@ BOOL_ARG("--no-huge", NULL, "Disable hugetlbfs support", no_huge)
> BOOL_ARG("--no-pci", NULL, "Disable all PCI devices", no_pci)
> BOOL_ARG("--no-shconf", NULL, "Disable shared config file generation", no_shconf)
> BOOL_ARG("--no-telemetry", NULL, "Disable telemetry", no_telemetry)
> +LIST_ARG("--pagesz-mem", NULL, "Memory allocation per hugepage size (format: <pagesz>:<limit>, e.g. 2M:32G). Repeat option for multiple page sizes.", pagesz_mem)
> STR_ARG("--proc-type", NULL, "Type of process (primary|secondary|auto)", proc_type)
> OPT_STR_ARG("--remap-lcore-ids", "-R", "Remap lcore IDs to be contiguous starting from 0, or supplied value", remap_lcore_ids)
> STR_ARG("--service-corelist", "-S", "List of cores to use for service threads", service_corelist)
> --
> 2.47.3
>
^ permalink raw reply
* RE: [PATCH v5] mempool: improve cache behaviour and performance
From: Morten Brørup @ 2026-05-26 17:45 UTC (permalink / raw)
To: Morten Brørup, Bruce Richardson
Cc: dev, Andrew Rybchenko, Jingjing Wu, Praveen Shetty,
Hemant Agrawal, Sachin Saxena
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35F6589D@smartserver.smartshare.dk>
> From: Morten Brørup [mailto:mb@smartsharesystems.com]
> Sent: Tuesday, 26 May 2026 12.37
>
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Tuesday, 26 May 2026 11.40
> >
[...]
> > [In all this, I am making the assumption that burst size is well less
> > than
> > cache size. Also, similar logic would be applicable for the inverse
> > scenario, e.g. flush to empty (and fill burst) and fill to 75%]
>
> I'm not so sure about this assumption.
> With a cache size of 512 and a bursts of 64, the cache only holds 8
> bursts.
> 50% is 4 bursts, and 25% is only 2 bursts.
>
> Using a replenish/drain level in the middle requires 5 bursts in either
> direction to pass the edge (and trigger replenish/flush).
> Using a replenish/drain level 25% from the edge requires only 3 bursts
> in the wrong direction to pass the edge (and trigger replenish/flush).
> Much higher probability with random get/put.
>
> >
> > Now, all said, I tend to agree that we want to leave space for a
> decent
> > size burst after a fill. That is why I think that filling to 75% is
> > reasonable. After an alloc that triggers a fill, I don't want the
> cache
> > less than 50% full, but not completely full so there is room for a
> free
> > without a flush, and similarly for a free that triggers a flush, the
> > cache
> > should not be empty, but also should not be more than half full.
> >
> > One suggestion - we could always add a simple tunable that specifies
> > the
> > margin, or reserved entries for alloc and free. We can then guide in
> > the
> > docs that the value should be e.g. "zero for apps where alloc and
> free
> > take
> > place on different cores. 20%-50% of cache is recommended where alloc
> > and
> > free take place on the same core"
>
> Yes, a simple tunable is a really good idea.
>
> At this point, I think we should optimize for use case #1, and go for
> the 50% fill level.
> Then we can add a tunable to optimize for use case #2 later. I will try
> to come up with a draft for such a follow-up patch within the next few
> days.
Adding a tunable is not so simple...
The choice of mempool cache algorithm (drain/replenish to 50% vs. drain/replenish completely) should be passed via the "flags" parameter in rte_mempool_create(), but rte_pktmbuf_pool_create() is missing the "flags" parameter.
We can add it at the next ABI breaking release.
WDYT?
We should use that addition as an opportunity to move the case where the objects are not entirely handled by the cache into non-inlined functions, so the inlined functions don't grow too much in size, when they need to handle two different algorithms.
>
> The 50% fill level in this patch is not as bad for use case #2 (roughly
> doubling the burst miss rate from 1/8 to 1/4), compared to how bad the
> original algorithm is for use case #1 (very high miss probability -
> only two ops in the wrong direction - after drain/replenish).
>
> -Morten
^ permalink raw reply
* [PATCH 0/2] ethdev: fix out-of-bounds writes in rte_flow_conv()
From: James Raphael Tiovalen @ 2026-05-26 18:11 UTC (permalink / raw)
To: orika, thomas, andrew.rybchenko; +Cc: dev, stable, James Raphael Tiovalen
rte_flow_conv() is documented to truncate output to the caller-supplied
buffer size, but two paths handling variable-length trailing data
ignored that contract and copied the full payload whenever the
destination pointer was non-NULL. A caller passing a buffer just large
enough for the fixed-size header had adjacent memory clobbered:
- GENEVE_OPT: up to option_len * 4 bytes
- FLEX: up to 4 GiB, since src->length is a uint32_t and the API places
no bounds on it
Patch 1 aligns the GENEVE_OPT guard with the sibling RAW branch, which
already gates its copy on the remaining buffer size.
Patch 2 plumbs the remaining buffer size into the flex-item desc_fn
callback (which previously took no size argument at all) and gates the
inner rte_memcpy() on it.
James Raphael Tiovalen (2):
ethdev: fix out-of-bounds write in GENEVE option conversion
ethdev: fix out-of-bounds write in flex item conversion
lib/ethdev/rte_flow.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH 1/2] ethdev: fix out-of-bounds write in GENEVE option conversion
From: James Raphael Tiovalen @ 2026-05-26 18:11 UTC (permalink / raw)
To: orika, thomas, andrew.rybchenko; +Cc: dev, stable, James Raphael Tiovalen
In-Reply-To: <20260526181159.287633-1-jamestiotio@gmail.com>
rte_flow_conv_item_spec() is documented to truncate output to the
caller-supplied buffer size. For RTE_FLOW_ITEM_TYPE_GENEVE_OPT, the
deep-copy of the variable-length option data was gated on `size > 0`
instead of `size >= off + tmp`, the form used by the sibling RAW
branch. A caller passing a buffer just large enough for the header
struct had adjacent memory clobbered by up to `option_len * 4` bytes of
option payload.
Align the GENEVE_OPT guard with the RAW one.
Fixes: 841a0445442d ("ethdev: fix GENEVE option item conversion")
Cc: stable@dpdk.org
Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
lib/ethdev/rte_flow.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index fe8f43caff..63b686ddfb 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -697,7 +697,7 @@ rte_flow_conv_item_spec(void *buf, const size_t size,
src.geneve_opt = data;
dst.geneve_opt = buf;
tmp = spec.geneve_opt->option_len << 2;
- if (size > 0 && src.geneve_opt->data) {
+ if (size >= off + tmp && src.geneve_opt->data) {
deep_src = (void *)((uintptr_t)(dst.geneve_opt + 1));
dst.geneve_opt->data = rte_memcpy(deep_src,
src.geneve_opt->data,
--
2.43.0
^ permalink raw reply related
* [PATCH 2/2] ethdev: fix out-of-bounds write in flex item conversion
From: James Raphael Tiovalen @ 2026-05-26 18:11 UTC (permalink / raw)
To: orika, thomas, andrew.rybchenko; +Cc: dev, stable, James Raphael Tiovalen
In-Reply-To: <20260526181159.287633-1-jamestiotio@gmail.com>
rte_flow_item_flex_conv() is dispatched from rte_flow_conv_copy() to
deep-copy the variable-length pattern that follows a flex item header.
The function took no size argument at all, so the trailing rte_memcpy()
of `src->length` bytes was gated only on `buf != NULL`, violating the
documented contract that output is truncated to the caller-supplied
buffer size. A caller passing a buffer just large enough for the header
struct had adjacent memory clobbered by up to 4 GiB of pattern data,
since `src->length` is uint32_t and unbounded.
Propagate the remaining buffer size `size - sz` from
rte_flow_conv_copy() into the desc_fn callback and gate the inner
memcpy on it.
Fixes: dc4d860e8a89 ("ethdev: introduce configurable flexible item")
Cc: stable@dpdk.org
Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
lib/ethdev/rte_flow.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 63b686ddfb..2744cfab2f 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -36,7 +36,7 @@ uint64_t rte_flow_dynf_metadata_mask;
struct rte_flow_desc_data {
const char *name;
size_t size;
- size_t (*desc_fn)(void *dst, const void *src);
+ size_t (*desc_fn)(void *dst, const void *src, size_t size);
};
/**
@@ -68,16 +68,17 @@ rte_flow_conv_copy(void *buf, const void *data, const size_t size,
if (buf != NULL)
rte_memcpy(buf, data, (size > sz ? sz : size));
if (rte_type && desc[type].desc_fn)
- sz += desc[type].desc_fn(size > 0 ? buf : NULL, data);
+ sz += desc[type].desc_fn(size > 0 ? buf : NULL, data,
+ size > sz ? size - sz : 0);
return sz;
}
static size_t
-rte_flow_item_flex_conv(void *buf, const void *data)
+rte_flow_item_flex_conv(void *buf, const void *data, size_t size)
{
struct rte_flow_item_flex *dst = buf;
const struct rte_flow_item_flex *src = data;
- if (buf) {
+ if (buf && size >= src->length) {
dst->pattern = rte_memcpy
((void *)((uintptr_t)(dst + 1)), src->pattern,
src->length);
--
2.43.0
^ permalink raw reply related
* Re: [PATCH V2 00/15] power: unify and improve lcore ID verification
From: Stephen Hemminger @ 2026-05-26 19:18 UTC (permalink / raw)
To: lihuisong (C)
Cc: anatoly.burakov, sivaprasad.tummala, dev, thomas, fengchengwen,
yangxingui, zhanjie9
In-Reply-To: <a7483ea3-1ad7-4bb5-91d5-fa8539331c98@huawei.com>
On Tue, 19 May 2026 21:09:47 +0800
"lihuisong (C)" <lihuisong@huawei.com> wrote:
> Thanks for giving me this AI review feedback.
> Some of them are still acceptable. will fix it in next version.
>
> Is this summary from AI generated for each patche series?
> May ask where you obtained this information about AI review?
There is automated review from CI which is done after CI tests runs pass.
But it is using older version of AGENTS.md and missing tool skills support.
I use Claude AI project with current version of AGENTS.md.
It gives the best reviews since it is willing to go look at the existing code.
Other methods just look at the patch itself.
^ permalink raw reply
* Re: [PATCH v2] app/test-pmd: add generic PROG action parser support
From: Stephen Hemminger @ 2026-05-26 19:22 UTC (permalink / raw)
To: Megha Ajmera; +Cc: bruce.richardson, cristian.dumitrescu, praveen.shetty, dev
In-Reply-To: <20260521101354.726240-1-megha.ajmera@intel.com>
On Thu, 21 May 2026 15:43:54 +0530
Megha Ajmera <megha.ajmera@intel.com> wrote:
> Add parser support for a generic PROG flow action in testpmd.
>
> The update adds CLI tokens and parsing logic for program name and
> argument tuples (name, size, value), enabling programmable action
> configuration through the flow command interface.
>
> Example flow rule:
> flow create 0 ingress pattern eth / end actions prog name my_prog
> argument name arg0 size 4 value 10 / end
>
> Signed-off-by: Megha Ajmera <megha.ajmera@intel.com>
> Signed-off-by: Praveen Shetty <praveen.shetty@intel.com>
> ---
This looks like a third attempt to parse text into rte_flow.
Not sure how this fits in and why it would be useful?
^ permalink raw reply
* Re: [PATCH v3 0/2] net/mana: add device reset support
From: Stephen Hemminger @ 2026-05-26 19:37 UTC (permalink / raw)
To: Wei Hu; +Cc: dev, longli, weh
In-Reply-To: <20260522065943.126703-1-weh@linux.microsoft.com>
On Thu, 21 May 2026 23:59:41 -0700
Wei Hu <weh@linux.microsoft.com> wrote:
> From: Wei Hu <weh@microsoft.com>
>
> Add support for handling hardware service reset events in the
> MANA driver. When the MANA kernel driver receives a hardware
> service event, it initiates a device reset and notifies userspace
> via IBV_EVENT_DEVICE_FATAL. The MANA PMD handles this by
> performing an automatic teardown and recovery sequence.
>
> The driver uses ethdev recovery events (ERR_RECOVERING,
> RECOVERY_SUCCESS, RECOVERY_FAILED) to notify upper layers of
> the reset lifecycle, and a PCI device removal event callback
> to distinguish hot-remove from service reset.
>
> Changes since v2:
> - Fixed dev_state_qsv memory leak on device removal
> - Fixed reset thread TCB/stack leak: reset_thread_active is now
> only cleared by the joiner, not the thread itself
> - Fixed second reset crash: removed reset thread join logic from
> mana_dev_close (inner function) to avoid corrupting dev_state
> when called from mana_reset_enter
> - Made reset_thread_active RTE_ATOMIC(bool) with explicit ordering
> - Added retry loop for rte_dev_event_callback_unregister on -EAGAIN
> - Initialized condvar/mutex with PTHREAD_PROCESS_SHARED since priv
> is in hugepage shared memory
> - Added re-check of dev_state after lock acquisition in
> mana_intr_handler to prevent racing with pci_remove_event_cb
> - Replaced (void *)0 with NULL in mp.c
> - Added lock ownership comment block at mana_reset_enter
> - Documented rte_dev_event_monitor_start() requirement
> - Added mana.rst documentation and release note (patch 2/2)
>
> Changes since v1:
> - Removed net/netvsc patch from this series
> - Simplified reset exit: mana_reset_exit calls
> mana_reset_exit_delay directly instead of spawning a thread
> - Added __rte_no_thread_safety_analysis annotations for clang
> - Switched to rte_thread_create_internal_control
> - Fixed declaration-after-statement style issues
> - Removed unnecessary blank lines and stale comments
>
> Wei Hu (2):
> net/mana: add device reset support
> net/mana: add documentation for device reset support
>
> doc/guides/nics/mana.rst | 33 +
> doc/guides/rel_notes/release_26_07.rst | 8 +
> drivers/net/mana/mana.c | 995 ++++++++++++++++++++++---
> drivers/net/mana/mana.h | 33 +-
> drivers/net/mana/meson.build | 2 +-
> drivers/net/mana/mp.c | 89 ++-
> drivers/net/mana/mr.c | 6 +-
> drivers/net/mana/rx.c | 24 +-
> drivers/net/mana/tx.c | 40 +-
> 9 files changed, 1123 insertions(+), 107 deletions(-)
>
AI review found a few things...
Thanks for the v3. Most of the v2 items are addressed: dev_state_qsv
is freed via mana_dev_free_resources, reset_thread_active is now
RTE_ATOMIC(bool), the joiner owns the flag, pthread mutex/cond use
PROCESS_SHARED attrs, intr_handler re-checks state under the lock,
the lock hand-off has a clear comment, and a doc + release note
patch is included.
A few items remain or were introduced by the v2->v3 changes.
Errors
1. Deadlock in dev_close_lock + EAGAIN loop on PCI-remove callback.
The v3 fix for the discarded unregister return uses a busy-wait:
if (dev->device) {
do {
ret = rte_dev_event_callback_unregister(...);
} while (ret == -EAGAIN);
}
mana_intr_uninstall runs from mana_dev_close, which runs from
mana_dev_close_lock with reset_ops_lock held. EAL returns
-EAGAIN while cb_lst->active is 1 (callback dispatching). The
callback in question is mana_pci_remove_event_cb, which itself
blocks on rte_spinlock_lock(&priv->reset_ops_lock). Result:
dev_close holds the spinlock, the callback waits for the
spinlock, the loop waits for the callback to finish -- three-way
deadlock if a hot-remove event arrives during close.
Either drop the lock before unregistering and re-take it after,
or (better) use a sleeping mutex for reset_ops_lock so the
callback can complete while close is suspended.
2. reset_ops_lock is still rte_spinlock_t held across blocking work.
Unchanged from v2: the spinlock is held through
- mana_reset_enter: rte_rcu_qsbr_check spin,
mana_dev_stop,
mana_mp_req_on_rxtx (5s IPC timeout),
mana_dev_close
- mana_reset_exit_delay: ibv_close_device, mana_pci_probe,
mana_mp_req_on_rxtx, mana_dev_start
Any other ethdev op that hits dev_ops while reset is in flight
spins for tens of seconds. Blocking calls under a spinlock are
an Error; use a properly-initialised pthread_mutex_t (you
already have PROCESS_SHARED attrs handy).
Warnings
3. Secondary process: qsbr does not actually quiesce secondary lcores.
rte_rcu_qsbr_thread_register is only called from
mana_dev_configure, which the secondary never runs. The
secondary's rx_burst/tx_burst still call thread_online/offline
against the shared qsv, but the secondary tids are unregistered,
so they are invisible to rte_rcu_qsbr_check in the primary, and
the secondary MP handler (mana_mp_reset_enter) does not call
qsbr_check at all -- it just sets db_page = NULL and munmaps.
The dev_state check at the top of secondary tx_burst is racy:
the page can be munmapped after the in-loop read of db_page but
before the doorbell write at the bottom. The "All secondary
threads are quiescent" log line in mana_mp_reset_enter is not
true.
The secondary needs a real reader-side primitive -- its own
qsbr with secondary lcore registration, or an rwlock the MP
handler takes before munmap.
4. Double-join race between dev_close_lock and mana_reset_enter.
dev_close_lock signals + joins the reset thread *before* taking
reset_ops_lock. intr_handler may fire in that window:
1. dev_close_lock: pthread_cond_signal, rte_thread_join
2. intr_handler: take lock, see ACTIVE, call reset_enter
3. reset_enter: reset_thread_active still true (step 1
hasn't cleared it yet) -> rte_thread_join
on a thread that step 1 just joined -> UB
The pre-lock signal/join was added to avoid deadlock with the
reset thread holding the lock, which is fine in itself, but the
reset_thread_active update needs to be inside that critical
section, or reset_enter has to recheck under the lock.
5. mana_dev_close (non-_lock) does not join the reset thread.
mana_dev_uninit -> mana_dev_close path doesn't go through
dev_close_lock, so if pci_remove arrives while the reset thread
is alive, mana_dev_free_resources can destroy reset_cond_mutex /
reset_cond before the thread has exited. In practice the
pci_remove callback signals RESET_FAILED and the reset thread
should exit quickly, but the ordering isn't enforced. Either
join in mana_dev_uninit, or document the assumption.
Info
6. mana_reset_exit_delay sets priv->ib_ctx = NULL even when
ibv_close_device fails:
ret = ibv_close_device(priv->ib_ctx);
priv->ib_ctx = NULL;
if (ret) { ... goto out; }
The handle is leaked if close fails. Either keep the old
pointer on failure or accept that close-failure is unrecoverable
and free it anyway with a comment.
Patch 2/2
7. The events list is a term/description pattern and should be a
definition list per DPDK RST style:
``RTE_ETH_EVENT_ERR_RECOVERING``
Reset has started.
``RTE_ETH_EVENT_RECOVERY_SUCCESS``
Device has recovered successfully.
``RTE_ETH_EVENT_RECOVERY_FAILED``
Recovery failed.
8. Convention is to fold doc + release note updates into the same
commit as the feature. Patches 1/2 and 2/2 can be squashed.
^ permalink raw reply
* Re: [PATCH v3 14/25] bus: refactor device probe
From: Stephen Hemminger @ 2026-05-26 21:38 UTC (permalink / raw)
To: David Marchand
Cc: dev, thomas, bruce.richardson, Parav Pandit, Xueming Li,
Nipun Gupta, Nikhil Agarwal, Hemant Agrawal, Sachin Saxena,
Rosen Xu, Chenbo Xia, Tomasz Duszynski, Chengwen Feng, Long Li,
Wei Hu, Kevin Laatz
In-Reply-To: <20260526085257.3148516-1-david.marchand@redhat.com>
On Tue, 26 May 2026 10:52:44 +0200
David Marchand <david.marchand@redhat.com> wrote:
> Introduce a new rte_bus_probe_device_t operation with signature
> (struct rte_driver *drv, struct rte_device *dev).
>
> Replace the existing .plug field in the struct rte_bus with .probe_device.
>
> Update all in-tree buses to use .probe_device instead of .plug.
> Each bus probe() function now calls rte_bus_find_driver() (which uses the
> match operation added in previous commit) and passes the found driver
> to bus.probe_device(driver, device).
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
FYI - deep dive (more than normal) AI review had these minor
findings. I would ignore it.
That leaves the following items from the full series review:
Patch 13 style nit: dsa_match in drivers/dma/idxd/idxd_bus.c has return type and brace on the same line, inconsistent with surrounding functions in the file. Info-level.
Patch 16 (NXP scan init): fslmc process_once = 1 is moved from immediately after the early-return check to the end of the function. If any of the new in-scan init steps fails the function returns 0 early and process_once stays 0, allowing re-scan on the next call. In current EAL flows scan() is called exactly once so this is unreachable. Worth fixing for defensive consistency; not blocking.
^ permalink raw reply
* [PATCH v4 00/27] deprecate rte_atomicNN family
From: Stephen Hemminger @ 2026-05-26 23:23 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
In-Reply-To: <20260521042043.1590536-1-stephen@networkplumber.org>
The rte_atomicNN_* family was flagged for deprecation in 2021 by
commit 3ec965b6de12 ("doc: update atomic operation deprecation")
but enforcement never landed and in-tree usage continued to grow.
This series finishes converting every remaining in-tree caller to
the C11-style rte_atomic_*_explicit() / RTE_ATOMIC() API, then
marks the legacy functions __rte_deprecated so future in-tree and
out-of-tree uses are caught at compile time.
Performance: ran the DPDK perf-tests suite (mempool, hash, stack,
ring, distributor, rcu_qsbr, etc.) on the full series; only
lib/ring showed a regression, addressed by the wrapper in patch 03.
Patch organisation
==================
01-02 EAL: drop the inline-asm fallback paths now that intrinsics
work on all platforms; reimplement rte_smp_*mb on top of
rte_atomic_thread_fence.
03-04 lib/ring and lib/bpf -- the last legacy callers in lib/.
05-25 Drivers and selftests, one patch per directory.
26 Suppress deprecation warnings in app/test/test_atomic.c,
which exercises the legacy API until it goes away.
27 Mark rte_atomicNN_* with __rte_deprecated and drop the
corresponding checkpatch grep; new uses are now caught
at compile time.
Changes since v3
================
- lib/ring: keep the existing C11 element-access code; the
earlier rewrite regressed ring_perf 20-30% on x86 with GCC's
handling of atomic_compare_exchange_weak_explicit(). v4
keeps the original structure and adds a wrapper for the one
performance-sensitive CAS.
- lib/bpf: keep the BPF_ST_ATOMIC_REG macro structure rather
than open-coding the converted callers; the macro body is
rewritten to use stdatomic.
- Compilation fixes across the driver conversions caught
during review (CAS expected-value types, format-string
specifiers, dpaax HWDEBUG path).
Targeting 26.11 rather than the next release. The driver
conversions touch many maintainers' code and several are likely
to need cycles of review/respin; a longer review window avoids
rushing contested orderings into an earlier release.
Feedback wanted
===============
- vmbus producer commit-order pattern (patch 17)
- the ring CAS GCC bug workaround might be needed on other
similar uses of ring buffers in vmbus and netvsc.
- Dekker-style seq_cst handshake in net/vhost (patch 24),
which also closes a pre-existing ordering hole on
weakly-ordered ISAs
- netvsc rndis_pending claim/timeout/clear cmpxchg orderings
(patch 15)
Stephen Hemminger (27):
eal: use intrinsics for rte_atomic on all platforms
eal: reimplement rte_smp_*mb with rte_atomic_thread_fence
ring: unify memory model on C11, remove atomic32
bpf: use C11 atomics in BPF_ST_ATOMIC_REG
net/bonding: use stdatomic
net/nbl: remove unused rte_atomic16 field
net/ena: replace use of rte_atomicNN
net/failsafe: convert to stdatomic
net/enic: do not use deprecated rte_atomic64
net/pfe: use ethdev linkstatus helpers
net/sfc: replace rte_atomic with stdatomic
crypto/ccp: replace use of rte_atomic64 with stdatomic
bus/dpaa: replace rte_atomic16 with stdatomic
drivers: replace rte_atomic16 with stdatomic
net/netvsc: replace rte_atomic32 with stdatomic
event/sw: convert from rte_atomic32 to stdatomic
bus/vmbus: convert from rte_atomic to stdatomic
common/dpaax: use stdatomic instead of rte_atomic
net/bnx2x: convert from rte_atomic32 to stdatomic
bus/fslmc: replace rte_atomic32 with stdatomic
drivers/event: replace rte_atomic32 in selftests
net/hinic: replace rte_atomic32 with stdatomic
net/txgbe: replace rte_atomic32 with stdatomic
net/vhost: use stdatomic instead of rte_atomic32
vdpa/ifc: replace rte_atomic32 with stdatomic
test/atomic: suppress deprecation warnings for legacy APIs
eal: mark rte_atomicNN as deprecated
app/test/test_atomic.c | 12 +
devtools/checkpatches.sh | 16 -
doc/guides/rel_notes/deprecation.rst | 12 +-
doc/guides/rel_notes/release_26_07.rst | 4 +
drivers/bus/dpaa/base/qbman/qman.c | 9 +-
drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c | 10 +-
drivers/bus/fslmc/portal/dpaa2_hw_dpci.c | 10 +-
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 12 +-
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 8 +-
drivers/bus/fslmc/qbman/include/compat.h | 21 +-
drivers/bus/vmbus/private.h | 2 +-
drivers/bus/vmbus/vmbus_bufring.c | 39 ++-
drivers/common/dpaax/compat.h | 21 +-
drivers/crypto/ccp/ccp_crypto.c | 11 +-
drivers/crypto/ccp/ccp_crypto.h | 2 +-
drivers/crypto/ccp/ccp_dev.c | 10 +-
drivers/crypto/ccp/ccp_dev.h | 4 +-
drivers/event/dpaa2/dpaa2_eventdev_selftest.c | 26 +-
drivers/event/dpaa2/dpaa2_hw_dpcon.c | 11 +-
drivers/event/octeontx/ssovf_evdev_selftest.c | 61 ++--
drivers/event/sw/sw_evdev.c | 8 +-
drivers/event/sw/sw_evdev.h | 4 +-
drivers/event/sw/sw_evdev_worker.c | 16 +-
drivers/net/bnx2x/bnx2x.c | 6 +-
drivers/net/bnx2x/bnx2x.h | 2 +-
drivers/net/bnx2x/ecore_sp.c | 6 +-
drivers/net/bonding/eth_bond_8023ad_private.h | 6 +-
drivers/net/bonding/rte_eth_bond_8023ad.c | 35 +-
drivers/net/ena/base/ena_plat_dpdk.h | 14 +-
drivers/net/ena/ena_ethdev.c | 21 +-
drivers/net/ena/ena_ethdev.h | 7 +-
drivers/net/enic/enic.h | 6 +-
drivers/net/enic/enic_compat.h | 1 -
drivers/net/enic/enic_main.c | 17 +-
drivers/net/enic/enic_rxtx.c | 14 +-
drivers/net/enic/enic_rxtx_vec_avx2.c | 4 +-
drivers/net/failsafe/failsafe_ops.c | 12 +-
drivers/net/failsafe/failsafe_private.h | 29 +-
drivers/net/failsafe/failsafe_rxtx.c | 2 +-
drivers/net/hinic/base/hinic_compat.h | 2 +-
drivers/net/hinic/base/hinic_pmd_hwdev.c | 24 +-
drivers/net/hinic/base/hinic_pmd_hwdev.h | 4 +-
drivers/net/nbl/nbl_hw/nbl_resource.h | 1 -
drivers/net/netvsc/hn_rndis.c | 28 +-
drivers/net/netvsc/hn_rxtx.c | 12 +-
drivers/net/netvsc/hn_var.h | 6 +-
drivers/net/pfe/pfe_ethdev.c | 32 +-
drivers/net/sfc/sfc.c | 9 +-
drivers/net/sfc/sfc.h | 4 +-
drivers/net/sfc/sfc_port.c | 7 +-
drivers/net/sfc/sfc_stats.h | 2 +-
drivers/net/txgbe/base/txgbe_mng.c | 4 +-
drivers/net/txgbe/base/txgbe_type.h | 2 +-
drivers/net/vhost/rte_eth_vhost.c | 103 +++---
drivers/vdpa/ifc/ifcvf_vdpa.c | 37 +--
lib/bpf/bpf_exec.c | 13 +-
lib/eal/arm/include/rte_atomic_32.h | 10 -
lib/eal/arm/include/rte_atomic_64.h | 10 -
lib/eal/include/generic/rte_atomic.h | 306 +++++-------------
lib/eal/include/rte_common.h | 2 +
lib/eal/loongarch/include/rte_atomic.h | 10 -
lib/eal/ppc/include/rte_atomic.h | 179 ----------
lib/eal/riscv/include/rte_atomic.h | 10 -
lib/eal/x86/include/rte_atomic.h | 205 +-----------
lib/eal/x86/include/rte_atomic_32.h | 188 -----------
lib/eal/x86/include/rte_atomic_64.h | 157 ---------
lib/ring/meson.build | 2 +-
lib/ring/rte_ring_c11_pvt.h | 75 ++---
lib/ring/rte_ring_elem_pvt.h | 125 ++++++-
..._ring_generic_pvt.h => rte_ring_x86_pvt.h} | 61 +---
lib/ring/soring.c | 15 +-
71 files changed, 667 insertions(+), 1489 deletions(-)
rename lib/ring/{rte_ring_generic_pvt.h => rte_ring_x86_pvt.h} (60%)
--
2.53.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox