mempool cache change

mempool cache change

[Kishore Padmanabha]

[-- Attachment #1.1: Type: text/plain, Size: 738 bytes --]

Hi Thomas,



The recent change done as part of BUG ID: 1027, with commit id
https://git.dpdk.org/dpdk/commit/?id=f5e1310f16e0909e7e7f71807123644c63b23cba

Is causing performance drop for the bnxt NIC in 26.07 release.



The smaller, shallower cache increases the miss rate, causing refills to
hit the shared ring more frequently.
In the old code, the cache only touched the shared ring when it climbed
past 768 instead of 512. That extra 256 elements of headroom allowed
significantly more frees to remain local.



Could we make the change in config/rte_config.h for
RTE_MEMPOOL_CACHE_MAX_SIZE to be 768 instead of 512. Since we use
–mbcache=512 as argument for the performance tests.



Thanks,

Kishore

[-- Attachment #1.2: Type: text/html, Size: 2685 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5493 bytes --]

RE: mempool cache change

[Morten Brørup]

Hi Kishore,

For your testing purposes, please follow the guidance provided to Wisam Jaddo:
https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35F65964@smartserver.smartshare.dk/

Thank you for testing!

Venlig hilsen / Kind regards,
-Morten Brørup

From: Kishore Padmanabha [mailto:kishore.padmanabha@broadcom.com] 
Sent: Tuesday, 14 July 2026 18.49
To: Thomas Monjalon; Morten Brørup
Cc: dev@dpdk.org
Subject: mempool cache change

Hi Thomas,
 
The recent change done as part of BUG ID: 1027, with commit id  https://git.dpdk.org/dpdk/commit/?id=f5e1310f16e0909e7e7f71807123644c63b23cba
Is causing performance drop for the bnxt NIC in 26.07 release.
 
The smaller, shallower cache increases the miss rate, causing refills to hit the shared ring more frequently. 
In the old code, the cache only touched the shared ring when it climbed past 768 instead of 512. That extra 256 elements of headroom allowed significantly more frees to remain local.
 
Could we make the change in config/rte_config.h for RTE_MEMPOOL_CACHE_MAX_SIZE to be 768 instead of 512. Since we use –mbcache=512 as argument for the performance tests.
 
Thanks,
Kishore
 
 

Re: mempool cache change

[fengchengwen]

On 7/15/2026 4:08 AM, Morten Brørup wrote:
> Hi Kishore,
> 
> For your testing purposes, please follow the guidance provided to Wisam Jaddo:
> https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35F65964@smartserver.smartshare.dk/

We need to recompile in this case. We should try to avoid recompilation.
I think it is necessary to adjust RTE_MEMPOOL_CACHE_MAX_SIZE to 768 as a
default configuration.

Thanks

> 
> Thank you for testing!
> 
> Venlig hilsen / Kind regards,
> -Morten Brørup
> 
> From: Kishore Padmanabha [mailto:kishore.padmanabha@broadcom.com] 
> Sent: Tuesday, 14 July 2026 18.49
> To: Thomas Monjalon; Morten Brørup
> Cc: dev@dpdk.org
> Subject: mempool cache change
> 
> Hi Thomas,
>  
> The recent change done as part of BUG ID: 1027, with commit id  https://git.dpdk.org/dpdk/commit/?id=f5e1310f16e0909e7e7f71807123644c63b23cba
> Is causing performance drop for the bnxt NIC in 26.07 release.
>  
> The smaller, shallower cache increases the miss rate, causing refills to hit the shared ring more frequently. 
> In the old code, the cache only touched the shared ring when it climbed past 768 instead of 512. That extra 256 elements of headroom allowed significantly more frees to remain local.
>  
> Could we make the change in config/rte_config.h for RTE_MEMPOOL_CACHE_MAX_SIZE to be 768 instead of 512. Since we use –mbcache=512 as argument for the performance tests.
>  
> Thanks,
> Kishore
>  
>  

RE: mempool cache change

[Morten Brørup]

> From: fengchengwen [mailto:fengchengwen@huawei.com]
> Sent: Wednesday, 15 July 2026 10.12
> 
> On 7/15/2026 4:08 AM, Morten Brørup wrote:
> > Hi Kishore,
> >
> > For your testing purposes, please follow the guidance provided to
> Wisam Jaddo:
> >
> https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35F65964@smart
> server.smartshare.dk/
> 
> We need to recompile in this case. We should try to avoid
> recompilation.
> I think it is necessary to adjust RTE_MEMPOOL_CACHE_MAX_SIZE to 768 as
> a default configuration.

Changing RTE_MEMPOOL_CACHE_MAX_SIZE breaks both the API and the ABI; so it has to be done when building locally, where API/ABI breakage is acceptable.

For DPDK 26.11, where API/ABI breakage is acceptable, we can discuss increasing the default from 512 to a higher value. I do have some input to that discussion, but let's postpone it until after DPDK 26.07 has been released.

> 
> Thanks
> 
> >
> > Thank you for testing!
> >
> > Venlig hilsen / Kind regards,
> > -Morten Brørup
> >
> > From: Kishore Padmanabha [mailto:kishore.padmanabha@broadcom.com]
> > Sent: Tuesday, 14 July 2026 18.49
> > To: Thomas Monjalon; Morten Brørup
> > Cc: dev@dpdk.org
> > Subject: mempool cache change
> >
> > Hi Thomas,
> >
> > The recent change done as part of BUG ID: 1027, with commit id
> https://git.dpdk.org/dpdk/commit/?id=f5e1310f16e0909e7e7f71807123644c63
> b23cba
> > Is causing performance drop for the bnxt NIC in 26.07 release.
> >
> > The smaller, shallower cache increases the miss rate, causing refills
> to hit the shared ring more frequently.
> > In the old code, the cache only touched the shared ring when it
> climbed past 768 instead of 512. That extra 256 elements of headroom
> allowed significantly more frees to remain local.
> >
> > Could we make the change in config/rte_config.h for
> RTE_MEMPOOL_CACHE_MAX_SIZE to be 768 instead of 512. Since we use –
> mbcache=512 as argument for the performance tests.
> >
> > Thanks,
> > Kishore
> >
> >

Re: mempool cache change

[Kishore Padmanabha]

[-- Attachment #1.1: Type: text/plain, Size: 2473 bytes --]

On Wed, Jul 15, 2026 at 5:02 AM Morten Brørup <mb@smartsharesystems.com>
wrote:

> > From: fengchengwen [mailto:fengchengwen@huawei.com]
> > Sent: Wednesday, 15 July 2026 10.12
> >
> > On 7/15/2026 4:08 AM, Morten Brørup wrote:
> > > Hi Kishore,
> > >
> > > For your testing purposes, please follow the guidance provided to
> > Wisam Jaddo:
> > >
> > https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35F65964@smart
> > server.smartshare.dk/
> >
> > We need to recompile in this case. We should try to avoid
> > recompilation.
> > I think it is necessary to adjust RTE_MEMPOOL_CACHE_MAX_SIZE to 768 as
> > a default configuration.
>
> Changing RTE_MEMPOOL_CACHE_MAX_SIZE breaks both the API and the ABI; so it
> has to be done when building locally, where API/ABI breakage is acceptable.
>
> For DPDK 26.11, where API/ABI breakage is acceptable, we can discuss
> increasing the default from 512 to a higher value. I do have some input to
> that discussion, but let's postpone it until after DPDK 26.07 has been
> released.
>
We should increase this value to avoid performance degradation, as users
may not realize they need to change it.  We do not have do it right away
for 26.07 release but we should it right after the release.

>
> >
> > Thanks
> >
> > >
> > > Thank you for testing!
> > >
> > > Venlig hilsen / Kind regards,
> > > -Morten Brørup
> > >
> > > From: Kishore Padmanabha [mailto:kishore.padmanabha@broadcom.com]
> > > Sent: Tuesday, 14 July 2026 18.49
> > > To: Thomas Monjalon; Morten Brørup
> > > Cc: dev@dpdk.org
> > > Subject: mempool cache change
> > >
> > > Hi Thomas,
> > >
> > > The recent change done as part of BUG ID: 1027, with commit id
> > https://git.dpdk.org/dpdk/commit/?id=f5e1310f16e0909e7e7f71807123644c63
> > b23cba
> > > Is causing performance drop for the bnxt NIC in 26.07 release.
> > >
> > > The smaller, shallower cache increases the miss rate, causing refills
> > to hit the shared ring more frequently.
> > > In the old code, the cache only touched the shared ring when it
> > climbed past 768 instead of 512. That extra 256 elements of headroom
> > allowed significantly more frees to remain local.
> > >
> > > Could we make the change in config/rte_config.h for
> > RTE_MEMPOOL_CACHE_MAX_SIZE to be 768 instead of 512. Since we use –
> > mbcache=512 as argument for the performance tests.
> > >
> > > Thanks,
> > > Kishore
> > >
> > >
>
>

[-- Attachment #1.2: Type: text/html, Size: 3797 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5493 bytes --]

Re: mempool cache change

[Bruce Richardson]

On Wed, Jul 15, 2026 at 11:12:59AM -0400, Kishore Padmanabha wrote:
>    On Wed, Jul 15, 2026 at 5:02 AM Morten Brørup
>    <[1]mb@smartsharesystems.com> wrote:
> 
>      > From: fengchengwen [mailto:[2]fengchengwen@huawei.com]
>      > Sent: Wednesday, 15 July 2026 10.12
>      >
>      > On 7/15/2026 4:08 AM, Morten Brørup wrote:
>      > > Hi Kishore,
>      > >
>      > > For your testing purposes, please follow the guidance provided
>      to
>      > Wisam Jaddo:
>      > >
>      >
>      [3]https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35F65964
>      @smart
>      > [4]server.smartshare.dk/
>      >
>      > We need to recompile in this case. We should try to avoid
>      > recompilation.
>      > I think it is necessary to adjust RTE_MEMPOOL_CACHE_MAX_SIZE to
>      768 as
>      > a default configuration.
>      Changing RTE_MEMPOOL_CACHE_MAX_SIZE breaks both the API and the ABI;
>      so it has to be done when building locally, where API/ABI breakage
>      is acceptable.
>      For DPDK 26.11, where API/ABI breakage is acceptable, we can discuss
>      increasing the default from 512 to a higher value. I do have some
>      input to that discussion, but let's postpone it until after DPDK
>      26.07 has been released.
> 
>    We should increase this value to avoid performance degradation, as
>    users may not realize they need to change it.  We do not have do it
>    right away for 26.07 release but we should it right after the
>    release.
> 

Out of interest, does adjusting the 50% fill/flush threshold to be a 75%
one, i.e. fill to 75% rather than 50%, flush to 25% rather than 50%, help at
all? Draft patch below, can you test it quickly, perhaps?

/Bruce

diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h
index 50d958c7c6..2526a23903 100644
--- a/lib/mempool/rte_mempool.h
+++ b/lib/mempool/rte_mempool.h
@@ -1417,6 +1417,7 @@ rte_mempool_do_generic_put(struct rte_mempool *mp, void * const *obj_table,
                           unsigned int n, struct rte_mempool_cache *cache)
 {
        void **cache_objs;
+       uint32_t quarter, flush;
 
        /* No cache provided? */
        if (unlikely(cache == NULL))
@@ -1426,30 +1427,39 @@ rte_mempool_do_generic_put(struct rte_mempool *mp, void * const *obj_table,
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_bulk, 1);
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_objs, n);
 
+       /* A quarter (25%) of the cache size, computed with a shift to avoid
+        * a divide. Draining the cache down to this level on overflow leaves
+        * room for a burst of up to (size - quarter), i.e. three quarters
+        * (75%), of the cache size.
+        */
+       quarter = cache->size >> 2;
+
        __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
-       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
+       __rte_assume(quarter <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
        __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
        __rte_assume(cache->len <= cache->size);
        if (likely(cache->len + n <= cache->size)) {
                /* Sufficient room in the cache for the objects. */
                cache_objs = &cache->objs[cache->len];
                cache->len += n;
-       } else if (n <= cache->size / 2) {
+       } else if (n <= cache->size - quarter) {
                /*
                 * The number of objects is within the cache bounce buffer limit,
                 * but - as detected by the comparison above - the cache has
                 * insufficient room for them.
                 * Flush the cache to the backend to make room for the objects;
-                * flush (size / 2) objects from the bottom of the cache, where
-                * objects are less hot, and move down the remaining objects, which
-                * are more hot, from the upper half of the cache.
+                * flush objects from the bottom of the cache, where objects are
+                * less hot, draining it down to a quarter (25%) of its size, and
+                * move down the remaining quarter of objects, which are more hot,
+                * from the upper part of the cache.
                 */
-               __rte_assume(cache->len > cache->size / 2);
-               rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0], cache->size / 2);
-               rte_memcpy(&cache->objs[0], &cache->objs[cache->size / 2],
-                               sizeof(void *) * (cache->len - cache->size / 2));
-               cache_objs = &cache->objs[cache->len - cache->size / 2];
-               cache->len = cache->len - cache->size / 2 + n;
+               __rte_assume(cache->len > quarter);
+               flush = cache->len - quarter;
+               rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0], flush);
+               rte_memcpy(&cache->objs[0], &cache->objs[flush],
+                               sizeof(void *) * quarter);
+               cache_objs = &cache->objs[quarter];
+               cache->len = quarter + n;
        } else {
                /* The request itself is too big for the cache. */
                goto driver_enqueue_stats_incremented;
@@ -1557,6 +1567,7 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
        int ret;
        unsigned int remaining;
        uint32_t index, len;
+       uint32_t quarter, fill;
        void **cache_objs;
 
        /* No cache provided? */
@@ -1592,13 +1603,23 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
        for (index = 0; index < len; index++)
                *obj_table++ = *--cache_objs;
 
+       /* A quarter (25%) of the cache size, computed with a shift to avoid
+        * a divide; 'fill' is the complementary three quarters (75%), which
+        * is the amount fetched from the backend to fill the cache up to 75%,
+        * and also the burst limit for this bounce buffer (since the cache
+        * was just fully drained above, up to 'fill' objects can be filled
+        * and handed back to the caller in one go).
+        */
+       quarter = cache->size >> 2;
+       fill = cache->size - quarter;
+
        /* Dequeue below would exceed the cache bounce buffer limit? */
-       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
-       if (unlikely(remaining > cache->size / 2))
+       __rte_assume(fill <= RTE_MEMPOOL_CACHE_MAX_SIZE);
+       if (unlikely(remaining > fill))
                goto driver_dequeue;
 
-       /* Fill the cache from the backend; fetch (size / 2) objects. */
-       ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, cache->size / 2);
+       /* Fill the cache from the backend, up to 75% of its size. */
+       ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, fill);
        if (unlikely(ret < 0)) {
                /*
                 * We are buffer constrained, and not able to fetch all that.
@@ -1612,11 +1633,10 @@ rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk, 1);
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_objs, n);
 
-       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
-       __rte_assume(remaining <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
-       __rte_assume(remaining <= cache->size / 2);
-       cache_objs = &cache->objs[cache->size / 2];
-       cache->len = cache->size / 2 - remaining;
+       __rte_assume(fill <= RTE_MEMPOOL_CACHE_MAX_SIZE);
+       __rte_assume(remaining <= fill);
+       cache_objs = &cache->objs[fill];
+       cache->len = fill - remaining;
        for (index = 0; index < remaining; index++)
                *obj_table++ = *--cache_objs;

RE: mempool cache change

[Kishore Padmanabha]

[-- Attachment #1: Type: text/plain, Size: 8443 bytes --]

Hi Bruce,

The below patch works fine for us. We tested all the different packet sizes.
Thanks for the patch. Do you want to push this patch since it is not
changing the ABI/API?

Rgds,
Kishore

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com>
Sent: Thursday, July 16, 2026 5:57 AM
To: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Cc: Morten Brørup <mb@smartsharesystems.com>; fengchengwen
<fengchengwen@huawei.com>; Thomas Monjalon <thomas@monjalon.net>;
dev@dpdk.org; Wisam Jaddo <wisamm@nvidia.com>
Subject: Re: mempool cache change

On Wed, Jul 15, 2026 at 11:12:59AM -0400, Kishore Padmanabha wrote:
>    On Wed, Jul 15, 2026 at 5:02 AM Morten Brørup
>    <[1]mb@smartsharesystems.com> wrote:
>
>      > From: fengchengwen [mailto:[2]fengchengwen@huawei.com]
>      > Sent: Wednesday, 15 July 2026 10.12
>      >
>      > On 7/15/2026 4:08 AM, Morten Brørup wrote:
>      > > Hi Kishore,
>      > >
>      > > For your testing purposes, please follow the guidance provided
>      to
>      > Wisam Jaddo:
>      > >
>      >
>      [3]https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35F65964
>      @smart
>      > [4]server.smartshare.dk/
>      >
>      > We need to recompile in this case. We should try to avoid
>      > recompilation.
>      > I think it is necessary to adjust RTE_MEMPOOL_CACHE_MAX_SIZE to
>      768 as
>      > a default configuration.
>      Changing RTE_MEMPOOL_CACHE_MAX_SIZE breaks both the API and the ABI;
>      so it has to be done when building locally, where API/ABI breakage
>      is acceptable.
>      For DPDK 26.11, where API/ABI breakage is acceptable, we can discuss
>      increasing the default from 512 to a higher value. I do have some
>      input to that discussion, but let's postpone it until after DPDK
>      26.07 has been released.
>
>    We should increase this value to avoid performance degradation, as
>    users may not realize they need to change it.  We do not have do it
>    right away for 26.07 release but we should it right after the
>    release.
>

Out of interest, does adjusting the 50% fill/flush threshold to be a 75%
one, i.e. fill to 75% rather than 50%, flush to 25% rather than 50%, help at
all? Draft patch below, can you test it quickly, perhaps?

/Bruce

diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h index
50d958c7c6..2526a23903 100644
--- a/lib/mempool/rte_mempool.h
+++ b/lib/mempool/rte_mempool.h
@@ -1417,6 +1417,7 @@ rte_mempool_do_generic_put(struct rte_mempool *mp,
void * const *obj_table,
                           unsigned int n, struct rte_mempool_cache *cache)
{
        void **cache_objs;
+       uint32_t quarter, flush;

        /* No cache provided? */
        if (unlikely(cache == NULL))
@@ -1426,30 +1427,39 @@ rte_mempool_do_generic_put(struct rte_mempool *mp,
void * const *obj_table,
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_bulk, 1);
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_objs, n);

+       /* A quarter (25%) of the cache size, computed with a shift to avoid
+        * a divide. Draining the cache down to this level on overflow
leaves
+        * room for a burst of up to (size - quarter), i.e. three quarters
+        * (75%), of the cache size.
+        */
+       quarter = cache->size >> 2;
+
        __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
-       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
+       __rte_assume(quarter <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
        __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
        __rte_assume(cache->len <= cache->size);
        if (likely(cache->len + n <= cache->size)) {
                /* Sufficient room in the cache for the objects. */
                cache_objs = &cache->objs[cache->len];
                cache->len += n;
-       } else if (n <= cache->size / 2) {
+       } else if (n <= cache->size - quarter) {
                /*
                 * The number of objects is within the cache bounce buffer
limit,
                 * but - as detected by the comparison above - the cache has
                 * insufficient room for them.
                 * Flush the cache to the backend to make room for the
objects;
-                * flush (size / 2) objects from the bottom of the cache,
where
-                * objects are less hot, and move down the remaining
objects, which
-                * are more hot, from the upper half of the cache.
+                * flush objects from the bottom of the cache, where objects
are
+                * less hot, draining it down to a quarter (25%) of its
size, and
+                * move down the remaining quarter of objects, which are
more hot,
+                * from the upper part of the cache.
                 */
-               __rte_assume(cache->len > cache->size / 2);
-               rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0],
cache->size / 2);
-               rte_memcpy(&cache->objs[0], &cache->objs[cache->size / 2],
-                               sizeof(void *) * (cache->len - cache->size /
2));
-               cache_objs = &cache->objs[cache->len - cache->size / 2];
-               cache->len = cache->len - cache->size / 2 + n;
+               __rte_assume(cache->len > quarter);
+               flush = cache->len - quarter;
+               rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0], flush);
+               rte_memcpy(&cache->objs[0], &cache->objs[flush],
+                               sizeof(void *) * quarter);
+               cache_objs = &cache->objs[quarter];
+               cache->len = quarter + n;
        } else {
                /* The request itself is too big for the cache. */
                goto driver_enqueue_stats_incremented; @@ -1557,6 +1567,7 @@
rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
        int ret;
        unsigned int remaining;
        uint32_t index, len;
+       uint32_t quarter, fill;
        void **cache_objs;

        /* No cache provided? */
@@ -1592,13 +1603,23 @@ rte_mempool_do_generic_get(struct rte_mempool *mp,
void **obj_table,
        for (index = 0; index < len; index++)
                *obj_table++ = *--cache_objs;

+       /* A quarter (25%) of the cache size, computed with a shift to avoid
+        * a divide; 'fill' is the complementary three quarters (75%), which
+        * is the amount fetched from the backend to fill the cache up to
75%,
+        * and also the burst limit for this bounce buffer (since the cache
+        * was just fully drained above, up to 'fill' objects can be filled
+        * and handed back to the caller in one go).
+        */
+       quarter = cache->size >> 2;
+       fill = cache->size - quarter;
+
        /* Dequeue below would exceed the cache bounce buffer limit? */
-       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
-       if (unlikely(remaining > cache->size / 2))
+       __rte_assume(fill <= RTE_MEMPOOL_CACHE_MAX_SIZE);
+       if (unlikely(remaining > fill))
                goto driver_dequeue;

-       /* Fill the cache from the backend; fetch (size / 2) objects. */
-       ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, cache->size /
2);
+       /* Fill the cache from the backend, up to 75% of its size. */
+       ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, fill);
        if (unlikely(ret < 0)) {
                /*
                 * We are buffer constrained, and not able to fetch all
that.
@@ -1612,11 +1633,10 @@ rte_mempool_do_generic_get(struct rte_mempool *mp,
void **obj_table,
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk, 1);
        RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_objs, n);

-       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
-       __rte_assume(remaining <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
-       __rte_assume(remaining <= cache->size / 2);
-       cache_objs = &cache->objs[cache->size / 2];
-       cache->len = cache->size / 2 - remaining;
+       __rte_assume(fill <= RTE_MEMPOOL_CACHE_MAX_SIZE);
+       __rte_assume(remaining <= fill);
+       cache_objs = &cache->objs[fill];
+       cache->len = fill - remaining;
        for (index = 0; index < remaining; index++)
                *obj_table++ = *--cache_objs;

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5493 bytes --]

RE: mempool cache change

[Morten Brørup]

> From: Kishore Padmanabha [mailto:kishore.padmanabha@broadcom.com]
> Sent: Friday, 17 July 2026 19.10
> 
> Hi Bruce,
> 
> The below patch works fine for us. We tested all the different packet
> sizes.
> Thanks for the patch. Do you want to push this patch since it is not
> changing the ABI/API?

Too late in the release process.

Let's postpone the discussion for DPDK 26.11, where API/ABI breakage is allowed.

I'm not strongly opposed to Bruce's algorithm, targeting a fill level of 25 % from the edges and flushing/refilling up to 75 % of the cache when necessary. It does have its advantages for some mempool access patterns (which are not exotic).
I just prefer the current algorithm, targeting a fill level of 50 % and only flushing/refilling up to 50 % of the cache when necessary. It performs better at random get/put access patterns, and the backend transactions are smaller.

For DPDK 26.11, where we can break the API/ABI, we can simply double RTE_MEMPOOL_CACHE_MAX_SIZE to 1024, to compensate for reducing the effective cache size from 150 % to 100 %.
The mempool cache objs array will no longer be [RTE_MEMPOOL_CACHE_MAX_SIZE * 2], but only [RTE_MEMPOOL_CACHE_MAX_SIZE], so doubling RTE_MEMPOOL_CACHE_MAX_SIZE will not increase the memory footprint, but allow using a cache size up to 1024.

Please also note that the current implementation is carefully designed to keep the transfers to/from the mempool backend CPU cache aligned (assuming cache->size is 2^N and large enough).
Refer to the parameters passed to rte_mempool_ops_enqueue/dequeue_bulk().
E.g. with mempool cache size 256, backend transfers are 128 objects, 16 full cache lines.
Using CPU cache aligned transfers has a few advantages:
- There are no cache line ownership issues across different CPU cores repeatedly accessing the backend.
- The mempool backend drivers can be performance optimized for transferring full CPU cache lines. (Both source and destination addresses, and number of objects copied, are CPU cache aligned. Assuming all transfers go via the mempool cache.)

These details should be fine tuned in the implementation, if we do proceed with Bruce's algorithm.

-Morten

> 
> Rgds,
> Kishore
> 
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Thursday, July 16, 2026 5:57 AM
> To: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
> 
> On Wed, Jul 15, 2026 at 11:12:59AM -0400, Kishore Padmanabha wrote:
> >    On Wed, Jul 15, 2026 at 5:02 AM Morten Brørup
> >    <[1]mb@smartsharesystems.com> wrote:
> >
> >      > From: fengchengwen [mailto:[2]fengchengwen@huawei.com]
> >      > Sent: Wednesday, 15 July 2026 10.12
> >      >
> >      > On 7/15/2026 4:08 AM, Morten Brørup wrote:
> >      > > Hi Kishore,
> >      > >
> >      > > For your testing purposes, please follow the guidance
> provided
> >      to
> >      > Wisam Jaddo:
> >      > >
> >      >
> >
> [3]https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35F65964
> >      @smart
> >      > [4]server.smartshare.dk/
> >      >
> >      > We need to recompile in this case. We should try to avoid
> >      > recompilation.
> >      > I think it is necessary to adjust RTE_MEMPOOL_CACHE_MAX_SIZE
> to
> >      768 as
> >      > a default configuration.
> >      Changing RTE_MEMPOOL_CACHE_MAX_SIZE breaks both the API and the
> ABI;
> >      so it has to be done when building locally, where API/ABI
> breakage
> >      is acceptable.
> >      For DPDK 26.11, where API/ABI breakage is acceptable, we can
> discuss
> >      increasing the default from 512 to a higher value. I do have
> some
> >      input to that discussion, but let's postpone it until after DPDK
> >      26.07 has been released.
> >
> >    We should increase this value to avoid performance degradation, as
> >    users may not realize they need to change it.  We do not have do
> it
> >    right away for 26.07 release but we should it right after the
> >    release.
> >
> 
> Out of interest, does adjusting the 50% fill/flush threshold to be a
> 75%
> one, i.e. fill to 75% rather than 50%, flush to 25% rather than 50%,
> help at
> all? Draft patch below, can you test it quickly, perhaps?
> 
> /Bruce
> 
> diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h
> index
> 50d958c7c6..2526a23903 100644
> --- a/lib/mempool/rte_mempool.h
> +++ b/lib/mempool/rte_mempool.h
> @@ -1417,6 +1417,7 @@ rte_mempool_do_generic_put(struct rte_mempool
> *mp,
> void * const *obj_table,
>                            unsigned int n, struct rte_mempool_cache
> *cache)
> {
>         void **cache_objs;
> +       uint32_t quarter, flush;
> 
>         /* No cache provided? */
>         if (unlikely(cache == NULL))
> @@ -1426,30 +1427,39 @@ rte_mempool_do_generic_put(struct rte_mempool
> *mp,
> void * const *obj_table,
>         RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_bulk, 1);
>         RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_objs, n);
> 
> +       /* A quarter (25%) of the cache size, computed with a shift to
> avoid
> +        * a divide. Draining the cache down to this level on overflow
> leaves
> +        * room for a burst of up to (size - quarter), i.e. three
> quarters
> +        * (75%), of the cache size.
> +        */
> +       quarter = cache->size >> 2;
> +
>         __rte_assume(cache->size <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> -       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE /
> 2);
> +       __rte_assume(quarter <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
>         __rte_assume(cache->len <= RTE_MEMPOOL_CACHE_MAX_SIZE);
>         __rte_assume(cache->len <= cache->size);
>         if (likely(cache->len + n <= cache->size)) {
>                 /* Sufficient room in the cache for the objects. */
>                 cache_objs = &cache->objs[cache->len];
>                 cache->len += n;
> -       } else if (n <= cache->size / 2) {
> +       } else if (n <= cache->size - quarter) {
>                 /*
>                  * The number of objects is within the cache bounce
> buffer
> limit,
>                  * but - as detected by the comparison above - the
> cache has
>                  * insufficient room for them.
>                  * Flush the cache to the backend to make room for the
> objects;
> -                * flush (size / 2) objects from the bottom of the
> cache,
> where
> -                * objects are less hot, and move down the remaining
> objects, which
> -                * are more hot, from the upper half of the cache.
> +                * flush objects from the bottom of the cache, where
> objects
> are
> +                * less hot, draining it down to a quarter (25%) of its
> size, and
> +                * move down the remaining quarter of objects, which
> are
> more hot,
> +                * from the upper part of the cache.
>                  */
> -               __rte_assume(cache->len > cache->size / 2);
> -               rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0],
> cache->size / 2);
> -               rte_memcpy(&cache->objs[0], &cache->objs[cache->size /
> 2],
> -                               sizeof(void *) * (cache->len - cache-
> >size /
> 2));
> -               cache_objs = &cache->objs[cache->len - cache->size /
> 2];
> -               cache->len = cache->len - cache->size / 2 + n;
> +               __rte_assume(cache->len > quarter);
> +               flush = cache->len - quarter;
> +               rte_mempool_ops_enqueue_bulk(mp, &cache->objs[0],
> flush);
> +               rte_memcpy(&cache->objs[0], &cache->objs[flush],
> +                               sizeof(void *) * quarter);
> +               cache_objs = &cache->objs[quarter];
> +               cache->len = quarter + n;
>         } else {
>                 /* The request itself is too big for the cache. */
>                 goto driver_enqueue_stats_incremented; @@ -1557,6
> +1567,7 @@
> rte_mempool_do_generic_get(struct rte_mempool *mp, void **obj_table,
>         int ret;
>         unsigned int remaining;
>         uint32_t index, len;
> +       uint32_t quarter, fill;
>         void **cache_objs;
> 
>         /* No cache provided? */
> @@ -1592,13 +1603,23 @@ rte_mempool_do_generic_get(struct rte_mempool
> *mp,
> void **obj_table,
>         for (index = 0; index < len; index++)
>                 *obj_table++ = *--cache_objs;
> 
> +       /* A quarter (25%) of the cache size, computed with a shift to
> avoid
> +        * a divide; 'fill' is the complementary three quarters (75%),
> which
> +        * is the amount fetched from the backend to fill the cache up
> to
> 75%,
> +        * and also the burst limit for this bounce buffer (since the
> cache
> +        * was just fully drained above, up to 'fill' objects can be
> filled
> +        * and handed back to the caller in one go).
> +        */
> +       quarter = cache->size >> 2;
> +       fill = cache->size - quarter;
> +
>         /* Dequeue below would exceed the cache bounce buffer limit? */
> -       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE /
> 2);
> -       if (unlikely(remaining > cache->size / 2))
> +       __rte_assume(fill <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> +       if (unlikely(remaining > fill))
>                 goto driver_dequeue;
> 
> -       /* Fill the cache from the backend; fetch (size / 2) objects.
> */
> -       ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, cache->size
> /
> 2);
> +       /* Fill the cache from the backend, up to 75% of its size. */
> +       ret = rte_mempool_ops_dequeue_bulk(mp, cache->objs, fill);
>         if (unlikely(ret < 0)) {
>                 /*
>                  * We are buffer constrained, and not able to fetch all
> that.
> @@ -1612,11 +1633,10 @@ rte_mempool_do_generic_get(struct rte_mempool
> *mp,
> void **obj_table,
>         RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_bulk, 1);
>         RTE_MEMPOOL_CACHE_STAT_ADD(cache, get_success_objs, n);
> 
> -       __rte_assume(cache->size / 2 <= RTE_MEMPOOL_CACHE_MAX_SIZE /
> 2);
> -       __rte_assume(remaining <= RTE_MEMPOOL_CACHE_MAX_SIZE / 2);
> -       __rte_assume(remaining <= cache->size / 2);
> -       cache_objs = &cache->objs[cache->size / 2];
> -       cache->len = cache->size / 2 - remaining;
> +       __rte_assume(fill <= RTE_MEMPOOL_CACHE_MAX_SIZE);
> +       __rte_assume(remaining <= fill);
> +       cache_objs = &cache->objs[fill];
> +       cache->len = fill - remaining;
>         for (index = 0; index < remaining; index++)
>                 *obj_table++ = *--cache_objs;