RE: mempool cache change

["Morten Brørup" <mb@smartsharesystems.com>]

[-- Attachment #1: Type: text/plain, Size: 5177 bytes --]

The mempool cache size is configurable.

The application developer should configure it to strike a balance between the two performance parameters:

- Throughput: Large mempool cache reduces the number of backend transfers.

- Latency: Large mempool cache causes latency spikes (CPU cache misses) when transferring objects from/to cold backend memory.

 

However, none of the DPDK tests consider latency or jitter, only throughput, so larger cache sizes yield better performance.

 

Another detail (for completeness only): Compared to the size of the mempool itself, the mempool cache must also not be too big, or the entire mempool may be depleted by a few per-lcore mempool caches.

 

PS:

Loading 64 objects (a block of 512 byte) from L2 CPU Cache takes ~2.4 ns, while loading it from System RAM takes ~76 ns.

Loading 256 objects (a block of 2048 byte) from L2 CPU Cache takes ~9 ns, while loading it from System RAM takes ~90 ns.

Loading 512 objects (a block of 4096 byte) from L2 CPU Cache takes ~18 ns, while loading it from System RAM takes ~121 ns.

(Source: Google AI)

 

Venlig hilsen / Kind regards,

-Morten Brørup

 

From: Stephen Hemminger [mailto:stephen@networkplumber.org] 
Sent: Tuesday, 21 July 2026 08.35
To: Bruce Richardson
Cc: Morten Brørup; Kishore Padmanabha; fengchengwen; Thomas Monjalon; dev; Wisam Jaddo; Andrew Rybchenko
Subject: Re: mempool cache change

 

I can't help thinking that there needs to be a more dynamic heuristic.

Keep existing upper limit but only cache as much as gets used.

 

There are two main usage models.

One is fast direct recycling which run to completion generates. The other is when pipeline is used and arrival and departure is on different cores.

 

On Mon, Jul 20, 2026, 10:13 Bruce Richardson <bruce.richardson@intel.com> wrote:

	On Sat, Jul 18, 2026 at 04:07:05PM +0200, Morten Brørup wrote:
	> > From: Kishore Padmanabha [mailto:kishore.padmanabha@broadcom.com]
	> > Sent: Friday, 17 July 2026 19.10
	> > 
	> > Hi Bruce,
	> > 
	> > The below patch works fine for us. We tested all the different packet
	> > sizes.
	> > Thanks for the patch. Do you want to push this patch since it is not
	> > changing the ABI/API?
	> 
	> Too late in the release process.
	> 
	> Let's postpone the discussion for DPDK 26.11, where API/ABI breakage is allowed.
	
	Agreed. Let's not unnecessarily rush this.
	
	> 
	> I'm not strongly opposed to Bruce's algorithm, targeting a fill level of 25 % from the edges and flushing/refilling up to 75 % of the cache when necessary. It does have its advantages for some mempool access patterns (which are not exotic).
	> I just prefer the current algorithm, targeting a fill level of 50 % and only flushing/refilling up to 50 % of the cache when necessary. It performs better at random get/put access patterns, and the backend transactions are smaller.
	> 
	> For DPDK 26.11, where we can break the API/ABI, we can simply double RTE_MEMPOOL_CACHE_MAX_SIZE to 1024, to compensate for reducing the effective cache size from 150 % to 100 %.
	> The mempool cache objs array will no longer be [RTE_MEMPOOL_CACHE_MAX_SIZE * 2], but only [RTE_MEMPOOL_CACHE_MAX_SIZE], so doubling RTE_MEMPOOL_CACHE_MAX_SIZE will not increase the memory footprint, but allow using a cache size up to 1024.
	
	My concern with this approach is that it won't automatically fix the
	problem if we have users who experience a performance regression due to the
	mempool changes. While testpmd allows the mbcache size to be provided via
	parameter, end applications are likely to have it hardcoded. That means
	that if an app does experience a regression, the author/user has to be
	either aware of the mempool changes, or has to debug it down to the mempool
	and then know to increase the mempool cache size in the app.
	
	It's not an insurmountable problem, but one that needs to be very clearly
	called out in our documentation, what the change is, how it may affect
	things and how to fix it.
	
	On the other hand, in realworld, i.e. not just testpmd/l3fwd cases doing
	little packet processesing, I would be fairly hopeful that regressions are
	going to be few and very small.
	
	> 
	> Please also note that the current implementation is carefully designed to keep the transfers to/from the mempool backend CPU cache aligned (assuming cache->size is 2^N and large enough).
	> Refer to the parameters passed to rte_mempool_ops_enqueue/dequeue_bulk().
	> E.g. with mempool cache size 256, backend transfers are 128 objects, 16 full cache lines.
	> Using CPU cache aligned transfers has a few advantages:
	> - There are no cache line ownership issues across different CPU cores repeatedly accessing the backend.
	> - The mempool backend drivers can be performance optimized for transferring full CPU cache lines. (Both source and destination addresses, and number of objects copied, are CPU cache aligned. Assuming all transfers go via the mempool cache.)
	> 
	> These details should be fine tuned in the implementation, if we do proceed with Bruce's algorithm.
	> 
	Yep, good points.
	
	/Bruce


[-- Attachment #2: Type: text/html, Size: 10142 bytes --]