RE: mempool cache change

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Tuesday, 21 July 2026 11.12
> 
> On Tue, Jul 21, 2026 at 11:02:34AM +0200, Morten Brørup wrote:
> >    The mempool cache size is configurable.
> >
> >    The application developer should configure it to strike a balance
> >    between the two performance parameters:
> >
> >    - Throughput: Large mempool cache reduces the number of backend
> >    transfers.
> >
> >    - Latency: Large mempool cache causes latency spikes (CPU cache
> misses)
> >    when transferring objects from/to cold backend memory.
> >
> >
> Since 26.11 is an ABI break release, perhaps we can reconsider the
> option
> to have the "keep" value configurable for the mempool caches, i.e. for
> random alloc/free's set it to 50% like you have, or 25% as proposed by
> my
> patch, while for pipeline apps it should be set to 0%. Rather than us
> constantly arguing^H^H^H discussing what the value should be, we can
> make
> it tunable for the developer?
> 
> As an alternative version of that proposal, we can simplify and just
> have a
> couple of predefined simple profiles for 2 or 3 values, one for
> pipeline
> apps, and one or two for run-to-completion, rather than exposing the
> whole
> range of possibilities to the user.

At this time, I'm not planning major changes to the mempool cache algorithm.

I have sent a patch [DIV32] requiring the mempool cache size to be divisible by 32.
This opens a road towards operating on the mempool cache in chunks of 32 objects.

Bruce strongly hinted that the ring driver might not be optimal for mbuf pools,
so I played around with the lock-free stack mempool driver, and it is extremely slow.
To address this issue, I developed a faster stack-like "pile" [PILE-STACK] and a mempool driver [PILE-MEMPOOL] on top of it.

All this playing around and working with the details got me thinking. Which leads back to the subject of 32-object chunks...
The current mempool cache algorithm is targeting large flush/replenish operations.
If the mempool driver is slow (and its runtime scales with the number of objects), large operations will cause latency spikes.

A new algorithm could take a different approach to address latency spikes, targeting small driver operations instead of large driver operations:
When pulling objects from a cache with insufficient objects, only refill the necessary number of objects to satisfy the request, rounded up to 32-object chunks.
When pushing objects from a cache with insufficient space, only flush the necessary number of objects to satisfy the request, rounded up to 32-object chunks.

As a further optimization, instead of moving objects down in the stack when flushing from the bottom, we could use an integer offset to point at the bottom of the stack, and wrap around the index at size when accessing the array. E.g. for push:
for (i = 0; i < num; i++) cache->objs[(cache->bottom + cache->len + i) % size] = obj_table[i];

I don't have plans to implement such an algorithm (or the bottom offset) for DPDK 26.11.
Just sharing my thoughts.

[DIV32]: https://patchwork.dpdk.org/project/dpdk/patch/20260812120626.1772120-1-mb@smartsharesystems.com/
[PILE-STACK]: https://patchwork.dpdk.org/project/dpdk/patch/20260812134756.1829613-1-mb@smartsharesystems.com/
[PILE-MEMPOOL]: https://patchwork.dpdk.org/project/dpdk/patch/20260812141511.1854348-1-mb@smartsharesystems.com/