Re: [RFC PATCH] greatly reduce SLOB external fragmentation

[Christoph Lameter <cl@linux-foundation.org>]

Andi Kleen wrote:
>> Finally getting rid of SLAB is a much trickier proposition because SLUB
>> still loses in a few important corner cases.
> 
> The big issue is that we haven't really made much progress on at least
> some of these test cases (like the database benchmarks) for quite some
> time (and that wasn't for a lack of trying) :-/ Might be a fundamental
> problem.

It would be good to have more of these benchmark regressions than just TPC which requires a database setup etc etc. Preferably something that is easily runnable by everyone.

There is a fundamental difference in how frees are handled. Slub is queueless so it must use an atomic operation on the page struct of the slab that we free to (in the case that the freeing does not occur to the current cpu slab) to serialize access.

SLAB can avoid that by just stuffing the pointer to the object to be freed into a per cpu queue. Then later the queue is processed and the objects are merged into the slabs. But the later workqueue processing then causes run time variabilities which impact network performance and cause latencies. And the queuing only works in the SMP case. In the NUMA case we need to first check the locality of the object and then stuff it into alien queues (if its not local node). Then we need to expire the alien queues at some point. We have these alien queues per node which means they require locks and we have NODES * CPUS of them.