Re: how expensive are mallocs?

[Andi Kleen <andi@firstfloor.org>]

Andrew Grover <andy.grover@gmail.com> writes:

> How much effort generally makes sense to avoid mallocs? For example,

The slab allocators are very optimized, with a fast path for
allocation and freeing that's essentially "disable interrupts ; 
unlink object from a list ; reenable". That's not expensive
(unless you're running on a CPU where disabling interrupts is)

The main costs in them come when you free objects on a different CPU
(or worse node) than where you allocate them. In that case you can end
up with some bounced cache lines, which are slow. If you can avoid
that you're good. If you can't even then you would need to make major
effort to do better.

> Also, RDS has its own per-cpu page remainder allocator (see
> net/rds/page.c) for kernel send buffers. Would cutting this code and
> just using kmalloc be recommended? Doesn't SL?B already do per-cpu
> pools?

Slab is all per cpu in the fast path, but see above.

> Does this stuff even matter enough to rise above the noise in benchmarks?

Yes it does in some circumstances, but it's hard to do better.

One example of doing better for special circumstances would be Eric's
rps work, but doing these things is not easy and only worth it
for really critical cases.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.