Re: how expensive are mallocs?

[David Miller <davem@davemloft.net>]

From: Andrew Grover <andy.grover@gmail.com>
Date: Tue, 24 Nov 2009 09:57:34 -0800

> How much effort generally makes sense to avoid mallocs? For example,
> rds has a function that does a kmalloc for an array of scatterlist
> entries, which is freed at the bottom of the function. We couldn't
> allocate off the stack in all cases (too big), but we could allocate
> an array of say 8 scatterlists, and use it if that's big enough,
> falling back to kmalloc if it's not.
> 
> Is this a good idea?

This is a poor idea, especially if this function executes frequently.

Better to have a per-socket or per-cpu (softirq protected, if
necessary) work area to do such things if you really can't fit it on
the stack.

> Also, RDS has its own per-cpu page remainder allocator (see
> net/rds/page.c) for kernel send buffers. Would cutting this code and
> just using kmalloc be recommended? Doesn't SL?B already do per-cpu
> pools?
> 
> Does this stuff even matter enough to rise above the noise in benchmarks?

We already have a per-socket page allocation scheme for sendmsg handling.

In general adding more and more specialized allocators is heavily
discouraged.  If the generic kernel facilities don't fit the bill,
fix them.