Re: can we reuse an skb

[Evgeniy Polyakov <zbr@ioremap.net>]

On Sat, Jun 20, 2009 at 09:24:52AM +0530, Peter Chacko (peterchacko35@gmail.com) wrote:
> Assume that we have n-cores capable of processing packet at the same
> time. if we trade off memory for computing, why don't we pre-allocate
> "n" dedicated skb buffers regardless of the size of each packet, but
> just as big as the size of the MTU itself.(forget JUMBO packet for
> now).(  today, dev_alloc_skb() allocate based on the packet len, which
> is memory usage optimized.).
> 
> each dedicated memory buffer is now a per-CPU/thread data structure.
> 
> In the  IO stack world , we have buffer cache, identified by buffer
> headers. That design was conceived so early as each block level write
> was typically 512 bytes and same for all blocks. why don't we adapt
> that into network stack ?

And page cache has ugly buffer_head strucures to point partially updated
blocks within the page, while block IO has to perform read-modify-write
cycles when it updates less than a single block, which is rather costly,
especially on big-sized blocks.

Consider 100-byte writes and 1500 MTU (or consider 9k MTU for a moment)
- this is a huge overhead and if socket queue is 10 MB for example, you
will waste 140 MB of RAM or have miserable performance if the whole 1500
bytes are accounted into socket buffer while only 100 bytes are used.

For those who wants to play with skb recycling - just implement own pool
of skbs and provide private destructor, which will requeue the same
object into private pool. Not sure this will work any bit faster than
existing scheme though if proper memory accounting is implemented.

-- 
	Evgeniy Polyakov