Re: Introduce FCLONE_SCRATCH skbs to reduce stack memory useage and napi jitter

[Eric Dumazet <eric.dumazet@gmail.com>]

Le jeudi 27 octobre 2011 à 21:37 -0400, Neil Horman a écrit :
> > 
> Yes, I'd experimented with some of this, and had mixed results.  Specifically I
> came up with a way to use the additional space as an array of list heads, with
> backpointers to the socket_queue each list_head was owned by and the skb that
> owned the entry.  It was nice because it let me enqueue the same skb to hundreds
> of sockets at the same time, which was really nice.  It was bad however, because
> it completely broke socket accounting (any likely anything else that required on
> any part of the socket state).  Any thoughts on ways I might improve on that.
> If I could make some sort of reduced sk_buff so that I didn't have to allocate
> all 256 bytes of a full sk_buff, that would be great!

It seems your objectives are contradictory (memory saving and cpu
saving). Most of the time we have to fight false sharing first.

I dont want to try to use the available space from skb, since its cache
unfriendly, in case you have one CPU 100% in softirq handling,
dispatching messages to XX other application cpus.

You dont want each application cpu slowing down other cpus by
manipulating a list anchor, contained in a shared cache line, highly
contended. We already have one high contention point (skb refcount or
data refcount).

Another point of interest is that modern NIC tend to use paged frag
skbs, with very litle space available in skb head. frag part is
readonly, and sometime with no available space neither.

So your idea only works on some (old gen) nics and some narrow
conditions.

I believe it adds too much complex code for this.

I understand you are disappointed, but maybe you should have shared your
ideas before coding them !