skb diet

skb diet

[Hisham Kotry]

Hi,

I just read David S. Miller's skb redundancy page and in it he seems to suggest
taking an approach similar to that of BSD's mbufs to reduce the skb's
size. I was going to do some janitor work on the network stack and I
thought that maybe I could start by adding a tag list to the skb
similar to BSD's mbuf tags and change the code to store its private
variables in tags rather than in skb->cb. That should save 40 bytes
per skb and maybe later the tc and nf related variables could go in
tags too. The transition shouldn't be hard for the most part, many
users of skb->cb use macros like FRAG_CB(skb) so I could redefine
those macros to search the tag-list for the appropriate tag and return
its contents. Is this approach acceptable or is there a consensus on
another approach?

Re: skb diet

[Andi Kleen]

On Saturday 15 April 2006 13:17, Hisham Kotry wrote:
> I just read David S. Miller's skb redundancy page and in it he seems to suggest
> taking an approach similar to that of BSD's mbufs to reduce the skb's
> size. I was going to do some janitor work on the network stack and I
> thought that maybe I could start by adding a tag list to the skb
> similar to BSD's mbuf tags and change the code to store its private
> variables in tags rather than in skb->cb. That should save 40 bytes
> per skb and maybe later the tc and nf related variables could go in
> tags too. The transition shouldn't be hard for the most part, many
> users of skb->cb use macros like FRAG_CB(skb) so I could redefine
> those macros to search the tag-list for the appropriate tag and return
> its contents. Is this approach acceptable or is there a consensus on
> another approach?

Where would that tag list be stored if you want to remove the 
40 bytes of ->cb?

In general I don't think it will help much even if you find
a magic way to store it without using memory @) because
->cb is already private to each layer, so it's only the single 40 bytes
which is reused as needed.

You seem to think they accumulate which is not the case.

Linux 2.0 did something like this, but that was removed for good 
reasons. Now TCP always clones skbs before sending it out.

And optimizing for uncommon cases (not TCP) doesn't seem too useful.

-Andi

Re: skb diet

[bert hubert]

On Sat, Apr 15, 2006 at 09:22:01PM +0200, Andi Kleen wrote:

> And optimizing for uncommon cases (not TCP) doesn't seem too useful.

There are servers that do tens of megabits of UDP these days (think VoIP, or
in my case, DNS), so it not that uncommon.

	Bert

-- 
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://netherlabs.nl              Open and Closed source services

Re: skb diet

[Hisham Kotry]

> Where would that tag list be stored if you want to remove the
> 40 bytes of ->cb?

I apologize if I wasn't clear, the tag list would go in a new
skb->tags field replacing the existsing skb->cb array, so the skb
would lose 40-sizeof(void*) bytes wich seems reasonable to me.

> Linux 2.0 did something like this, but that was removed for good
> reasons. Now TCP always clones skbs before sending it out.

Do you remember what those reasons were? I couldn't find a related
discussion in the archives. I think the BSD mbuf tags approach is
sound enough to justify the move.

> And optimizing for uncommon cases (not TCP) doesn't seem too useful.

As pointed out by Bert Hubert, there are people who have heavy traffic
on non-tcp connections.

Thanks,
Hisham

Re: skb diet

[Andi Kleen]

On Sunday 16 April 2006 14:56, Hisham Kotry wrote:
> > Where would that tag list be stored if you want to remove the
> > 40 bytes of ->cb?
>
> I apologize if I wasn't clear, the tag list would go in a new
> skb->tags field replacing the existsing skb->cb array, so the skb
> would lose 40-sizeof(void*) bytes wich seems reasonable to me.

This means for the common TCP case you would actually 
need more memory than before - a new pointer and overhead
from the tags. Currently we neither need pointer nor tags 
for anything.

Also you would need to complicate alloc_skb to preallocate
this memory and complicate the freeing by checking for it
and freeing it if it was allocated dynamically 
(e.g. if a later layer needed it, not the layer that first allocates
it you would need to allocate a new buffer later which would then
need to be freed) 

>
> > Linux 2.0 did something like this, but that was removed for good
> > reasons. Now TCP always clones skbs before sending it out.
>
> Do you remember what those reasons were? I couldn't find a related
> discussion in the archives. I think the BSD mbuf tags approach is
> sound enough to justify the move.

>From your description so far it seems to only have disadvantages.

> > And optimizing for uncommon cases (not TCP) doesn't seem too useful.
>
> As pointed out by Bert Hubert, there are people who have heavy traffic
> on non-tcp connections.

It's a small minority compared to TCP users.

-Andi

Re: skb diet

[David S. Miller]

From: Andi Kleen <ak@suse.de>
Date: Sun, 16 Apr 2006 17:16:31 +0200

> On Sunday 16 April 2006 14:56, Hisham Kotry wrote:
> > > Linux 2.0 did something like this, but that was removed for good
> > > reasons. Now TCP always clones skbs before sending it out.
> >
> > Do you remember what those reasons were? I couldn't find a related
> > discussion in the archives. I think the BSD mbuf tags approach is
> > sound enough to justify the move.
> 
> From your description so far it seems to only have disadvantages.

I totally agree, tags are very stupid.

I only brought them up long as food-for-thought, not as a very
serious candidate for implementation.

Indirection is a performance killer, and we've seen this time and
time again in the past.  Tags add more indirection for questionable
gains.