Re: [E1000-devel] Page Allocation Failure with e1000 using jumboframe

[Andi Kleen <ak@suse.de>]

> I could certainly be mistaken.  The difference I saw was that suse kernels 
> recycle the same skb pointers back to our driver, and the redhat kernels 
> seem to march through a much larger range before the values repeat.  This 
> is all observation based, so I may be completely wrong on this issue.

Maybe you're comparing 2.4 to 2.6 or different 2.6s. But I don't 
think there are any SUSE specific patches affecting this.

> 
> >> the formula for the size that the current e1000 looks for is something
> >> like
> >>
> >> a = MTU roundup to next power of 2
> >> a += 2 (skb_reserve(NET_IP_ALIGN))
> >> a += 16 (skb_reserve 16 by __dev_alloc_skb)
> >>
> >> so, a = 2048 + 2 + 16, or 2066
> >> request (a) from slab, which does a power of 2 roundup
> >> so the skb comes from the 4k (single page) slab for standard mtu.
> >
> >That's very suboptimal because you're wasting nearly 2k. It would
> >be better if you allocated 4k or exactly 2k
> 
> we have to give the full 2k to hardware, unfortunately.  which means 
> mapping the full 2k.  we do the skb reserve because of cache/alighment 
> effects which show a (big) hit in performance if we don't align the IP 
> header.  Yes I know that dword unaligned DMA really hurts on some arches, 
> but thats why the arch can #def NET_IP_ALIGN 0.

What is the requirement of your hardware? power of two alignment or 
power of two size?  If the later does it really trash the data behind it?

But surely it doesn't use all of the 2k for the 1.5k MTU, so it
would be good if you could fit the header alignment in there
and only get the exact needed amount from the underlying allocator.


> 
> if thats the case, then we're left asking the question, who uses that 16 
> bytes that are skb_reserved by __dev_alloc_skb???

Nothing, except maybe routing to a different class of link layer that
needs bigger headers (e.g. PPP).  Even then it's just a performance
optimization to avoid a skb copy.

I suppose it would be possible to keep track of the largest supported
hard_header_len of all devices and if it's all identical don't add the 
16 bytes.

-Andi


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf