Re: [E1000-devel] Page Allocation Failure with e1000 using jumboframe

[Ming Zhang <mingz@ele.uri.edu>]

I am sorry that the guy who found this problem is running suse linux,
but with vanilla kernel. so this is a generic problem, not suse
specific.

I am sorry for my insaneness at that time. :P

Ming


On Fri, 2005-08-19 at 20:01 +0200, Andi Kleen wrote:
> > I could certainly be mistaken.  The difference I saw was that suse kernels 
> > recycle the same skb pointers back to our driver, and the redhat kernels 
> > seem to march through a much larger range before the values repeat.  This 
> > is all observation based, so I may be completely wrong on this issue.
> 
> Maybe you're comparing 2.4 to 2.6 or different 2.6s. But I don't 
> think there are any SUSE specific patches affecting this.
> 
> > 
> > >> the formula for the size that the current e1000 looks for is something
> > >> like
> > >>
> > >> a = MTU roundup to next power of 2
> > >> a += 2 (skb_reserve(NET_IP_ALIGN))
> > >> a += 16 (skb_reserve 16 by __dev_alloc_skb)
> > >>
> > >> so, a = 2048 + 2 + 16, or 2066
> > >> request (a) from slab, which does a power of 2 roundup
> > >> so the skb comes from the 4k (single page) slab for standard mtu.
> > >
> > >That's very suboptimal because you're wasting nearly 2k. It would
> > >be better if you allocated 4k or exactly 2k
> > 
> > we have to give the full 2k to hardware, unfortunately.  which means 
> > mapping the full 2k.  we do the skb reserve because of cache/alighment 
> > effects which show a (big) hit in performance if we don't align the IP 
> > header.  Yes I know that dword unaligned DMA really hurts on some arches, 
> > but thats why the arch can #def NET_IP_ALIGN 0.
> 
> What is the requirement of your hardware? power of two alignment or 
> power of two size?  If the later does it really trash the data behind it?
> 
> But surely it doesn't use all of the 2k for the 1.5k MTU, so it
> would be good if you could fit the header alignment in there
> and only get the exact needed amount from the underlying allocator.
> 
> 
> > 
> > if thats the case, then we're left asking the question, who uses that 16 
> > bytes that are skb_reserved by __dev_alloc_skb???
> 
> Nothing, except maybe routing to a different class of link layer that
> needs bigger headers (e.g. PPP).  Even then it's just a performance
> optimization to avoid a skb copy.
> 
> I suppose it would be possible to keep track of the largest supported
> hard_header_len of all devices and if it's all identical don't add the 
> 16 bytes.
> 
> -Andi



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf