Re: tcp bw in 2.6

[David Miller <davem@davemloft.net>]

From: lm@bitmover.com (Larry McVoy)
Date: Tue, 2 Oct 2007 15:36:44 -0700

> On Tue, Oct 02, 2007 at 03:32:16PM -0700, David Miller wrote:
> > I'm starting to have a theory about what the bad case might
> > be.
> > 
> > A strong sender going to an even stronger receiver which can
> > pull out packets into the process as fast as they arrive.
> > This might be part of what keeps the receive window from
> > growing.
> 
> I can back you up on that.  When I straced the receiving side that goes
> slowly, all the reads were short, like 1-2K.  The way that works the 
> reads were a lot larger as I recall.

My issue turns out to be hardware specific too.

The two Broadcom 5714 onboard NICs on my Niagara t1000 give bad packet
receive performance for some reason, the other two which are Broadcom
5704's are perfectly fine.  I'll figure out what the problem is,
probably some misprogramed register in either the chip or the bridge
it's behind.

The UDP stream test of netperf is great for isolating TCP/TSO vs.
hardware issues.  If you can't saturate the pipe or the cpu with
the UDP stream test, it's likely a hardware issue.

The cpu utilization and service demand numbers provided, on both
send and receive, are really useful for diagnosing problems like
this.

Rick deserves several beers for his work on this cool toy. :)