Re: TCP connection hang problem with 2.6.16.16, e1000 (found work-around)

[Ben Greear <greearb@candelatech.com>]

Stephen Hemminger wrote:
> On Wed, 31 May 2006 14:45:13 -0700
> Ben Greear <greearb@candelatech.com> wrote:
> 
> 
>>Ok, I think I found the problem.  It appears to be at least partially
>>self-inflicted.  I was modifying the kernel buffer settings via /proc/*
>>to improve performance.  I noticed that when I used kernel-defaults,
>>the TCP connection hangs stopped (and ran ~20% slower).  My personal opinion
>>is that this is still a bug somewhere in TCP, but at least I have a work-around.
> 
> 
> If you set bogus values, you can end up consuming all the memory on your system.

I still had some sort of memory..ie I could run other applications and could
start a second TCP connection on the same interface.  It is true that you can
truly OOM systems with bad settings here, but my machine has 2GB of RAM, and
I think my setting were low enough that it should not have run out.  It also
had enough memory left to spam the duplicate TCP acks at around 200kpps in
both directions!  Manually over-riding the write buffer size had little or
no effect:  It would still hang.

>>The new settings that appear to work around the TCP connection hang,
>>and still give decent performance are below.  Documentation being thin,
>>I used the suggested values in Documentation/networking/ixgb.txt as
>>hints:
> 
> 
> Settings don't belong in this Documentation file because it a general
> thing not specific to that hardware.

The file was describing how to tune for good performance using their
NICs.

At least most of the TCP settings are described in the networking/ip-sysctl.txt
file, but I could find nothing on the core.rmem_max setting, except in examples
in the 10Gbe NIC help texts.

It would be nice to either have the core.* values described in the
ip-sysctl.txt file, or perhaps add a new file.

There are tuning tips all over the web too...often quite badly
outdated it seems.  Might be a good thing to add to the linux-net wiki.

I'd give it a try, but obviously my info is suspect! :P

Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com