re: [partially solved] tcp_window_scaling degrades performance

re: [partially solved] tcp_window_scaling degrades performance

[Dan Kegel]

Maciej Soltysiak <solt@dns.toxicfilms.tv> wrote:
> It appears that checkpoint fw-1 that is here spoils everything so that
> current linux boxes need to disable tcp_windows_scaling to get
> reasonable throughput.

Just checking- have you read
http://lwn.net/Articles/91976/
("TCP window scaling and broken routers")?

- Dan

-- 
My technical stuff: http://kegel.com
My politics: see http://www.misleader.org for examples of why I'm for regime change

Re: tcp_window_scaling degrades performance

[Andi Kleen]

Maciej Soltysiak <solt@dns.toxicfilms.tv> writes:


> I have been experiencied weird problems with network throughput
> lately and I after experimenting with /proc/sys/net/ipv4 knobs
> I found that when I have tcp_window_scaling 0 I can
> get throughput from a distant server of about 600kB/s (well, 200kB/s
> is fast enough)

It's pretty easy for you to find out. Do a tcpdump -v or ethereal -v 
from both the side of a host you download from and from the linux side.
Then compare all packets. If they don't match the firewall is 
doing something bad. Especially check window values and TCP options
in the SYN packets

It is very very likely the firewall, window scaling works for a lot
of people.

-Andi

Re[2]: tcp_window_scaling degrades performance

[Maciej Soltysiak]

AK> It's pretty easy for you to find out. Do a tcpdump -v or ethereal -v
AK> from both the side of a host you download from and from the linux side.
AK> Then compare all packets. If they don't match the firewall is 
AK> doing something bad. Especially check window values and TCP options
AK> in the SYN packets
I will do that for sure, but preliminary investigation shows that
this behaviour does not show with 2.6.7 and earlier, but appears for sure in
2.6.7-bk13 (Haven't tried earlier bk snapshots)

AK> It is very very likely the firewall, window scaling works for a lot
AK> of people.
It is probable, but here: only 2.6.7+ machines behave like this.
I also noticed, that turning tcp_window_scaling off does not always
fix the problem, turning tcp_bic to 0 too helps even more.

Regards,
Maciej

[partially solved] tcp_window_scaling degrades performance

[Maciej Soltysiak]

> AK> It is very very likely the firewall, window scaling works for a lot
> AK> of people.
> It is probable, but here: only 2.6.7+ machines behave like this.
> I also noticed, that turning tcp_window_scaling off does not always
> fix the problem, turning tcp_bic to 0 too helps even more.
It appears that checkpoint fw-1 that is here spoils everything so that
current linux boxes need to disable tcp_windows_scaling to get
reasonable throughput. When I switched a server to a different link
that bypasses checkpoint it worked well.

Although it most propably is checkpoint's fault (This one is based on
linux 2.4.9-39cpsmp) I belive that the change in the kernel that started
producing these problems is this one:
http://linux.bkbits.net:8080/linux-2.6/cset@1.1784.10.7?nav=index.html|ChangeSet@-3w

When I removed this patch from a 2.6.8-rc1 kernel it started to work
good again. But of course it may be a blind shot.

I am planning on investigating this issue from the checkpoint's
perspective.

Anyway if there is anyone willing to investigate the tcpdumps and
tcptraces of the slow/fast throughput, here it is:
http://soltysiak.com/tcp.php

I am no guru, but it shows:
1) slower throughput
2) twice as much packets exchanged
3) different advertised windows

Best regards,
Maciek