Re: tcp_window_scaling degrades performance

Re: tcp_window_scaling degrades performance

[Andi Kleen]

Maciej Soltysiak <solt@dns.toxicfilms.tv> writes:


> I have been experiencied weird problems with network throughput
> lately and I after experimenting with /proc/sys/net/ipv4 knobs
> I found that when I have tcp_window_scaling 0 I can
> get throughput from a distant server of about 600kB/s (well, 200kB/s
> is fast enough)

It's pretty easy for you to find out. Do a tcpdump -v or ethereal -v 
from both the side of a host you download from and from the linux side.
Then compare all packets. If they don't match the firewall is 
doing something bad. Especially check window values and TCP options
in the SYN packets

It is very very likely the firewall, window scaling works for a lot
of people.

-Andi

Re[2]: tcp_window_scaling degrades performance

[Maciej Soltysiak]

AK> It's pretty easy for you to find out. Do a tcpdump -v or ethereal -v
AK> from both the side of a host you download from and from the linux side.
AK> Then compare all packets. If they don't match the firewall is 
AK> doing something bad. Especially check window values and TCP options
AK> in the SYN packets
I will do that for sure, but preliminary investigation shows that
this behaviour does not show with 2.6.7 and earlier, but appears for sure in
2.6.7-bk13 (Haven't tried earlier bk snapshots)

AK> It is very very likely the firewall, window scaling works for a lot
AK> of people.
It is probable, but here: only 2.6.7+ machines behave like this.
I also noticed, that turning tcp_window_scaling off does not always
fix the problem, turning tcp_bic to 0 too helps even more.

Regards,
Maciej

Re[3]: tcp_window_scaling degrades performance

[Maciej Soltysiak]

AK>> It's pretty easy for you to find out. Do a tcpdump -v or ethereal -v
AK>> from both the side of a host you download from and from the linux side.
AK>> Then compare all packets. If they don't match the firewall is 
AK>> doing something bad. Especially check window values and TCP options
AK>> in the SYN packets
MS> I will do that for sure, but preliminary investigation shows that
MS> this behaviour does not show with 2.6.7 and earlier, but appears for sure in
MS> 2.6.7-bk13 (Haven't tried earlier bk snapshots)
I seem to have isolated the changeset that causes my machines to have
very slow throughput, even as low as 2kB/s when tcp_window_scaling is
enabled.

http://linux.bkbits.net:8080/linux-2.6/cset@1.1757.1.25?nav=index.html|ChangeSet@-3w
[TCP]: TCP acts like it is always out of memory.

Signed-off-by: Stephen Hemminger <shemminger@osdl.org>
Signed-off-by: David S. Miller <davem@redhat.com>

Stephen, David, could you review this changeset, figure out possible
causes or suggest more methods for me to analyse this problem ?

Note that I am not saying that window_scaling is badly implemented,
but it just seems related.

Regards,
Maciej

Re: Re[3]: tcp_window_scaling degrades performance

[David S. Miller]

On Fri, 16 Jul 2004 16:01:57 +0200
Maciej Soltysiak <solt@dns.toxicfilms.tv> wrote:

> I seem to have isolated the changeset that causes my machines to have
> very slow throughput, even as low as 2kB/s when tcp_window_scaling is
> enabled.

The change is valid, firewalls are just corrupting the connection
when window scaling is in use which is a valid RFC standard
TCP feature.

Please, get the firewall fixed.

[partially solved] tcp_window_scaling degrades performance

[Maciej Soltysiak]

> AK> It is very very likely the firewall, window scaling works for a lot
> AK> of people.
> It is probable, but here: only 2.6.7+ machines behave like this.
> I also noticed, that turning tcp_window_scaling off does not always
> fix the problem, turning tcp_bic to 0 too helps even more.
It appears that checkpoint fw-1 that is here spoils everything so that
current linux boxes need to disable tcp_windows_scaling to get
reasonable throughput. When I switched a server to a different link
that bypasses checkpoint it worked well.

Although it most propably is checkpoint's fault (This one is based on
linux 2.4.9-39cpsmp) I belive that the change in the kernel that started
producing these problems is this one:
http://linux.bkbits.net:8080/linux-2.6/cset@1.1784.10.7?nav=index.html|ChangeSet@-3w

When I removed this patch from a 2.6.8-rc1 kernel it started to work
good again. But of course it may be a blind shot.

I am planning on investigating this issue from the checkpoint's
perspective.

Anyway if there is anyone willing to investigate the tcpdumps and
tcptraces of the slow/fast throughput, here it is:
http://soltysiak.com/tcp.php

I am no guru, but it shows:
1) slower throughput
2) twice as much packets exchanged
3) different advertised windows

Best regards,
Maciek