2.6.27.8 (+the idr fix) TCP Ack issue

2.6.27.8 (+the idr fix) TCP Ack issue

[Russell King]

While trying to access a website on a FC5 machine, I encountered what seemed
to be excessive traffic without much progress.

tcpdumping the connection showed a permanent stream of acks from both ends
of the connection.  Ben Hutchings suggested that 607bfbf might fix it, so
I built 2.6.27.8 which has this fix in some 20 days ago.  After encountering
other problems, a fix to lib/idr.c was applied.  This kernel seemed to be
fine, until...

19:47:32.062670 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: S 3543174870:3543174870(0) win 5840 <mss 1460,sackOK,timestamp 26999689 0,nop,wscale 6>
19:47:32.135812 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: S 3016012818:3016012818(0) ack 3543174871 win 8192 <mss 1276>
19:47:32.135837 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 1 win 5840
19:47:32.135899 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: P 1:644(643) ack 1 win 5840
19:47:32.167644 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 644 win 7073
19:47:32.174366 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: P 1:1190(1189) ack 644 win 7073
19:47:32.174414 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 1190 win 8323
19:47:32.174701 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . 644:3196(2552) ack 1190 win 8323
19:47:32.174720 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: P 3196:3216(20) ack 1190 win 8323
19:47:32.218718 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: P 1190:2215(1025) ack 1920 win 8932
19:47:32.258402 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.285388 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.285397 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.320287 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.320300 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.353016 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.353022 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.382702 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.382712 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.404786 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.404793 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.446827 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.446835 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.451343 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . 1920:3196(1276) ack 2215 win 10701
19:47:32.480976 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.480984 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.530343 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.530356 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.554244 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.554251 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.582139 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.582146 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.613093 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.613121 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701

and then it ploughs into ack-madness at as high a speed as the link can
handle:

19:47:32.634725 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.634753 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.654389 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.654408 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.674332 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.674339 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.692550 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.692555 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
19:47:32.712249 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
19:47:32.712265 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701

which is the same thing as the FC5 kernel.  It took three attempts
(killing off the browser and restarting it) to access the website.

The retransmission at 19:47:32.451343 looks like quite silly behaviour
from the Linux kernel - the remote end has acked data up to 3216 but
it's resending old data.

Any ideas?

-- 
Russell King

Re: 2.6.27.8 (+the idr fix) TCP Ack issue

[Ilpo Järvinen]

On Tue, 30 Dec 2008, Russell King wrote:

> While trying to access a website on a FC5 machine, I encountered what seemed
> to be excessive traffic without much progress.

And what kernel that would be, btw?

> tcpdumping the connection showed a permanent stream of acks from both ends
> of the connection.  Ben Hutchings suggested that 607bfbf might fix it, so
> I built 2.6.27.8 which has this fix in some 20 days ago.

Considering your dump, 607bfbf cannot do any change.

> After encountering
> other problems, a fix to lib/idr.c was applied.  This kernel seemed to be
> fine, until...
> 
> 19:47:32.062670 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: S 3543174870:3543174870(0) win 5840 <mss 1460,sackOK,timestamp 26999689 0,nop,wscale 6>
> 19:47:32.135812 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: S 3016012818:3016012818(0) ack 3543174871 win 8192 <mss 1276>
> 19:47:32.135837 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 1 win 5840
> 19:47:32.135899 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: P 1:644(643) ack 1 win 5840
> 19:47:32.167644 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 644 win 7073

Just for the record, the peer certainly shrinks window here, not
that it should affect us in bad way since we don't even get that far 
and the window opens more again in the later packet anyway...

> 19:47:32.174366 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: P 1:1190(1189) ack 644 win 7073
> 19:47:32.174414 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 1190 win 8323
> 19:47:32.174701 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . 644:3196(2552) ack 1190 win 8323
> 19:47:32.174720 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: P 3196:3216(20) ack 1190 win 8323
> 19:47:32.218718 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: P 1190:2215(1025) ack 1920 win 8932

Here the peer acks to 1920, this is likely last feedback we consider 
valid.

> 19:47:32.258402 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.285388 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701

It would be interesting to know if this segment gets discarded. It's most 
likely considered out-of-sequence for some reason because the response is
an immediate duplicate ack:

> 19:47:32.285397 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.320287 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.320300 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.353016 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.353022 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.382702 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.382712 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.404786 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.404793 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.446827 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.446835 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.451343 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . 1920:3196(1276) ack 2215 win 10701
> 19:47:32.480976 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.480984 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.530343 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.530356 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.554244 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.554251 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.582139 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.582146 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.613093 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.613121 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 
> and then it ploughs into ack-madness at as high a speed as the link can
> handle:
> 
> 19:47:32.634725 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.634753 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.654389 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.654408 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.674332 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.674339 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.692550 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.692555 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 19:47:32.712249 IP 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: . ack 3216 win 10701
> 19:47:32.712265 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . ack 2215 win 10701
> 
> which is the same thing as the FC5 kernel.  It took three attempts
> (killing off the browser and restarting it) to access the website.

I tried couple of times but got at perfectly working connection from 
here (that's so typical wrt. these bugs).

> The retransmission at 19:47:32.451343 looks like quite silly behaviour
> from the Linux kernel - the remote end has acked data up to 3216 but
> it's resending old data.
>
> Any ideas?

Most likely the latest acks are considered invalid for some reason.
...That perfectly explains why you get that retransmission there.

If you have the dump still at handy, could you add couple of -v -v for
tcpdump. It could be that the peer is using bogus seqnos in the
duplicate ACK but by default that's not visible for zero sized segs
(checked in tcp_validate_incoming in the 2.6.28.7 kernel).

...Grr, we seem to be lacking a mib for that seqno check failure.


-- 
 i.

Re: 2.6.27.8 (+the idr fix) TCP Ack issue

[Russell King]

On Wed, Dec 31, 2008 at 12:38:24PM +0200, Ilpo Järvinen wrote:
> If you have the dump still at handy, could you add couple of -v -v for
> tcpdump. It could be that the peer is using bogus seqnos in the
> duplicate ACK but by default that's not visible for zero sized segs
> (checked in tcp_validate_incoming in the 2.6.28.7 kernel).

Is "seqno" another name for the IP ID field?

-- 
Russell King

Re: 2.6.27.8 (+the idr fix) TCP Ack issue

[Ilpo Järvinen]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 864 bytes --]

On Wed, 31 Dec 2008, Russell King wrote:

> On Wed, Dec 31, 2008 at 12:38:24PM +0200, Ilpo Järvinen wrote:
> > If you have the dump still at handy, could you add couple of -v -v for
> > tcpdump. It could be that the peer is using bogus seqnos in the
> > duplicate ACK but by default that's not visible for zero sized segs
> > (checked in tcp_validate_incoming in the 2.6.28.7 kernel).
> 
> Is "seqno" another name for the IP ID field?

No, it's a sequence number, there are two sequence number spaces per flow, 
one for each direction (each has a corresponding field in the tcp header). 
For pure ACKs tcpdump won't show the sequence number by default even 
though it's still there...

With tcpdump -n -v -v I get this for pure acks:

...: ., cksum 0x43c0 (correct), 671:671(0) ack 14649 win 36500

That 671:671(0) won't be visible with smaller verbosity.

-- 
 i.

Re: 2.6.27.8 (+the idr fix) TCP Ack issue

[Herbert Xu]

On Tue, Dec 30, 2008 at 07:58:58PM +0000, Russell King wrote:
>
> 19:47:32.174701 IP dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . 644:3196(2552) ack 1190 win 8323

You are to be using TSO/GSO, what driver are you using and what
does ethtool -k <ifname> say? You could try disabling TSO to see
if the problem goes away with ethtool -K ifname tso off.

If you could please take a dump one hop away from your host to
see the segmented packets.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: 2.6.27.8 (+the idr fix) TCP Ack issue

[Russell King]

On Wed, Dec 31, 2008 at 10:20:46PM +0200, Ilpo Järvinen wrote:
> No, it's a sequence number, there are two sequence number spaces per flow, 
> one for each direction (each has a corresponding field in the tcp header). 
> For pure ACKs tcpdump won't show the sequence number by default even 
> though it's still there...
> 
> With tcpdump -n -v -v I get this for pure acks:
> 
> ...: ., cksum 0x43c0 (correct), 671:671(0) ack 14649 win 36500
> 
> That 671:671(0) won't be visible with smaller verbosity.

Ok, I think I see the problem - it's the remote end.

19:47:32.062670 IP (tos 0x0, ttl  64, id 30047, offset 0, flags [DF], proto: TCP (6), length: 60) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: S, cksum 0x6cf8 (correct), 3543174870:3543174870(0) win 5840 <mss 1460,sackOK,timestamp 26999689 0,nop,wscale 6>
19:47:32.135812 IP (tos 0x0, ttl 238, id 14, offset 0, flags [none], proto: TCP (6), length: 44) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: S, cksum 0x49e2 (correct), 3016012818:3016012818(0) ack 3543174871 win 8192 <mss 1276>
19:47:32.135837 IP (tos 0x0, ttl  64, id 30048, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x6a17 (correct), 1:1(0) ack 1 win 5840
19:47:32.135899 IP (tos 0x0, ttl  64, id 30049, offset 0, flags [DF], proto: TCP (6), length: 683) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: P 1:644(643) ack 1 win 5840
19:47:32.167644 IP (tos 0x0, ttl  47, id 53954, offset 0, flags [DF], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x62c3 (correct), 1:1(0) ack 644 win 7073
19:47:32.174366 IP (tos 0x0, ttl  47, id 53955, offset 0, flags [DF], proto: TCP (6), length: 1229) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: P 1:1190(1189) ack 644 win 7073

Remote end sends 1:1190

19:47:32.174414 IP (tos 0x0, ttl  64, id 30050, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x593c (correct), 644:644(0) ack 1190 win 8323
19:47:32.174701 IP (tos 0x0, ttl  64, id 30051, offset 0, flags [DF], proto: TCP (6), length: 2592) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: . 644:3196(2552) ack 1190 win 8323

We ack 1190

19:47:32.174720 IP (tos 0x0, ttl  64, id 30053, offset 0, flags [DF], proto: TCP (6), length: 60) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: P, cksum 0xcd57 (incorrect (-> 0x1cd0), 3196:3216(20) ack 1190 win 8323
19:47:32.218718 IP (tos 0x0, ttl  47, id 53956, offset 0, flags [DF], proto: TCP (6), length: 1065) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: P 1190:2215(1025) ack 1920 win 8932

Remote end sends 1190:2215

19:47:32.258402 IP (tos 0x0, ttl  64, id 30054, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x41e5 (correct), 3216:3216(0) ack 2215 win 10701

We ack 2215

19:47:32.285388 IP (tos 0x0, ttl 238, id 5, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x45e6 (correct), 1190:1190(0) ack 3216 win 10701

Remote end sequence number is still at 1190.

19:47:32.285397 IP (tos 0x0, ttl  64, id 30055, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x41e5 (correct), 3216:3216(0) ack 2215 win 10701
19:47:32.320287 IP (tos 0x0, ttl 238, id 10, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x45e6 (correct), 1190:1190(0) ack 3216 win 10701
19:47:32.320300 IP (tos 0x0, ttl  64, id 30056, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x41e5 (correct), 3216:3216(0) ack 2215 win 10701
19:47:32.353016 IP (tos 0x0, ttl 238, id 12, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x45e6 (correct), 1190:1190(0) ack 3216 win 10701
...

So it looks to me like the remote end doesn't like our ack of 2215 for
some reason.


-- 
Russell King

Re: 2.6.27.8 (+the idr fix) TCP Ack issue

[David Miller]

From: Russell King <rmk@arm.linux.org.uk>
Date: Fri, 2 Jan 2009 14:26:09 +0000

> Remote end sequence number is still at 1190.
> 
> 19:47:32.285397 IP (tos 0x0, ttl  64, id 30055, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x41e5 (correct), 3216:3216(0) ack 2215 win 10701
> 19:47:32.320287 IP (tos 0x0, ttl 238, id 10, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x45e6 (correct), 1190:1190(0) ack 3216 win 10701
> 19:47:32.320300 IP (tos 0x0, ttl  64, id 30056, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x41e5 (correct), 3216:3216(0) ack 2215 win 10701
> 19:47:32.353016 IP (tos 0x0, ttl 238, id 12, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x45e6 (correct), 1190:1190(0) ack 3216 win 10701
> ...
> 
> So it looks to me like the remote end doesn't like our ack of 2215 for
> some reason.

Let's make sure the checksum is OK.  That's a reason the ACK might
get dropped at the remote end.

Re: 2.6.27.8 (+the idr fix) TCP Ack issue

[Russell King]

On Fri, Jan 02, 2009 at 01:34:35PM -0800, David Miller wrote:
> From: Russell King <rmk@arm.linux.org.uk>
> Date: Fri, 2 Jan 2009 14:26:09 +0000
> 
> > Remote end sequence number is still at 1190.
> > 
> > 19:47:32.285397 IP (tos 0x0, ttl  64, id 30055, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x41e5 (correct), 3216:3216(0) ack 2215 win 10701
> > 19:47:32.320287 IP (tos 0x0, ttl 238, id 10, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x45e6 (correct), 1190:1190(0) ack 3216 win 10701
> > 19:47:32.320300 IP (tos 0x0, ttl  64, id 30056, offset 0, flags [DF], proto: TCP (6), length: 40) dyn-67.arm.linux.org.uk.38803 > 193.108.74.209.http: ., cksum 0x41e5 (correct), 3216:3216(0) ack 2215 win 10701
> > 19:47:32.353016 IP (tos 0x0, ttl 238, id 12, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > dyn-67.arm.linux.org.uk.38803: ., cksum 0x45e6 (correct), 1190:1190(0) ack 3216 win 10701
> > ...
> > 
> > So it looks to me like the remote end doesn't like our ack of 2215 for
> > some reason.
> 
> Let's make sure the checksum is OK.  That's a reason the ACK might
> get dropped at the remote end.

I doubt the remote end will co-operate by tcpdumping my session as
received at their end, the best I can do is a dump from my firewall
(which is doing masq, public facing NIC is a 3c589 PCMCIA card.)

But, locally:

8139too Fast Ethernet driver 0.9.28
8139too 0000:01:05.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
eth0: RealTek RTL8139 at 0xf8914c00, 00:13:8f:cb:34:ef, IRQ 22
eth0:  Identified 8139 chip type 'RTL-8101'

# ethtool -k eth0
Offload parameters for eth0:
Cannot get device rx csum settings: Operation not supported
rx-checksumming: off
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: off

Obtaining a tcpdump off my firewall shows that the checksums are correct:

21:48:42.629025 IP (tos 0x0, ttl  48, id 21841, offset 0, flags [DF], proto: TCP (6), length: 432) 193.108.74.209.http > caramon.arm.linux.org.uk.38519: P, cksum 0xb7de (correct), 44198:44590(392) ack 12652 win 26796
21:48:42.630359 IP (tos 0x0, ttl  48, id 21842, offset 0, flags [DF], proto: TCP (6), length: 595) 193.108.74.209.http > caramon.arm.linux.org.uk.38519: P, cksum 0xd18b (correct), 44590:45145(555) ack 12652 win 26796
21:48:42.633342 IP (tos 0x0, ttl  48, id 21843, offset 0, flags [DF], proto: TCP (6), length: 458) 193.108.74.209.http > caramon.arm.linux.org.uk.38519: P, cksum 0x5e49 (correct), 45145:45563(418) ack 13836 win 29348
21:48:42.637288 IP (tos 0x0, ttl  63, id 39663, offset 0, flags [DF], proto: TCP (6), length: 40) caramon.arm.linux.org.uk.38519 > 193.108.74.209.http: ., cksum 0x8d85 (correct), 13836:13836(0) ack 45145 win 62780
21:48:42.639376 IP (tos 0x0, ttl  63, id 39664, offset 0, flags [DF], proto: TCP (6), length: 1316) caramon.arm.linux.org.uk.38519 > 193.108.74.209.http: ., cksum 0x8978 (correct), 13836:15112(1276) ack 45145 win 62780
21:48:42.641154 IP (tos 0x0, ttl  63, id 39665, offset 0, flags [DF], proto: TCP (6), length: 47) caramon.arm.linux.org.uk.38519 > 193.108.74.209.http: P, cksum 0x0987 (correct), 15112:15119(7) ack 45563 win 62780
21:48:42.667103 IP (tos 0x0, ttl  48, id 21844, offset 0, flags [DF], proto: TCP (6), length: 446) 193.108.74.209.http > caramon.arm.linux.org.uk.38519: P, cksum 0x99e7 (correct), 45563:45969(406) ack 13836 win 29348
21:48:42.721132 IP (tos 0x0, ttl  63, id 39666, offset 0, flags [DF], proto: TCP (6), length: 40) caramon.arm.linux.org.uk.38519 > 193.108.74.209.http: ., cksum 0x854a (correct), 15119:15119(0) ack 45969 win 62780
21:48:42.739767 IP (tos 0x0, ttl 239, id 7, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > caramon.arm.linux.org.uk.38519: ., cksum 0x86e0 (correct), 45563:45563(0) ack 15119 win 62780
21:48:42.741603 IP (tos 0x0, ttl  63, id 39667, offset 0, flags [DF], proto: TCP (6), length: 40) caramon.arm.linux.org.uk.38519 > 193.108.74.209.http: ., cksum 0x854a (correct), 15119:15119(0) ack 45969 win 62780
21:48:42.759254 IP (tos 0x0, ttl 239, id 6, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > caramon.arm.linux.org.uk.38519: ., cksum 0x86e0 (correct), 45563:45563(0) ack 15119 win 62780
21:48:42.761080 IP (tos 0x0, ttl  63, id 39668, offset 0, flags [DF], proto: TCP (6), length: 40) caramon.arm.linux.org.uk.38519 > 193.108.74.209.http: ., cksum 0x854a (correct), 15119:15119(0) ack 45969 win 62780
21:48:42.779423 IP (tos 0x0, ttl 239, id 4, offset 0, flags [none], proto: TCP (6), length: 40) 193.108.74.209.http > caramon.arm.linux.org.uk.38519: ., cksum 0x86e0 (correct), 45563:45563(0) ack 15119 win 62780
21:48:42.781247 IP (tos 0x0, ttl  63, id 39669, offset 0, flags [DF], proto: TCP (6), length: 40) caramon.arm.linux.org.uk.38519 > 193.108.74.209.http: ., cksum 0x854a (correct), 15119:15119(0) ack 45969 win 62780

-- 
Russell King