Re: TCP IPv4 strange retransmits

[Arnd Hannemann <hannemann@nets.rwth-aachen.de>]

Ilpo Järvinen wrote:
> On Tue, 4 Mar 2008, Arnd Hannemann wrote:
> 
>> Hi,
>>
>> Ilpo Järvinen wrote:
>>> On Tue, 4 Mar 2008, Arnd Hannemann wrote:
>>>
>>>> I'm observing some retransmits with kernel 2.6.24.2, which I don't 
>>>> understand. For instance in this cutout[1] of a sequence diagram which 
>>>> was captured[2] on the TCP sender, 4 retransmits are made.
>>> They don't correspond to each other?
>> Hmm, they should.
> 
> Yeah, they probably do, I was just too hasty and failed to notice those 
> small negative offsets.
> 
>>>> According to netstat -st output[3][4] all those 4 retransmits were "fast 
>>>> retransmit".
>>>> But there are no three DUPACKs which I expected would be needed for fast 
>>>> retransmit?
>>> With FACK it's enough that you have fackets_out > tp->reordering 
>>> (=dupThresh).
>> If it is FACK shouldn't it be accounted for LINUX_MIB_TCPFORWARDRETRANS
>> instead of LINUX_MIB_TCPFASTRETRANS?
> 
> No, if there's any skb which is more than fackets_out-tp->reordering from 
> the highest SACKed skb, it will be marked TCPCB_LOST (see 
> tcp_mark_head_lost & it's caller), and all LOST segments are retransmitted 
> by the earlier loop (for a while still as I'm going to very likely change 
> that in net-2.6.26, commits for consolidating both, nearly identical loops 
> are already in my local git and await some testing).
> 
> Forwardretrans is only incremented when there isn't TCPCB_LOST set for a 
> segment and it doesn't apply in this case anyway because you have new data 
> to send (see the decision making for forward retransmits, it's well 
> commented btw).

Ah, I see. Thank you for clarifying.
However fackets_out is not so well documented ;-)
But it now makes all sense (with dump order):
An ACK 19225 arrives with SACK block {27745:29165}, so fackets_out becomes ~6 ((27745-19225)/1450)
tp->reordering is 3 at this time so he starts to retransmit.
However some SACK ACK comes early enough so he stops at 4 retransmits.
Or something like that...

> 
>>>> Also interesting all retransmits happen _after_ those segments were
>>>> already acked and sacked, internal queuing or latency issues?
>>> I think your viewer is doing something wrong, sender.dump is not giving 
>>> such information (or you draw that from wrong end?). Or it just draws
>>> DSACK like that?
>> Viewer is tcptrace and xplot. So nothing special at all.
>> You see it also in wireshark, if you draw a sequence diagram.
> 
> Ah, now I noticed those small timeleaps, very small enough to not
> catch my eye earlier as the amount of numbers in such screen is just
> overhelming... :-)

Very small indeed. Probably the time a packets travels in kernel through the layer
is higher than the difference between ACK and retransmit.

> 
>> You also see it in wireshark if you sort by capture timestamp. I always 
>> thought that capture timestamp order is correct and not dump order, but 
>> maybe I'm wrong?
> 
> I'm not sure, in the other order they make very much sense. In addition, 
> the ACKs are processed in order and their effects are immediate even if 
> there's more information awaiting to be processed.

> 
>> Tcpdump:
>>
>> 12:08:20.667538 IP 192.168.0.7.33824 > 192.168.0.5.50139: . ack 23485 win 22720 <nop,nop,timestamp 969759 972885,nop,nop,sack 2 {24905:26325}{27745:29165}>
>> ^^^^^ got acked at .667538
> 
> Did you paste wrong timestamp as 667538 == 667538? ...It just makes no 
> sense for me, what are you trying to say here?
> 
>> 12:08:20.646749 IP 192.168.0.5.50139 > 192.168.0.7.33824: . 22065:23485(1420) ack 1 win 2864 <nop,nop,timestamp 972885 969754>
>> ^^^^^ got retransmitted at .646749
> 
> What's the problem here? At .646749 something was retransmitted, but only 
> after .667538 it was acked? Again, this makes very little sense for me...
> Why did you copy them wrong way around from the tcpdump log? Or are these 
> two lines related at all?

Sorry, this was just bogus. Just wanted to point out the timestamp differences and made a
wrong example. Screen full of numbers... ;-)

Thanks for your help.

Best regards,
Arnd