Re: [PATCH] tcp: ack when we get an OOO/lost packet

[Josef Bacik <jbacik@fb.com>]

On 08/13/2015 04:19 AM, Andrei Borzenkov wrote:
> On Wed, Aug 12, 2015 at 6:16 PM, Josef Bacik <jbacik@fb.com> wrote:
>> While adding tcp window scaling support I was finding that I'd get some packet
>> loss or reordering when transferring from large distances and grub would just
>> timeout.  This is because we weren't ack'ing when we got our OOO packet, so the
>> sender didn't know it needed to retransmit anything, so eventually it would fill
>> the window and stop transmitting, and we'd time out.  Fix this by ACK'ing when
>> we don't find our next sequence numbered packet.  With this fix I no longer time
>> out.  Thanks,
>
> I have a feeling that your description is misleading. Patch simply
> sends duplicated ACK, but partner does not know what has been received
> and what has not, so it must wait for ACK timeout anyway before
> retransmitting. What this patch may fix would be lost ACK packet
> *from* GRUB, by increasing rate of ACK packets it sends. Do you have
> packet trace for timeout case, ideally from both sides simultaneously?
>

The way linux works is that if you get <configurable amount> of DUP 
ack's it triggers a retransmit.  I only have traces from the server 
since tcpdump doesn't work in grub (or if it does I don't know how to do 
it).  The server is definitely getting all of the ACK's, and from my 
printf()'ing in grub we are either getting re-ordered packets (the most 
likely) or we are simply losing a packet here or there.  This is a 
pretty long distance and we have a lot of networking between Sweden and 
California so reordering or packet loss isn't out of the question.

Regardless we definitely need to be ACK'ing packets that come in with 
the last seq we had as the spec says so the sender knows the last bit we 
had, otherwise we see timeouts once the window is full.

> Did you consider implementing receive side SACK BTW? You have the
> right environment to test it :)
>

So I found this bug while implementing SACK, and decided it was faster 
to just do this rather than add SACK.  This method is still exceedingly 
slow, I only get around 800kb/s over the entire transfer whereas I can 
sustain around 5.5 mb/s before we start losing stuff, so I'm definitely 
going to go back and try the timestamp echo stuff since the timeout 
stuff takes like 6 seconds, and then if that doesn't work bite the 
bullet and add SACK.

But first I want to get my ipv6 patches right ;).  Thanks,

Josef