Non-standard TCP stack processing of packets with unacceptable ACK numbers

[Paul Fiterau Brostean <p.fiterau-brostean@science.ru.nl>]

[-- Attachment #1: Type: text/plain, Size: 3457 bytes --]

Hello,

My name is Paul Fiterau, I am a PhD student at Radboud University whose 
focus for the past few years has been among others to develop and apply 
inference techniques on TCP stacks in order to obtain nice models, and 
to verify them if possible using formal methods.  We contacted you on 
something similar 2 years back.

The older (3.19 kernel release) Linux TCP stack we analyze exhibits 
behavior that seems odd to me. The scenario is as follows (all packets 
have empty payloads, no window scaling, rcv/snd window size should not 
be a factor):

       TEST HARNESS (CLIENT)                        LINUX SERVER

   1.  -                                          LISTEN (server listen, then accepts)

   2.  - --> <SEQ=100><CTL=SYN>               --> SYN-RECEIVED

   3.  - <-- <SEQ=300><ACK=101><CTL=SYN,ACK>  <-- SYN-RECEIVED

   4.  - --> <SEQ=101><ACK=301><CTL=ACK>      --> ESTABLISHED
  
   5.  - <-- <SEQ=301><ACK=101><CTL=FIN,ACK>  <-- FIN WAIT-1 (server opts to close the data connection calling "close" on the connection socket)

   6.  - --> <SEQ=101><ACK=99999><CTL=FIN,ACK> --> CLOSING (client sends FIN,ACK with not yet sent acknowledgement number)

   7.  - <-- <SEQ=302><ACK=102><CTL=ACK>      <-- CLOSING (ACK is 102 instead of 101, why?)
  
... (silence from CLIENT)

   8.  - <-- <SEQ=301><ACK=102><CTL=FIN,ACK>  <-- CLOSING (retransmission, again ACK is 102)


Now, note that packet 6 while having the expected sequence number, 
acknowledges something that wasn't sent by the server. So I would expect 
the packet to maybe prompt an ACK response from the server, and then be 
ignored. Yet it is not ignored and actually leads to an increase of the 
acknowledgement number in the server's retransmission of the FIN,ACK 
packet. The explanation I found is that the FIN  in packet 6 was 
processed, despite the acknowledgement number being unacceptable. 
Further experiments indeed show that the server processes this FIN, 
transitioning to CLOSING, then on receiving an ACK for the FIN it had 
send in packet 5, the server (or better said connection) transitions 
from CLOSING to TIME_WAIT (as signaled by netstat).


I attached a capture showing the scenario, as well as an equivalent 
capture for a Windows 10 TCP server, which behaves exactly as I would 
expect by  not increasing the expected sequence number after packet 6, 
and thus not processing the FIN flag received.

I hope someone more knowledgeable can clear this up for me. Is it ok for 
the server to process the FIN bit in a packet with an unacceptable 
acknowledgement number? Could this be an inconsistency in the tested stack?


Thanks, Paul.

P.S. Our most recent publication on relating to TCP was at CAV (some big conference), article link for anyone interested
http://www.sws.cs.ru.nl/publications/papers/fvaan/FJV16/main.pdf . Now we are working on applying more advanced techniques that abstract less and capture more behavior.
P.S.2 Potentially violated RFC fragment:
    3.  If the connection is in a synchronized state (ESTABLISHED,
     FIN-WAIT-1, FIN-WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT),
     any unacceptable segment (out of window sequence number or
     unacceptible acknowledgment number) must elicit only an empty
     acknowledgment segment containing the current send-sequence number
     and an acknowledgment indicating the next sequence number expected
     to be received, and *the connection remains in the same state*.


[-- Attachment #2: linux.pcapng --]
[-- Type: application/octet-stream, Size: 916 bytes --]

[-- Attachment #3: win.pcapng --]
[-- Type: application/octet-stream, Size: 824 bytes --]