TCP - RST flag

TCP - RST flag

[Daniel Baluta]

[-- Attachment #1: Type: text/plain, Size: 2574 bytes --]

Hello,

Please help me understanding the behavior of the following
TCP conversation.

You can find bellow a snippet of the (FTP) conversation captured both
on client (C) and server (S).

[client]$ tcpdump  -n  -r client-6-conv.cap
[P1] 49.045690 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
197:220, ack 81, win 757, length 23
[P2] 49.046600 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
win 738, length 0
[P3] 49.047462 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [P.], seq
81:87, ack 220, win 738, length 6
[P5] 49.048757 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [F.], seq
242, ack 87, win 757, length 0
[P6] 49.048794 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
win 738, options [nop,nop,sack 1 {242:243}], length 0
[P4] 49.048801 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
220:242, ack 87, win 757, length 22
[P7] 49.048833 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 243,
win 715, length 0
[P8] 49.049566 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [F.], seq 87,
ack 243, win 715, length 0
[P9] 49.050889 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [R], seq
1672731590, win 0, length 0

[server]$ tcpdump  -n  -r server-6-conv.cap
[P1] 49.059740 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
197:220, ack 81, win 757, length 23
[P2] 49.061394 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
win 738, length 0
[P3] 49.061760 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [P.], seq
81:87, ack 220, win 738, length 6
[P4] 49.062794 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
220:242, ack 87, win 757, length 22
[P5] 49.062843 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [F.], seq
242, ack 87, win 757, length 0
[P6] 49.063808 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
win 738, options [nop,nop,sack 1 {242:243}], length 0
[P7] 49.063823 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 243,
win 715, length 0
[P8] 49.064271 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [F.], seq 87,
ack 243, win 715, length 0
[P9] 49.064481 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [R], seq
1672731590, win 0, length 0

What happens is that servers sends packets P4 and P5, but client
receives P5 before P4.
Since SACK is enabled, client will send a SACK (P6) ack-ing P5.

Then client sees P4, and will send an ack (P7) for P4, then sends P8
with FIN flag set.
What I don't understand, is why server responds with RST (P9) instead of ACK?

This is was obtained on 2.6.32.43. I have also attached full capture files.

I am reading TCP's RFC and kernel code, but so far I haven't reached
a conclusion.

thanks,
Daniel.

[-- Attachment #2: server-6-conv.cap --]
[-- Type: application/octet-stream, Size: 2209 bytes --]

[-- Attachment #3: client-6-conv.cap --]
[-- Type: application/octet-stream, Size: 2197 bytes --]

Re: TCP - RST flag

[Eric Dumazet]

Le mardi 23 août 2011 à 23:31 +0300, Daniel Baluta a écrit :
> Hello,
> 
> Please help me understanding the behavior of the following
> TCP conversation.
> 
> You can find bellow a snippet of the (FTP) conversation captured both
> on client (C) and server (S).
> 
> [client]$ tcpdump  -n  -r client-6-conv.cap
> [P1] 49.045690 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
> 197:220, ack 81, win 757, length 23
> [P2] 49.046600 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
> win 738, length 0
> [P3] 49.047462 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [P.], seq
> 81:87, ack 220, win 738, length 6
> [P5] 49.048757 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [F.], seq
> 242, ack 87, win 757, length 0
> [P6] 49.048794 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
> win 738, options [nop,nop,sack 1 {242:243}], length 0
> [P4] 49.048801 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
> 220:242, ack 87, win 757, length 22
> [P7] 49.048833 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 243,
> win 715, length 0
> [P8] 49.049566 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [F.], seq 87,
> ack 243, win 715, length 0
> [P9] 49.050889 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [R], seq
> 1672731590, win 0, length 0
> 
> [server]$ tcpdump  -n  -r server-6-conv.cap
> [P1] 49.059740 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
> 197:220, ack 81, win 757, length 23
> [P2] 49.061394 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
> win 738, length 0
> [P3] 49.061760 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [P.], seq
> 81:87, ack 220, win 738, length 6
> [P4] 49.062794 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
> 220:242, ack 87, win 757, length 22
> [P5] 49.062843 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [F.], seq
> 242, ack 87, win 757, length 0
> [P6] 49.063808 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
> win 738, options [nop,nop,sack 1 {242:243}], length 0
> [P7] 49.063823 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 243,
> win 715, length 0
> [P8] 49.064271 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [F.], seq 87,
> ack 243, win 715, length 0
> [P9] 49.064481 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [R], seq
> 1672731590, win 0, length 0
> 
> What happens is that servers sends packets P4 and P5, but client
> receives P5 before P4.
> Since SACK is enabled, client will send a SACK (P6) ack-ing P5.
> 
> Then client sees P4, and will send an ack (P7) for P4, then sends P8
> with FIN flag set.
> What I don't understand, is why server responds with RST (P9) instead of ACK?
> 
> This is was obtained on 2.6.32.43. I have also attached full capture files.
> 
> I am reading TCP's RFC and kernel code, but so far I haven't reached
> a conclusion.

TCP in RFC 1122 section 4.2.2.13:

 "A host MAY implement a "half-duplex" TCP close sequence, so that an
 application that has called CLOSE cannot continue to read data from the
 connection. If such a host issues a CLOSE call while received data is
 still pending in TCP, or if new data is received after CLOSE is called,
 its TCP SHOULD send a RST to show that data was lost." 

This is why some apps first call shutdown(), then drain receive queue,
then close()

Re: TCP - RST flag

[Daniel Baluta]

On Tue, Aug 23, 2011 at 11:55 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mardi 23 août 2011 à 23:31 +0300, Daniel Baluta a écrit :
>> Hello,
>>
>> Please help me understanding the behavior of the following
>> TCP conversation.
>>
>> You can find bellow a snippet of the (FTP) conversation captured both
>> on client (C) and server (S).
>>
>> [client]$ tcpdump  -n  -r client-6-conv.cap
>> [P1] 49.045690 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
>> 197:220, ack 81, win 757, length 23
>> [P2] 49.046600 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
>> win 738, length 0
>> [P3] 49.047462 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [P.], seq
>> 81:87, ack 220, win 738, length 6
>> [P5] 49.048757 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [F.], seq
>> 242, ack 87, win 757, length 0
>> [P6] 49.048794 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
>> win 738, options [nop,nop,sack 1 {242:243}], length 0
>> [P4] 49.048801 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
>> 220:242, ack 87, win 757, length 22
>> [P7] 49.048833 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 243,
>> win 715, length 0
>> [P8] 49.049566 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [F.], seq 87,
>> ack 243, win 715, length 0
>> [P9] 49.050889 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [R], seq
>> 1672731590, win 0, length 0
>>
>> [server]$ tcpdump  -n  -r server-6-conv.cap
>> [P1] 49.059740 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
>> 197:220, ack 81, win 757, length 23
>> [P2] 49.061394 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
>> win 738, length 0
>> [P3] 49.061760 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [P.], seq
>> 81:87, ack 220, win 738, length 6
>> [P4] 49.062794 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [P.], seq
>> 220:242, ack 87, win 757, length 22
>> [P5] 49.062843 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [F.], seq
>> 242, ack 87, win 757, length 0
>> [P6] 49.063808 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 220,
>> win 738, options [nop,nop,sack 1 {242:243}], length 0
>> [P7] 49.063823 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [.], ack 243,
>> win 715, length 0
>> [P8] 49.064271 IP 10.10.0.20.58277 > 10.10.0.1.21: Flags [F.], seq 87,
>> ack 243, win 715, length 0
>> [P9] 49.064481 IP 10.10.0.1.21 > 10.10.0.20.58277: Flags [R], seq
>> 1672731590, win 0, length 0
>>
>> What happens is that servers sends packets P4 and P5, but client
>> receives P5 before P4.
>> Since SACK is enabled, client will send a SACK (P6) ack-ing P5.
>>
>> Then client sees P4, and will send an ack (P7) for P4, then sends P8
>> with FIN flag set.
>> What I don't understand, is why server responds with RST (P9) instead of ACK?
>>
>> This is was obtained on 2.6.32.43. I have also attached full capture files.
>>
>> I am reading TCP's RFC and kernel code, but so far I haven't reached
>> a conclusion.
>
> TCP in RFC 1122 section 4.2.2.13:
>
>  "A host MAY implement a "half-duplex" TCP close sequence, so that an
>  application that has called CLOSE cannot continue to read data from the
>  connection. If such a host issues a CLOSE call while received data is
>  still pending in TCP, or if new data is received after CLOSE is called,
>  its TCP SHOULD send a RST to show that data was lost."

So, this means that server's CLOSE operation is issued while received
data is still pending? I will analyze ftp's server code, but this is strange
since P4 [221 Have a nice day!\r\n] it is generated as a response for
P3 [QUIT\r\n]. So P4 must have been fully received.

Also, looking at the capture no data is received from the client after
server calls CLOSE (P5) (there are only ACKs and FIN - P6, P7, P8).

> This is why some apps first call shutdown(), then drain receive queue,
> then close()

This makes sense :).

thanks Eric.

Daniel.

Re: TCP - RST flag

[Eric Dumazet]

Le mercredi 24 août 2011 à 00:32 +0300, Daniel Baluta a écrit :
> On Tue, Aug 23, 2011 at 11:55 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:

> > TCP in RFC 1122 section 4.2.2.13:
> >
> >  "A host MAY implement a "half-duplex" TCP close sequence, so that an
> >  application that has called CLOSE cannot continue to read data from the
> >  connection. If such a host issues a CLOSE call while received data is
> >  still pending in TCP, or if new data is received after CLOSE is called,
> >  its TCP SHOULD send a RST to show that data was lost."
> 
> So, this means that server's CLOSE operation is issued while received
> data is still pending? I will analyze ftp's server code, but this is strange
> since P4 [221 Have a nice day!\r\n] it is generated as a response for
> P3 [QUIT\r\n]. So P4 must have been fully received.
> 

tcpdump only shows TCP stack did receive the frame, not that it was
_read_ by application.

Only strace could eventually say if the input queue was really drained
before the close().

Maybe server decided to close the connexion before reading the QUIT
command from client.

> Also, looking at the capture no data is received from the client after
> server calls CLOSE (P5) (there are only ACKs and FIN - P6, P7, P8).

That doesnt matter.

If data was received (and ACKnowledged by TCP stack) _before_ close(fd),
but not yet read by application, TCP must send an RST to be RFC
compliant.

Re: TCP - RST flag

[Daniel Baluta]

On Wed, Aug 24, 2011 at 12:50 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mercredi 24 août 2011 à 00:32 +0300, Daniel Baluta a écrit :
>> On Tue, Aug 23, 2011 at 11:55 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
>> > TCP in RFC 1122 section 4.2.2.13:
>> >
>> >  "A host MAY implement a "half-duplex" TCP close sequence, so that an
>> >  application that has called CLOSE cannot continue to read data from the
>> >  connection. If such a host issues a CLOSE call while received data is
>> >  still pending in TCP, or if new data is received after CLOSE is called,
>> >  its TCP SHOULD send a RST to show that data was lost."
>>
>> So, this means that server's CLOSE operation is issued while received
>> data is still pending? I will analyze ftp's server code, but this is strange
>> since P4 [221 Have a nice day!\r\n] it is generated as a response for
>> P3 [QUIT\r\n]. So P4 must have been fully received.
>>
>
> tcpdump only shows TCP stack did receive the frame, not that it was
> _read_ by application.
>
> Only strace could eventually say if the input queue was really drained
> before the close().
> Maybe server decided to close the connexion before reading the QUIT
> command from client.
>
>> Also, looking at the capture no data is received from the client after
>> server calls CLOSE (P5) (there are only ACKs and FIN - P6, P7, P8).
>
> That doesnt matter.
>
> If data was received (and ACKnowledged by TCP stack) _before_ close(fd),
> but not yet read by application, TCP must send an RST to be RFC
> compliant.

I see. Thanks a lot for your explanations. I will further use strace
to see what's going on.

Daniel.