[TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[Yinglin Sun]

Hi,

I have a question about tcp_ca_state "TCP_CA_CWR".  Under what
circumstances does it enter this state?
I'm doing some experiments and have trace from tcp_info. From tcp_info
trace, I found that many places show ca_state is 2 (TCP_CA_CWR). Under
this state, the congestion window size decreases but the total number
of retransmissions is still 0.

I got some explanation from a very old message posted in 2003:
TCP_CA_CWR: The congestion window should be reduced due to some
congestion notification event, which can be ECN, ICMP source quench,
or local device congestion. Both sender and receiver don't have ECN
enabled, so ECN is not the case. ICMP source quench is so out-of-date
with RFC 1812 saying the routers should not issue these packets. So it
might be caused by local congestion. But this post is too old, almost
done 6 years ago. So I want to check if there are other possible
causes in today's kernel leading to TCP_CA_CWR with 0 retransmissions?

I need to find the exact cause for this phenomena, local congestion,
or other possible causes. Any hint will be very helpful for me.

Thanks a lot!

Yinglin

Re: [TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[David Miller]

From: Yinglin Sun <yinglin.s@gmail.com>
Date: Tue, 14 Jul 2009 14:38:07 -0400

> I have a question about tcp_ca_state "TCP_CA_CWR".  Under what
> circumstances does it enter this state?
> I'm doing some experiments and have trace from tcp_info. From tcp_info
> trace, I found that many places show ca_state is 2 (TCP_CA_CWR). Under
> this state, the congestion window size decreases but the total number
> of retransmissions is still 0.

What kind of "retransmissions"?  I bet the counter you are looking
at is counting timeout based retransmissions, which means it isn't
including the most predominant type of retransmissions, which are
fast retransmissions which are triggered in response to ACKs.

Re: [TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[Yinglin Sun]

Hi David,

I use the total number of retransmissions reported by tcp_info. The
field name is tcpi_total_retrans. From the kernel source code, I found
that this number is for both fast retransmit and timeout
retransmissions. On the other hand, if fast retransmission happened,
ca_state should be TCP_CA_Recovery or TCP_CA_Disorder, but it's
TCP_CA_CWR.

Thanks.

Yinglin

On Tue, Jul 14, 2009 at 2:42 PM, David Miller<davem@davemloft.net> wrote:
> From: Yinglin Sun <yinglin.s@gmail.com>
> Date: Tue, 14 Jul 2009 14:38:07 -0400
>
>> I have a question about tcp_ca_state "TCP_CA_CWR".  Under what
>> circumstances does it enter this state?
>> I'm doing some experiments and have trace from tcp_info. From tcp_info
>> trace, I found that many places show ca_state is 2 (TCP_CA_CWR). Under
>> this state, the congestion window size decreases but the total number
>> of retransmissions is still 0.
>
> What kind of "retransmissions"?  I bet the counter you are looking
> at is counting timeout based retransmissions, which means it isn't
> including the most predominant type of retransmissions, which are
> fast retransmissions which are triggered in response to ACKs.
>

Re: [TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[David Miller]

From: Yinglin Sun <yinglinsun@cs.pitt.edu>
Date: Tue, 14 Jul 2009 15:12:16 -0400

> I use the total number of retransmissions reported by tcp_info. The
> field name is tcpi_total_retrans. From the kernel source code, I found
> that this number is for both fast retransmit and timeout
> retransmissions. On the other hand, if fast retransmission happened,
> ca_state should be TCP_CA_Recovery or TCP_CA_Disorder, but it's
> TCP_CA_CWR.

Great, that if you're reading the code you also see that there
are many code paths that invoke tcp_enter_cwr() that can occur
without any retransmissions. :-)

One such case is when ECN congestion notification bits are
seen in an ACK packet.

Re: [TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[Yinglin Sun]

On Tue, Jul 14, 2009 at 3:15 PM, David Miller<davem@davemloft.net> wrote:
> From: Yinglin Sun <yinglinsun@cs.pitt.edu>
> Date: Tue, 14 Jul 2009 15:12:16 -0400
>
>> I use the total number of retransmissions reported by tcp_info. The
>> field name is tcpi_total_retrans. From the kernel source code, I found
>> that this number is for both fast retransmit and timeout
>> retransmissions. On the other hand, if fast retransmission happened,
>> ca_state should be TCP_CA_Recovery or TCP_CA_Disorder, but it's
>> TCP_CA_CWR.
>
> Great, that if you're reading the code you also see that there
> are many code paths that invoke tcp_enter_cwr() that can occur
> without any retransmissions. :-)
>
> One such case is when ECN congestion notification bits are
> seen in an ACK packet.
>

Hi David,

Thanks for your reminding. I have checked all paths to
tcp_enter_cwr(). Two of them are FRTO and ECN. Both frto and ECN on my
sender and receiver are NOT enabled, so these two are not causes.
Another one is tcp_transmit_skb(). It seems that tcp_transmit_skb
detects if TX Queue is full. If so, it calls tcp_enter_cwr. I double
checked these invocation paths. If  I found all paths, the cause for
me is that TX Q is full, that is, local congestion. But it is also
possible that I missed other possible causes. Do you have any idea
about other invocation to tecp_enter_cwr?

Thanks a lot.

Yinglin

Re: [TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[David Miller]

From: Yinglin Sun <yinglinsun@cs.pitt.edu>
Date: Tue, 14 Jul 2009 15:49:52 -0400

> Do you have any idea about other invocation to tecp_enter_cwr?

I really think I've helped you as much as I could as this
point, you can read the code as easily as I could and that's
all that I would do to answer your question.  I haven't
memorized this stuff nor do I remember things like "all the
code patchs that invoke function X"

Re: [TCP_CA_CWR] Causes for entering TCP_CA_CWR state with 0 retransmissions

[Yinglin Sun]

On Tue, Jul 14, 2009 at 3:53 PM, David Miller<davem@davemloft.net> wrote:
> From: Yinglin Sun <yinglinsun@cs.pitt.edu>
> Date: Tue, 14 Jul 2009 15:49:52 -0400
>
>> Do you have any idea about other invocation to tecp_enter_cwr?
>
> I really think I've helped you as much as I could as this
> point, you can read the code as easily as I could and that's
> all that I would do to answer your question.  I haven't
> memorized this stuff nor do I remember things like "all the
> code patchs that invoke function X"
>

David, I'm sorry. You are right. I shouldn't ask question like that.
You have given me enough hints about my questions. My fault.

Thanks again.

Yinglin