* [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix
@ 2007-10-11 11:41 Ilpo Järvinen
2007-10-11 11:41 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too Ilpo Järvinen
2007-10-12 0:37 ` [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix David Miller
0 siblings, 2 replies; 22+ messages in thread
From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw)
To: David Miller; +Cc: netdev, TAKANO Ryousei
Some small and simple changes. Minor testing done, until I got
one LostRetrans counted, which turned out to be hard task than
I thought (without specifically arranged scenario, just some
basic netem delay & drops).
Lost_retrans handling fix is now tested, I fixed one incorrectly
placed if from the lost_retrans_low patch after reading it
through.
...Recount removal patch could be more elegant in the way it
plays around with the tcp_clear_retrans, but I just wasn't able
to figure out how to do that nicely enough...
Those cleanups are preparation for DSACK fix, which is yet to
be done but IMHO they won't hurt if applied alone either.
Dave, please apply at will. FYI, I'll post also the SACK rewrite
patchset rebased after this (built on top of these).
--
i.
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too 2007-10-11 11:41 [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix Ilpo Järvinen @ 2007-10-11 11:41 ` Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L Ilpo Järvinen 2007-10-12 0:32 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too David Miller 2007-10-12 0:37 ` [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix David Miller 1 sibling, 2 replies; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw) To: David Miller; +Cc: netdev, TAKANO Ryousei I was reading tcp_enter_loss while looking for Cedric's bug and noticed bytes_acked adjustment is missing from FRTO side. Since bytes_acked will only be used in tcp_cong_avoid, I think it's safe to assume RTO would be spurious. During FRTO cwnd will be not controlled by tcp_cong_avoid and if FRTO calls for conventional recovery, cwnd is adjusted and the result of wrong assumption is cleared from bytes_acked. If RTO was in fact spurious, we did normal ABC already and can continue without any additional adjustments. Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 101054c..e40857e 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1687,6 +1687,7 @@ static void tcp_enter_frto_loss(struct sock *sk, int allowed_segments, int flag) tp->snd_cwnd_cnt = 0; tp->snd_cwnd_stamp = tcp_time_stamp; tp->frto_counter = 0; + tp->bytes_acked = 0; tp->reordering = min_t(unsigned int, tp->reordering, sysctl_tcp_reordering); @@ -2824,6 +2825,7 @@ static void tcp_conservative_spur_to_response(struct tcp_sock *tp) { tp->snd_cwnd = min(tp->snd_cwnd, tp->snd_ssthresh); tp->snd_cwnd_cnt = 0; + tp->bytes_acked = 0; TCP_ECN_queue_cwr(tp); tcp_moderate_cwnd(tp); } -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L 2007-10-11 11:41 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too Ilpo Järvinen @ 2007-10-11 11:41 ` Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag Ilpo Järvinen 2007-10-12 0:33 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L David Miller 2007-10-12 0:32 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too David Miller 1 sibling, 2 replies; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw) To: David Miller; +Cc: netdev, TAKANO Ryousei This condition (plain R) can arise at least in recovery that is triggered after tcp_undo_loss. There isn't any reason why they should not be marked as lost, not marking makes in_flight estimator to return too large values. Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index e40857e..c827285 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1987,7 +1987,7 @@ static void tcp_mark_head_lost(struct sock *sk, cnt += tcp_skb_pcount(skb); if (cnt > packets || after(TCP_SKB_CB(skb)->end_seq, high_seq)) break; - if (!(TCP_SKB_CB(skb)->sacked&TCPCB_TAGBITS)) { + if (!(TCP_SKB_CB(skb)->sacked & (TCPCB_SACKED_ACKED|TCPCB_LOST))) { TCP_SKB_CB(skb)->sacked |= TCPCB_LOST; tp->lost_out += tcp_skb_pcount(skb); tcp_verify_retransmit_hint(tp, skb); -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag 2007-10-11 11:41 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L Ilpo Järvinen @ 2007-10-11 11:41 ` Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code Ilpo Järvinen 2007-10-12 0:34 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag David Miller 2007-10-12 0:33 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L David Miller 1 sibling, 2 replies; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw) To: David Miller; +Cc: netdev, TAKANO Ryousei It's on the way for future cutting of that function. Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index c827285..5004704 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1314,7 +1314,7 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ flag |= FLAG_DATA_LOST; tcp_for_write_queue_from(skb, sk) { - int in_sack, pcount; + int in_sack; u8 sacked; if (skb == tcp_send_head(sk)) @@ -1336,9 +1336,7 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ in_sack = !after(start_seq, TCP_SKB_CB(skb)->seq) && !before(end_seq, TCP_SKB_CB(skb)->end_seq); - pcount = tcp_skb_pcount(skb); - - if (pcount > 1 && !in_sack && + if (tcp_skb_pcount(skb) > 1 && !in_sack && after(TCP_SKB_CB(skb)->end_seq, start_seq)) { unsigned int pkt_len; @@ -1353,10 +1351,9 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ TCP_SKB_CB(skb)->seq); if (tcp_fragment(sk, skb, pkt_len, skb_shinfo(skb)->gso_size)) break; - pcount = tcp_skb_pcount(skb); } - fack_count += pcount; + fack_count += tcp_skb_pcount(skb); sacked = TCP_SKB_CB(skb)->sacked; -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code 2007-10-11 11:41 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag Ilpo Järvinen @ 2007-10-11 11:41 ` Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO Ilpo Järvinen 2007-10-12 0:34 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code David Miller 2007-10-12 0:34 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag David Miller 1 sibling, 2 replies; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw) To: David Miller; +Cc: netdev, TAKANO Ryousei This is necessary for upcoming DSACK bugfix. Reduces sacktag length which is not very sad thing at all... :-) Notice that there's a need to handle out-of-mem at caller's place. Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 54 ++++++++++++++++++++++++++++++++----------------- 1 files changed, 35 insertions(+), 19 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 5004704..bd18c25 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1181,6 +1181,38 @@ static int tcp_check_dsack(struct tcp_sock *tp, struct sk_buff *ack_skb, return dup_sack; } +/* Check if skb is fully within the SACK block. In presence of GSO skbs, + * the incoming SACK may not exactly match but we can find smaller MSS + * aligned portion of it that matches. Therefore we might need to fragment + * which may fail and creates some hassle (caller must handle error case + * returns). + */ +int tcp_match_skb_to_sack(struct sock *sk, struct sk_buff *skb, + u32 start_seq, u32 end_seq) +{ + int in_sack, err; + unsigned int pkt_len; + + in_sack = !after(start_seq, TCP_SKB_CB(skb)->seq) && + !before(end_seq, TCP_SKB_CB(skb)->end_seq); + + if (tcp_skb_pcount(skb) > 1 && !in_sack && + after(TCP_SKB_CB(skb)->end_seq, start_seq)) { + + in_sack = !after(start_seq, TCP_SKB_CB(skb)->seq); + + if (!in_sack) + pkt_len = start_seq - TCP_SKB_CB(skb)->seq; + else + pkt_len = end_seq - TCP_SKB_CB(skb)->seq; + err = tcp_fragment(sk, skb, pkt_len, skb_shinfo(skb)->gso_size); + if (err < 0) + return err; + } + + return in_sack; +} + static int tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_una) { @@ -1333,25 +1365,9 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ if (!before(TCP_SKB_CB(skb)->seq, end_seq)) break; - in_sack = !after(start_seq, TCP_SKB_CB(skb)->seq) && - !before(end_seq, TCP_SKB_CB(skb)->end_seq); - - if (tcp_skb_pcount(skb) > 1 && !in_sack && - after(TCP_SKB_CB(skb)->end_seq, start_seq)) { - unsigned int pkt_len; - - in_sack = !after(start_seq, - TCP_SKB_CB(skb)->seq); - - if (!in_sack) - pkt_len = (start_seq - - TCP_SKB_CB(skb)->seq); - else - pkt_len = (end_seq - - TCP_SKB_CB(skb)->seq); - if (tcp_fragment(sk, skb, pkt_len, skb_shinfo(skb)->gso_size)) - break; - } + in_sack = tcp_match_skb_to_sack(sk, skb, start_seq, end_seq); + if (in_sack < 0) + break; fack_count += tcp_skb_pcount(skb); -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO 2007-10-11 11:41 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code Ilpo Järvinen @ 2007-10-11 11:41 ` Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems Ilpo Järvinen 2007-10-12 0:35 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO David Miller 2007-10-12 0:34 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code David Miller 1 sibling, 2 replies; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw) To: David Miller; +Cc: netdev, TAKANO Ryousei Both sacked_out and fackets_out are directly known from how parameter. Since fackets_out is accurate, there's no need for recounting (sacked_out was previously unnecessarily counted in the loop anyway). Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 26 ++++++++++++++++---------- 1 files changed, 16 insertions(+), 10 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index bd18c25..a066039 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1711,18 +1711,23 @@ static void tcp_enter_frto_loss(struct sock *sk, int allowed_segments, int flag) tcp_clear_retrans_hints_partial(tp); } -void tcp_clear_retrans(struct tcp_sock *tp) +static void tcp_clear_retrans_partial(struct tcp_sock *tp) { tp->retrans_out = 0; - - tp->fackets_out = 0; - tp->sacked_out = 0; tp->lost_out = 0; tp->undo_marker = 0; tp->undo_retrans = 0; } +void tcp_clear_retrans(struct tcp_sock *tp) +{ + tcp_clear_retrans_partial(tp); + + tp->fackets_out = 0; + tp->sacked_out = 0; +} + /* Enter Loss state. If "how" is not zero, forget all SACK information * and reset tags completely, otherwise preserve SACKs. If receiver * dropped its ofo queue, we will know this due to reneging detection. @@ -1732,7 +1737,6 @@ void tcp_enter_loss(struct sock *sk, int how) const struct inet_connection_sock *icsk = inet_csk(sk); struct tcp_sock *tp = tcp_sk(sk); struct sk_buff *skb; - int cnt = 0; /* Reduce ssthresh if it has not yet been made inside this window. */ if (icsk->icsk_ca_state <= TCP_CA_Disorder || tp->snd_una == tp->high_seq || @@ -1746,7 +1750,10 @@ void tcp_enter_loss(struct sock *sk, int how) tp->snd_cwnd_stamp = tcp_time_stamp; tp->bytes_acked = 0; - tcp_clear_retrans(tp); + tcp_clear_retrans_partial(tp); + + if (tcp_is_reno(tp)) + tcp_reset_reno_sack(tp); if (!how) { /* Push undo marker, if it was plain RTO and nothing @@ -1754,13 +1761,15 @@ void tcp_enter_loss(struct sock *sk, int how) tp->undo_marker = tp->snd_una; tcp_clear_retrans_hints_partial(tp); } else { + tp->sacked_out = 0; + tp->fackets_out = 0; tcp_clear_all_retrans_hints(tp); } tcp_for_write_queue(skb, sk) { if (skb == tcp_send_head(sk)) break; - cnt += tcp_skb_pcount(skb); + if (TCP_SKB_CB(skb)->sacked&TCPCB_RETRANS) tp->undo_marker = 0; TCP_SKB_CB(skb)->sacked &= (~TCPCB_TAGBITS)|TCPCB_SACKED_ACKED; @@ -1768,9 +1777,6 @@ void tcp_enter_loss(struct sock *sk, int how) TCP_SKB_CB(skb)->sacked &= ~TCPCB_SACKED_ACKED; TCP_SKB_CB(skb)->sacked |= TCPCB_LOST; tp->lost_out += tcp_skb_pcount(skb); - } else { - tp->sacked_out += tcp_skb_pcount(skb); - tp->fackets_out = cnt; } } tcp_verify_left_out(tp); -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems 2007-10-11 11:41 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO Ilpo Järvinen @ 2007-10-11 11:41 ` Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases Ilpo Järvinen 2007-10-12 0:35 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems David Miller 2007-10-12 0:35 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO David Miller 1 sibling, 2 replies; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw) To: David Miller; +Cc: netdev, TAKANO Ryousei Detection implemented with lost_retrans must work also when fastpath is taken, yet most of the queue is skipped including (very likely) those retransmitted skb's we're interested in. This problem appeared when the hints got added, which removed a need to always walk over the whole write queue head. Therefore decicion for the lost_retrans worker loop entry must be separated from the sacktag processing more than it was necessary before. It turns out to be problematic to optimize the worker loop very heavily because ack_seqs of skb may have a number of discontinuity points. Maybe similar approach as currently is implemented could be attempted but that's becoming more and more complex because the trend is towards less skb walking in sacktag marker. Trying a simple work until all rexmitted skbs heve been processed approach. Maybe after(highest_sack_end_seq, tp->high_seq) checking is not sufficiently accurate and causes entry too often in no-work-to-do cases. Since that's not known, I've separated solution to that from this patch. Noticed because of report against a related problem from TAKANO Ryousei <takano@axe-inc.co.jp>. He also provided a patch to that part of the problem. This patch includes solution to it (though this patch has to use somewhat different placement). TAKANO's description and patch is available here: http://marc.info/?l=linux-netdev&m=119149311913288&w=2 ...In short, TAKANO's problem is that end_seq the loop is using not necessarily the largest SACK block's end_seq because the current ACK may still have higher SACK blocks which are later by the loop. Cc: TAKANO Ryousei <takano@axe-inc.co.jp> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 37 ++++++++++++++++++++++--------------- 1 files changed, 22 insertions(+), 15 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index a066039..d5e0fcc 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1109,27 +1109,34 @@ static int tcp_is_sackblock_valid(struct tcp_sock *tp, int is_dsack, /* Check for lost retransmit. This superb idea is borrowed from "ratehalving". * Event "C". Later note: FACK people cheated me again 8), we have to account * for reordering! Ugly, but should help. + * + * Search retransmitted skbs from write_queue that were sent when snd_nxt was + * less than what is now known to be received by the other end (derived from + * SACK blocks by the caller). */ -static int tcp_mark_lost_retrans(struct sock *sk, u32 lost_retrans) +static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) { struct tcp_sock *tp = tcp_sk(sk); struct sk_buff *skb; int flag = 0; + int cnt = 0; tcp_for_write_queue(skb, sk) { u32 ack_seq = TCP_SKB_CB(skb)->ack_seq; if (skb == tcp_send_head(sk)) break; - if (after(TCP_SKB_CB(skb)->seq, lost_retrans)) + if (cnt == tp->retrans_out) break; if (!after(TCP_SKB_CB(skb)->end_seq, tp->snd_una)) continue; - if ((TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_RETRANS) && - after(lost_retrans, ack_seq) && + if (!(TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_RETRANS)) + continue; + + if (after(received_upto, ack_seq) && (tcp_is_fack(tp) || - !before(lost_retrans, + !before(received_upto, ack_seq + tp->reordering * tp->mss_cache))) { TCP_SKB_CB(skb)->sacked &= ~TCPCB_SACKED_RETRANS; tp->retrans_out -= tcp_skb_pcount(skb); @@ -1143,6 +1150,8 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 lost_retrans) flag |= FLAG_DATA_SACKED; NET_INC_STATS_BH(LINUX_MIB_TCPLOSTRETRANSMIT); } + } else { + cnt += tcp_skb_pcount(skb); } } return flag; @@ -1225,7 +1234,7 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ int num_sacks = (ptr[1] - TCPOLEN_SACK_BASE)>>3; int reord = tp->packets_out; int prior_fackets; - u32 lost_retrans = 0; + u32 highest_sack_end_seq = 0; int flag = 0; int found_dup_sack = 0; int cached_fack_count; @@ -1396,11 +1405,6 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ continue; } - if ((sacked&TCPCB_SACKED_RETRANS) && - after(end_seq, TCP_SKB_CB(skb)->ack_seq) && - (!lost_retrans || after(end_seq, lost_retrans))) - lost_retrans = end_seq; - if (!in_sack) continue; @@ -1454,9 +1458,10 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ if (fack_count > tp->fackets_out) tp->fackets_out = fack_count; - if (after(TCP_SKB_CB(skb)->seq, - tp->highest_sack)) + if (after(TCP_SKB_CB(skb)->seq, tp->highest_sack)) { tp->highest_sack = TCP_SKB_CB(skb)->seq; + highest_sack_end_seq = TCP_SKB_CB(skb)->end_seq; + } } else { if (dup_sack && (sacked&TCPCB_RETRANS)) reord = min(fack_count, reord); @@ -1476,8 +1481,10 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ } } - if (lost_retrans && icsk->icsk_ca_state == TCP_CA_Recovery) - flag |= tcp_mark_lost_retrans(sk, lost_retrans); + if (tp->retrans_out && highest_sack_end_seq && + after(highest_sack_end_seq, tp->high_seq) && + icsk->icsk_ca_state == TCP_CA_Recovery) + flag |= tcp_mark_lost_retrans(sk, highest_sack_end_seq); tcp_verify_left_out(tp); -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases 2007-10-11 11:41 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems Ilpo Järvinen @ 2007-10-11 11:41 ` Ilpo Järvinen 2007-10-12 0:36 ` David Miller 2007-10-12 0:35 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems David Miller 1 sibling, 1 reply; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-11 11:41 UTC (permalink / raw) To: David Miller; +Cc: netdev, TAKANO Ryousei This addition of lost_retrans_low to tcp_sock might be unnecessary, it's not clear how often lost_retrans worker is executed when there wasn't work to do. Cc: TAKANO Ryousei <takano@axe-inc.co.jp> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- include/linux/tcp.h | 2 ++ net/ipv4/tcp_input.c | 14 +++++++++++--- net/ipv4/tcp_output.c | 2 ++ 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/include/linux/tcp.h b/include/linux/tcp.h index 9ff456e..c5b94c1 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -348,6 +348,8 @@ struct tcp_sock { int lost_cnt_hint; int retransmit_cnt_hint; + u32 lost_retrans_low; /* Sent seq after any rxmit (lowest) */ + u16 advmss; /* Advertised MSS */ u16 prior_ssthresh; /* ssthresh saved at recovery start */ u32 lost_out; /* Lost packets */ diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index d5e0fcc..0a42e93 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1112,7 +1112,8 @@ static int tcp_is_sackblock_valid(struct tcp_sock *tp, int is_dsack, * * Search retransmitted skbs from write_queue that were sent when snd_nxt was * less than what is now known to be received by the other end (derived from - * SACK blocks by the caller). + * SACK blocks by the caller). Also calculate the lowest snd_nxt among the + * remaining retransmitted skbs to avoid some costly processing per ACKs. */ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) { @@ -1120,6 +1121,7 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) struct sk_buff *skb; int flag = 0; int cnt = 0; + u32 new_low_seq = 0; tcp_for_write_queue(skb, sk) { u32 ack_seq = TCP_SKB_CB(skb)->ack_seq; @@ -1151,9 +1153,15 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) NET_INC_STATS_BH(LINUX_MIB_TCPLOSTRETRANSMIT); } } else { + if (!new_low_seq || before(ack_seq, new_low_seq)) + new_low_seq = ack_seq; cnt += tcp_skb_pcount(skb); } } + + if (tp->retrans_out) + tp->lost_retrans_low = new_low_seq; + return flag; } @@ -1481,8 +1489,8 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ } } - if (tp->retrans_out && highest_sack_end_seq && - after(highest_sack_end_seq, tp->high_seq) && + if (tp->retrans_out && + after(highest_sack_end_seq, tp->lost_retrans_low) && icsk->icsk_ca_state == TCP_CA_Recovery) flag |= tcp_mark_lost_retrans(sk, highest_sack_end_seq); diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 5329675..324b420 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -1914,6 +1914,8 @@ int tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb) printk(KERN_DEBUG "retrans_out leaked.\n"); } #endif + if (!tp->retrans_out) + tp->lost_retrans_low = tp->snd_nxt; TCP_SKB_CB(skb)->sacked |= TCPCB_RETRANS; tp->retrans_out += tcp_skb_pcount(skb); -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases 2007-10-11 11:41 ` [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases Ilpo Järvinen @ 2007-10-12 0:36 ` David Miller 2007-10-18 3:50 ` TAKANO Ryousei 0 siblings, 1 reply; 22+ messages in thread From: David Miller @ 2007-10-12 0:36 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:07 +0300 > This addition of lost_retrans_low to tcp_sock might be > unnecessary, it's not clear how often lost_retrans worker is > executed when there wasn't work to do. > > Cc: TAKANO Ryousei <takano@axe-inc.co.jp> > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> Applied. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases 2007-10-12 0:36 ` David Miller @ 2007-10-18 3:50 ` TAKANO Ryousei 2007-10-18 9:40 ` [PATCH] [TCP]: Add highest_sack_end_seq check back to lost_retrans call Ilpo Järvinen 0 siblings, 1 reply; 22+ messages in thread From: TAKANO Ryousei @ 2007-10-18 3:50 UTC (permalink / raw) To: davem; +Cc: ilpo.jarvinen, y-kodama, netdev From: David Miller <davem@davemloft.net> Subject: Re: [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases Date: Thu, 11 Oct 2007 17:36:22 -0700 (PDT) > From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> > Date: Thu, 11 Oct 2007 14:41:07 +0300 > > > This addition of lost_retrans_low to tcp_sock might be > > unnecessary, it's not clear how often lost_retrans worker is > > executed when there wasn't work to do. > > > > Cc: TAKANO Ryousei <takano@axe-inc.co.jp> > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > > Applied. > + after(highest_sack_end_seq, tp->lost_retrans_low) && This limit degrades the performance of my test case described before, since it misses opportunities of detecting loss of retransmitted packets. The results are shown as follows: linux-2.6.23 iteration :001 .... done. T = 19.758205 BW = 404.895077 Mbps iteration :002 .... done. T = 19.241735 BW = 415.762924 Mbps iteration :003 .... done. T = 18.998427 BW = 421.087495 Mbps iteration :004 .... done. T = 18.971502 BW = 421.685113 Mbps iteration :005 .... done. T = 19.298313 BW = 414.544009 Mbps net-2.6 (current): iteration :001 .... done. T = 43.672971 BW = 183.179660 Mbps iteration :002 .... done. T = 16.668652 BW = 479.942828 Mbps iteration :003 .... done. T = 25.569106 BW = 312.877581 Mbps iteration :004 .... done. T = 55.314832 BW = 144.626671 Mbps iteration :005 .... done. T = 35.377025 BW = 226.135464 Mbps net-2.6 (no limit) iteration :001 .... done. T = 16.907727 BW = 473.156445 Mbps iteration :002 .... done. T = 16.662908 BW = 480.108272 Mbps iteration :003 .... done. T = 16.536396 BW = 483.781350 Mbps iteration :004 .... done. T = 16.565220 BW = 482.939560 Mbps iteration :005 .... done. T = 16.528676 BW = 484.007309 Mbps "no limit" does as follows: if (tp->retrans_out && highest_sack_end_seq && icsk->icsk_ca_state == TCP_CA_Recovery) flag |= tcp_mark_lost_retrans(sk, highest_sack_end_seq); Ryousei Takano ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH] [TCP]: Add highest_sack_end_seq check back to lost_retrans call 2007-10-18 3:50 ` TAKANO Ryousei @ 2007-10-18 9:40 ` Ilpo Järvinen 2007-10-18 10:17 ` [PATCH] [TCP]: Remove lost_retrans zero special cases Ilpo Järvinen 0 siblings, 1 reply; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-18 9:40 UTC (permalink / raw) To: TAKANO Ryousei; +Cc: David Miller, y-kodama, Netdev [-- Attachment #1: Type: TEXT/PLAIN, Size: 3809 bytes --] On Thu, 18 Oct 2007, TAKANO Ryousei wrote: > From: David Miller <davem@davemloft.net> > Subject: Re: [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases > Date: Thu, 11 Oct 2007 17:36:22 -0700 (PDT) > > > From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> > > Date: Thu, 11 Oct 2007 14:41:07 +0300 > > > > > This addition of lost_retrans_low to tcp_sock might be > > > unnecessary, it's not clear how often lost_retrans worker is > > > executed when there wasn't work to do. > > > > > > Cc: TAKANO Ryousei <takano@axe-inc.co.jp> > > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > > > > Applied. > > > + after(highest_sack_end_seq, tp->lost_retrans_low) && > > This limit degrades the performance of my test case described before, Thanks for testing.... Btw, just noticed that lost_retrans_low addition patch incorrectly dropped check for highest_sack_end_seq (probably due to incorrect resolution from my side at some point of development of those two patches as I added that check later on when realized it's necessary), patching that below... Since it causes zero received_upto in tcp_mark_lost_retrans, some RETRANS bits got cleared unintentionally because of that. > since it misses opportunities of detecting loss of retransmitted packets. Do you have an idea how it does that except the problem now being fixed? The lost_retrans_low was supposed to contain the minimum TCP_SKB_CB(skb)->ack_seq of those packets that hav TCPCB_SACKED_RETRANS set, so if the condition won't match, there shouldn't be any misses. Do you think there's something wrong in that approach? The point of lost_retrans_low is to limit walking in retrans queue to two cases: - There's at least one skb with SACKED_RETRANS to clear (except perhaps some corner cases where that specific skb got DSACKed in between and the R-bit was therefore cleared). or - New lost_retrans_low has to calculated (should only occur if snd_una advanced past it and there are still some retransmissions, not too sure if that can occur, and that won't be very likely case anyway). > "no limit" does as follows: > if (tp->retrans_out && highest_sack_end_seq && > icsk->icsk_ca_state == TCP_CA_Recovery) > flag |= tcp_mark_lost_retrans(sk, highest_sack_end_seq); Having that highest_sack_end_seq there probably solved the problem. Yet it won't be good in performance wise like that because many unnecessary walks would occur during CA_Recovery, which all are prone to induce some cache misses. -- [PATCH] [TCP]: Add highest_sack_end_seq check back to lost_retrans call This was unintentionally dropped (probably due to incorrect resolution from my side at some point of development of those two patches as I added that check later on when realized it's necessary). There won't be anything to mark if SACKs didn't advance. It causes passing of zero received_upto to tcp_mark_lost_retrans which confuses after relations within the marker loop causing incorrect TCPCB_SACKED_RETRANS clearing. This problem was noticed because of a performance report from TAKANO Ryousei <takano@axe-inc.co.jp>. Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 0f00966..c3c0183 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1489,7 +1489,7 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ } } - if (tp->retrans_out && + if (tp->retrans_out && highest_sack_end_seq && after(highest_sack_end_seq, tp->lost_retrans_low) && icsk->icsk_ca_state == TCP_CA_Recovery) flag |= tcp_mark_lost_retrans(sk, highest_sack_end_seq); -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH] [TCP]: Remove lost_retrans zero special cases 2007-10-18 9:40 ` [PATCH] [TCP]: Add highest_sack_end_seq check back to lost_retrans call Ilpo Järvinen @ 2007-10-18 10:17 ` Ilpo Järvinen 2007-10-18 10:55 ` TAKANO Ryousei 0 siblings, 1 reply; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-18 10:17 UTC (permalink / raw) To: TAKANO Ryousei, David Miller; +Cc: y-kodama, Netdev [-- Attachment #1: Type: TEXT/PLAIN, Size: 3275 bytes --] On Thu, 18 Oct 2007, Ilpo Järvinen wrote: > On Thu, 18 Oct 2007, TAKANO Ryousei wrote: > > > From: David Miller <davem@davemloft.net> > > Subject: Re: [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases > > Date: Thu, 11 Oct 2007 17:36:22 -0700 (PDT) > > > > > From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> > > > Date: Thu, 11 Oct 2007 14:41:07 +0300 > > > > > > > This addition of lost_retrans_low to tcp_sock might be > > > > unnecessary, it's not clear how often lost_retrans worker is > > > > executed when there wasn't work to do. > > > > > > > > Cc: TAKANO Ryousei <takano@axe-inc.co.jp> > > > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > > > > > > Applied. > > > > > + after(highest_sack_end_seq, tp->lost_retrans_low) && > > > > This limit degrades the performance of my test case described before, > > Thanks for testing.... Btw, just noticed that lost_retrans_low addition > patch incorrectly dropped check for highest_sack_end_seq (probably due to > incorrect resolution from my side at some point of development of those > two patches as I added that check later on when realized it's necessary), > patching that below... Since it causes zero received_upto in > tcp_mark_lost_retrans, some RETRANS bits got cleared unintentionally > because of that. ...snip... > -- > > [PATCH] [TCP]: Add highest_sack_end_seq check back to lost_retrans call Try this patch instead not on the top of the one I sent earlier (IMHO this is better approach): -- [PATCH] [TCP]: Remove lost_retrans zero seqno special cases Both high-sack detection and new lowest seq variables have unnecessary zero special case which are now removed by setting safe initial seqnos. This also fixes problem which caused zero received_upto being passed to tcp_mark_lost_retrans which confused after relations within the marker loop causing incorrect TCPCB_SACKED_RETRANS clearing. The problem was noticed because of a performance report from TAKANO Ryousei <takano@axe-inc.co.jp>. Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 0f00966..9288220 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1121,7 +1121,7 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) struct sk_buff *skb; int flag = 0; int cnt = 0; - u32 new_low_seq = 0; + u32 new_low_seq = tp->snd_nxt; tcp_for_write_queue(skb, sk) { u32 ack_seq = TCP_SKB_CB(skb)->ack_seq; @@ -1153,7 +1153,7 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) NET_INC_STATS_BH(LINUX_MIB_TCPLOSTRETRANSMIT); } } else { - if (!new_low_seq || before(ack_seq, new_low_seq)) + if (before(ack_seq, new_low_seq)) new_low_seq = ack_seq; cnt += tcp_skb_pcount(skb); } @@ -1242,7 +1242,7 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ int num_sacks = (ptr[1] - TCPOLEN_SACK_BASE)>>3; int reord = tp->packets_out; int prior_fackets; - u32 highest_sack_end_seq = 0; + u32 highest_sack_end_seq = tp->lost_retrans_low; int flag = 0; int found_dup_sack = 0; int cached_fack_count; -- 1.5.0.6 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH] [TCP]: Remove lost_retrans zero special cases 2007-10-18 10:17 ` [PATCH] [TCP]: Remove lost_retrans zero special cases Ilpo Järvinen @ 2007-10-18 10:55 ` TAKANO Ryousei 2007-10-18 11:07 ` Ilpo Järvinen 0 siblings, 1 reply; 22+ messages in thread From: TAKANO Ryousei @ 2007-10-18 10:55 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: davem, y-kodama, netdev From: "Ilpo Järvinen" <ilpo.jarvinen@helsinki.fi> Subject: [PATCH] [TCP]: Remove lost_retrans zero special cases Date: Thu, 18 Oct 2007 13:17:24 +0300 (EEST) > [PATCH] [TCP]: Remove lost_retrans zero seqno special cases > > Both high-sack detection and new lowest seq variables have > unnecessary zero special case which are now removed by setting > safe initial seqnos. > > This also fixes problem which caused zero received_upto being > passed to tcp_mark_lost_retrans which confused after relations > within the marker loop causing incorrect TCPCB_SACKED_RETRANS > clearing. The problem was noticed because of a performance > report from TAKANO Ryousei <takano@axe-inc.co.jp>. > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > --- > net/ipv4/tcp_input.c | 6 +++--- > 1 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c > index 0f00966..9288220 100644 > --- a/net/ipv4/tcp_input.c > +++ b/net/ipv4/tcp_input.c > @@ -1121,7 +1121,7 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) > struct sk_buff *skb; > int flag = 0; > int cnt = 0; > - u32 new_low_seq = 0; > + u32 new_low_seq = tp->snd_nxt; > > tcp_for_write_queue(skb, sk) { > u32 ack_seq = TCP_SKB_CB(skb)->ack_seq; > @@ -1153,7 +1153,7 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) > NET_INC_STATS_BH(LINUX_MIB_TCPLOSTRETRANSMIT); > } > } else { > - if (!new_low_seq || before(ack_seq, new_low_seq)) > + if (before(ack_seq, new_low_seq)) > new_low_seq = ack_seq; > cnt += tcp_skb_pcount(skb); > } > @@ -1242,7 +1242,7 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ > int num_sacks = (ptr[1] - TCPOLEN_SACK_BASE)>>3; > int reord = tp->packets_out; > int prior_fackets; > - u32 highest_sack_end_seq = 0; > + u32 highest_sack_end_seq = tp->lost_retrans_low; > int flag = 0; > int found_dup_sack = 0; > int cached_fack_count; > -- > 1.5.0.6 Thanks Ilpo! This patch solves the problem, as shown below: iteration :001 .... done. T = 17.365902 BW = 460.672876 Mbps iteration :002 .... done. T = 16.456351 BW = 486.134501 Mbps iteration :003 .... done. T = 16.658353 BW = 480.239550 Mbps iteration :004 .... done. T = 16.468834 BW = 485.766026 Mbps iteration :005 .... done. T = 16.596055 BW = 482.042268 Mbps Acked-by: Ryousei Takano <takano-ryousei@aist.go.jp> ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] [TCP]: Remove lost_retrans zero special cases 2007-10-18 10:55 ` TAKANO Ryousei @ 2007-10-18 11:07 ` Ilpo Järvinen 2007-10-18 12:08 ` David Miller 0 siblings, 1 reply; 22+ messages in thread From: Ilpo Järvinen @ 2007-10-18 11:07 UTC (permalink / raw) To: TAKANO Ryousei, David Miller; +Cc: y-kodama, Netdev [-- Attachment #1: Type: TEXT/PLAIN, Size: 3084 bytes --] On Thu, 18 Oct 2007, TAKANO Ryousei wrote: > From: "Ilpo Järvinen" <ilpo.jarvinen@helsinki.fi> > Subject: [PATCH] [TCP]: Remove lost_retrans zero special cases > Date: Thu, 18 Oct 2007 13:17:24 +0300 (EEST) > > > [PATCH] [TCP]: Remove lost_retrans zero seqno special cases > > > > Both high-sack detection and new lowest seq variables have > > unnecessary zero special case which are now removed by setting > > safe initial seqnos. > > > > This also fixes problem which caused zero received_upto being > > passed to tcp_mark_lost_retrans which confused after relations > > within the marker loop causing incorrect TCPCB_SACKED_RETRANS > > clearing. The problem was noticed because of a performance > > report from TAKANO Ryousei <takano@axe-inc.co.jp>. > > > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > > --- > > net/ipv4/tcp_input.c | 6 +++--- > > 1 files changed, 3 insertions(+), 3 deletions(-) > > > > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c > > index 0f00966..9288220 100644 > > --- a/net/ipv4/tcp_input.c > > +++ b/net/ipv4/tcp_input.c > > @@ -1121,7 +1121,7 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) > > struct sk_buff *skb; > > int flag = 0; > > int cnt = 0; > > - u32 new_low_seq = 0; > > + u32 new_low_seq = tp->snd_nxt; > > > > tcp_for_write_queue(skb, sk) { > > u32 ack_seq = TCP_SKB_CB(skb)->ack_seq; > > @@ -1153,7 +1153,7 @@ static int tcp_mark_lost_retrans(struct sock *sk, u32 received_upto) > > NET_INC_STATS_BH(LINUX_MIB_TCPLOSTRETRANSMIT); > > } > > } else { > > - if (!new_low_seq || before(ack_seq, new_low_seq)) > > + if (before(ack_seq, new_low_seq)) > > new_low_seq = ack_seq; > > cnt += tcp_skb_pcount(skb); > > } > > @@ -1242,7 +1242,7 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, u32 prior_snd_ > > int num_sacks = (ptr[1] - TCPOLEN_SACK_BASE)>>3; > > int reord = tp->packets_out; > > int prior_fackets; > > - u32 highest_sack_end_seq = 0; > > + u32 highest_sack_end_seq = tp->lost_retrans_low; > > int flag = 0; > > int found_dup_sack = 0; > > int cached_fack_count; > > -- > > 1.5.0.6 > > Thanks Ilpo! This patch solves the problem, as shown below: > > iteration :001 .... done. T = 17.365902 BW = 460.672876 Mbps > iteration :002 .... done. T = 16.456351 BW = 486.134501 Mbps > iteration :003 .... done. T = 16.658353 BW = 480.239550 Mbps > iteration :004 .... done. T = 16.468834 BW = 485.766026 Mbps > iteration :005 .... done. T = 16.596055 BW = 482.042268 Mbps > > Acked-by: Ryousei Takano <takano-ryousei@aist.go.jp> Thanks for quick testing, it was actually quite bad bug. I first thought that it's just performance issue and therefore suspected that the problem is elsewhere until I realized that the zero can be used as seqno. Dave, please put this one to net-2.6 and forget the other patch with title "[TCP]: Add highest_sack_end_seq check back...", it would just add another (wrong) zero special case back (they won't conflict with each other, so being this verbose here) :-). -- i. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH] [TCP]: Remove lost_retrans zero special cases 2007-10-18 11:07 ` Ilpo Järvinen @ 2007-10-18 12:08 ` David Miller 0 siblings, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-18 12:08 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: takano, y-kodama, netdev From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 18 Oct 2007 14:07:42 +0300 (EEST) > Dave, please put this one to net-2.6 and forget the other patch with > title "[TCP]: Add highest_sack_end_seq check back...", it would just > add another (wrong) zero special case back (they won't conflict with > each other, so being this verbose here) :-). Done, thanks everyone. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems 2007-10-11 11:41 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases Ilpo Järvinen @ 2007-10-12 0:35 ` David Miller 1 sibling, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-12 0:35 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:06 +0300 > Detection implemented with lost_retrans must work also when > fastpath is taken, yet most of the queue is skipped including > (very likely) those retransmitted skb's we're interested in. > This problem appeared when the hints got added, which removed > a need to always walk over the whole write queue head. > Therefore decicion for the lost_retrans worker loop entry must > be separated from the sacktag processing more than it was > necessary before. > > It turns out to be problematic to optimize the worker loop > very heavily because ack_seqs of skb may have a number of > discontinuity points. Maybe similar approach as currently is > implemented could be attempted but that's becoming more and > more complex because the trend is towards less skb walking > in sacktag marker. Trying a simple work until all rexmitted > skbs heve been processed approach. > > Maybe after(highest_sack_end_seq, tp->high_seq) checking is not > sufficiently accurate and causes entry too often in no-work-to-do > cases. Since that's not known, I've separated solution to that > from this patch. > > Noticed because of report against a related problem from TAKANO > Ryousei <takano@axe-inc.co.jp>. He also provided a patch to > that part of the problem. This patch includes solution to it > (though this patch has to use somewhat different placement). > TAKANO's description and patch is available here: > > http://marc.info/?l=linux-netdev&m=119149311913288&w=2 > > ...In short, TAKANO's problem is that end_seq the loop is using > not necessarily the largest SACK block's end_seq because the > current ACK may still have higher SACK blocks which are later > by the loop. > > Cc: TAKANO Ryousei <takano@axe-inc.co.jp> > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> Applied. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO 2007-10-11 11:41 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems Ilpo Järvinen @ 2007-10-12 0:35 ` David Miller 1 sibling, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-12 0:35 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:05 +0300 > Both sacked_out and fackets_out are directly known from how > parameter. Since fackets_out is accurate, there's no need for > recounting (sacked_out was previously unnecessarily counted > in the loop anyway). > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> Applied. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code 2007-10-11 11:41 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO Ilpo Järvinen @ 2007-10-12 0:34 ` David Miller 1 sibling, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-12 0:34 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:04 +0300 > This is necessary for upcoming DSACK bugfix. Reduces sacktag > length which is not very sad thing at all... :-) > > Notice that there's a need to handle out-of-mem at caller's > place. > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> Applied. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag 2007-10-11 11:41 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code Ilpo Järvinen @ 2007-10-12 0:34 ` David Miller 1 sibling, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-12 0:34 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:03 +0300 > It's on the way for future cutting of that function. > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> Applied. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L 2007-10-11 11:41 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag Ilpo Järvinen @ 2007-10-12 0:33 ` David Miller 1 sibling, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-12 0:33 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:02 +0300 > This condition (plain R) can arise at least in recovery that > is triggered after tcp_undo_loss. There isn't any reason why > they should not be marked as lost, not marking makes in_flight > estimator to return too large values. > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> Nice observation, applied, thanks Ilpo. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too 2007-10-11 11:41 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L Ilpo Järvinen @ 2007-10-12 0:32 ` David Miller 1 sibling, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-12 0:32 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:01 +0300 > I was reading tcp_enter_loss while looking for Cedric's bug and > noticed bytes_acked adjustment is missing from FRTO side. > > Since bytes_acked will only be used in tcp_cong_avoid, I think > it's safe to assume RTO would be spurious. During FRTO cwnd > will be not controlled by tcp_cong_avoid and if FRTO calls for > conventional recovery, cwnd is adjusted and the result of wrong > assumption is cleared from bytes_acked. If RTO was in fact > spurious, we did normal ABC already and can continue without > any additional adjustments. > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> Applied. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix 2007-10-11 11:41 [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too Ilpo Järvinen @ 2007-10-12 0:37 ` David Miller 1 sibling, 0 replies; 22+ messages in thread From: David Miller @ 2007-10-12 0:37 UTC (permalink / raw) To: ilpo.jarvinen; +Cc: netdev, takano From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi> Date: Thu, 11 Oct 2007 14:41:00 +0300 > Dave, please apply at will. Done. ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2007-10-18 12:07 UTC | newest] Thread overview: 22+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2007-10-11 11:41 [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems Ilpo Järvinen 2007-10-11 11:41 ` [PATCH 7/7] [TCP]: Limit processing lost_retrans loop to work-to-do cases Ilpo Järvinen 2007-10-12 0:36 ` David Miller 2007-10-18 3:50 ` TAKANO Ryousei 2007-10-18 9:40 ` [PATCH] [TCP]: Add highest_sack_end_seq check back to lost_retrans call Ilpo Järvinen 2007-10-18 10:17 ` [PATCH] [TCP]: Remove lost_retrans zero special cases Ilpo Järvinen 2007-10-18 10:55 ` TAKANO Ryousei 2007-10-18 11:07 ` Ilpo Järvinen 2007-10-18 12:08 ` David Miller 2007-10-12 0:35 ` [PATCH 6/7] [TCP]: Fix lost_retrans loop vs fastpath problems David Miller 2007-10-12 0:35 ` [PATCH 5/7] [TCP]: No need to re-count fackets_out/sacked_out at RTO David Miller 2007-10-12 0:34 ` [PATCH 4/7] [TCP]: Extract tcp_match_queue_to_sack from sacktag code David Miller 2007-10-12 0:34 ` [PATCH 3/7] [TCP]: Kill almost unused variable pcount from sacktag David Miller 2007-10-12 0:33 ` [PATCH 2/7] [TCP]: Fix mark_head_lost to ignore R-bit when trying to mark L David Miller 2007-10-12 0:32 ` [PATCH 1/7] [TCP]: Add bytes_acked (ABC) clearing to FRTO too David Miller 2007-10-12 0:37 ` [PATCH net-2.6 0/7] TCP: small changes/fixes + lost_retrans brokeness fix David Miller
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox