[PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[Gerrit Renker]

This is the patch which stops the nofeedback timer from spinning
every couple of microseconds, triggered by small RTTs. 

However, there are a few more bugs in CCID 3; this is work in progress.

-----------------> Commit Message <-----------------------------------
[DCCP]: Use higher RTO default for CCID3 

The TFRC nofeedback timer normally expires after the maximum of 4
RTTs and twice the current send interval (RFC 3448, 4.3). On LANs
with a small RTT this can mean a high processing load and reduced
performance, since then the nofeedback timer is triggered very
frequently. As a result, the sending rate quickly converges towards
zero.

This patch provides a configuration option to set the bound for the
nofeedback timer, using as default the TCP RTO timeout of 1 second.

By setting the configuration option to 0, RFC 3448 behaviour can
be enforced for the nofeedback timer.

Has been tested to compile and work.

Signed-off-by: Gerrit Renker <gerrit@erg.abdn.ac.uk>
---
 net/dccp/ccids/Kconfig |   22 ++++++++++++++++++++++
 net/dccp/ccids/ccid3.c |   20 +++++++++++---------
 2 files changed, 33 insertions(+), 9 deletions(-)

--- a/net/dccp/ccids/Kconfig
+++ b/net/dccp/ccids/Kconfig
@@ -89,4 +89,26 @@ config IP_DCCP_CCID3_DEBUG
 	    parameter to 0 or 1.
 
 	    If in doubt, say N.
+
+config IP_DCCP_CCID3_RTO
+	  int "Use higher (TCP) RTO for nofeedback timer"
+	  default 1
+	  depends on IP_DCCP_CCID3
+	  ---help---
+	    Use higher (TCP) RTO value for nofeedback expiration.
+
+	    The TFRC nofeedback timer normally expires after the maximum of 4
+	    RTTs and twice the current send interval (RFC 3448, 4.3). On LANs
+	    with a small RTT this can mean a high processing load and reduced
+	    performance, since then the nofeedback timer is triggered very
+	    frequently. As a result, the sending rate quickly converges towards
+	    zero.
+
+	    This option enables to set a higher upper bound for the nofeedback
+	    value. Values in units of seconds can be set here.
+
+	    A value of 0 disables this feature by enforcing the value specified
+	    in RFC 3448. It is recommended to use the default value of 1 second,
+	    as this is also the suggested TCP RTO default (RFC 2988, 2.4).
+
 endmenu
--- a/net/dccp/ccids/ccid3.c
+++ b/net/dccp/ccids/ccid3.c
@@ -245,9 +245,10 @@ static void ccid3_hc_tx_no_feedback_time
 		}
 		/*
 		 * Schedule no feedback timer to expire in
-		 * max(4 * R, 2 * s/X)  =  max(4 * R, 2 * t_ipi)
+		 * max(t_RTO, 2 * s/X)  =  max(t_RTO, 2 * t_ipi)
+		 * See comments above regarding the value of t_RTO.
 		 */
-		t_nfb = max(4 * hctx->ccid3hctx_rtt, 2 * hctx->ccid3hctx_t_ipi);
+		t_nfb = max(hctx->ccid3hctx_t_rto, 2 * hctx->ccid3hctx_t_ipi);
 		break;
 	case TFRC_SSTATE_NO_SENT:
 		DCCP_BUG("Illegal %s state NO_SENT, sk=%p", dccp_role(sk), sk);
@@ -481,7 +482,7 @@ static void ccid3_hc_tx_packet_recv(stru
 		}
 
 		ccid3_pr_debug("%s, sk=%p, New RTT estimate=%uus, "
-			       "r_sample=%us\n", dccp_role(sk), sk,
+			       "r_sample=%uus\n", dccp_role(sk), sk,
 			       hctx->ccid3hctx_rtt, r_sample);
 
 		/* Update receive rate */
@@ -512,16 +513,17 @@ static void ccid3_hc_tx_packet_recv(stru
 		 */
 		sk->sk_write_space(sk);
 
-		/* Update timeout interval. We use the alternative variant of
-		 * [RFC 3448, 3.1] which sets the upper bound of t_rto to one
-		 * second, as it is suggested for TCP (see RFC 2988, 2.4). */
+		/* Update timeout interval. A value of IP_DCCP_CCID3_RTO=0
+		 * lets the nofeedback timer expire as per [RFC 3448, 3.1];
+		 * a value of 1 second corresponds to the RTO suggested for
+		 * TCP (see RFC 2988, 2.4). */
 		hctx->ccid3hctx_t_rto = max_t(u32, 4 * hctx->ccid3hctx_rtt,
-					      	   USEC_PER_SEC            );
+					CONFIG_IP_DCCP_CCID3_RTO * USEC_PER_SEC);
 		/*
 		 * Schedule no feedback timer to expire in
-		 * max(4 * R, 2 * s/X)  =  max(4 * R, 2 * t_ipi)
+		 * max(t_RTO, 2 * s/X)  =  max(t_RTO, 2 * t_ipi)
 		 */
-		t_nfb = max(4 * hctx->ccid3hctx_rtt, 2 * hctx->ccid3hctx_t_ipi);
+		t_nfb = max(hctx->ccid3hctx_t_rto, 2 * hctx->ccid3hctx_t_ipi);
 			
 		ccid3_pr_debug("%s, sk=%p, Scheduled no feedback timer to "
 			       "expire in %lu jiffies (%luus)\n",

Re: [PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[Lars Eggert]

[-- Attachment #1: Type: text/plain, Size: 290 bytes --]

On Nov 30, 2006, at 11:13, Ian McDonald wrote:
> This patch provides a configuration option to set the bound for the
> nofeedback timer, using as default the TCP RTO timeout of 1 second.

The default TCP RTO is 3 seconds (RFC 2988). Linux is apparently not  
conformant to that RFC?

Lars


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3668 bytes --]

Re: [PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[Gerrit Renker]

Quoting Lars Eggert:
|  On Nov 30, 2006, at 11:13, Ian McDonald wrote:
|  > This patch provides a configuration option to set the bound for the
|  > nofeedback timer, using as default the TCP RTO timeout of 1 second.
|  
|  The default TCP RTO is 3 seconds (RFC 2988). Linux is apparently not  
|  conformant to that RFC?
Ian didn't copy the remainder of the mail, which had the clarifying explanation
that this setting is related to section 2.4 of RFC 2988:
  (2.4) "Whenever RTO is computed, if it is less than 1 second then the
         RTO SHOULD be rounded up to 1 second."

Also [RFC 3448, sec. 1]: 
  "Another possibility would be to set t_RTO = max(4R, one second), 
   to match the recommended minimum of one second on the RTO [5]." 
   where [5] = RFC 2988.

Currently, the nofeedback timer is set to 

  t_nfb = max(4 * R, 2*s/X)

When R is very small (e.g. 250 usec), 4 * R is close to the dimension of one or
two clock ticks.

Re: [PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[gorry]

Let me respond simply on the issue of 3 seconds.

I know the value of 3 seconds has not respected in all TCP
Implementations, but this is never the less the value that was put by the
IETF - for good reasons. Smaller values of RTO can and do cause
interoperability problems over some Internet Paths.

This issue is not new, many many years ago one major manufacturer chose a
smaller initial RTO and caused major upset among people using satellite
links... fortunately the implementation was soon patched, but not before
it entered deployment on some web servers, where the problem persisted.

I suggest a value of 3 Secs is sufficient for the General internet...

Gorry

> On Nov 30, 2006, at 11:13, Ian McDonald wrote:
>> This patch provides a configuration option to set the bound for the
>> nofeedback timer, using as default the TCP RTO timeout of 1 second.
>
> The default TCP RTO is 3 seconds (RFC 2988). Linux is apparently not
> conformant to that RFC?
>
> Lars
>
>

Re: [PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[Gerrit Renker]

Quoting Mark Handley:
|  I agree that running a very small no-feedback timer is a bad idea.
|  But I think that 1 second is probably far too large.  The purpose of
|  the nofeedback timer is to slow DCCP down when there is serious
|  network congestion.  Waiting 1 second on a LAN would mean sending for
|  thousands of RTTs before starting to slow down.  And on wide-area
|  links in places like the UK, it could be 100 RTTs before you slow
|  down, although this would be mitigated a little if the problem was
|  congestion, and a queue built up.
|  
|  My gut feeling is that there should be a lower bound on the nofeedback
|  timer, but that 100ms would be a more appropriate value.  This is
|  motivated by an attempt to compromise between a large value for
|  efficient DCCP implementations, and a small value to avoid disrupting
|  the network for too long when bad stuff is happening.  From a human
|  usability point of view, you probably can cope with dropouts in audio
|  of 100ms without it being too bad, but 1 second is too long.
Thanks a lot for this feedback. I will change the patch once again so that the
configuration option scales in units of 100 milliseconds - that will give users
a chance to test at different granularity:
	* 0 means use RFC 3448 as before
	* 1 means 100 milliseconds
	* 10 corresponds to the TCP timeout of 1 sec
	* ...

Gerrit

Re: [PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[Gerrit Renker]

Quoting Mark Handley:
|  On 12/1/06, Colin Perkins <csp@csperkins.org> wrote:
|  > I'd actually suggest something on the order of 16-20ms. The rationale
|  > would be to match the typical inter-frame interval for multimedia
|  > applications, so that the kernel will likely be processing a sent
|  > packet when the timer expires, and can amortise the costs of checking
|  > the nofeedback timer into the send routine.
|  
|  You make a good point.  The cost of checking the timer should be
|  fairly low if you're already in the DCCP send code for the relevant
|  connection.  You have to reset the timer anyway every time you get
|  feedback, and checking the timer shouldn't be more expensive than
|  resetting it. As you get feedback every RTT under normal conditions,
|  you ought to be able to check it at least once per RTT so long as
|  you're in the DCCP code anyway.
|  
|  Now, do you every need to check the timer when you're not in the DCCP
|  send code?  I think not.  It's not like TCP, where you need to send a
|  packet on RTO expiry.  I think it should be possible to only ever
|  check the nofeedback timer when you're in the send code for that
|  particular DCCP connection.  And if so, then surely you can afford to
|  check it as often as you reset it - ie around once per RTT?
|  
|  You just have to be a bit smart - if you go to send and discover that
|  the timer should have expired multiple times, then you have to process
|  multiple backoff events, and perhaps reschedule your send for later
|  after you've done that.
Once again thanks to both answers - both value range will be made configurable
and a default value of 100ms will be provided. With a default of 10ms, one is
close to the timer granularity of 100HZ on some systems. 
I will also add the helpful information provided to the configuration menu.

Gerrit

Re: [PATCH] [DCCP]: Use higher timeout value for nofeedback timer

[Eddie Kohler]

I agree with Mark here.  The cleanest implementation of the nofeedback 
timer would probably not involve actual Linux timers at all!  Unless I'm 
very confused, the nofeedback timer is only needed to change sending 
rates, and thus can safely be checked in the process of sending a packet.

Not suggesting an immediate change to the implementation, I don't have a 
patch ready, etc., etc., just documenting this. :)

Eddie


Mark Handley wrote:
> On 12/1/06, Colin Perkins <csp@csperkins.org> wrote:
>> I'd actually suggest something on the order of 16-20ms. The rationale
>> would be to match the typical inter-frame interval for multimedia
>> applications, so that the kernel will likely be processing a sent
>> packet when the timer expires, and can amortise the costs of checking
>> the nofeedback timer into the send routine.
> 
> You make a good point.  The cost of checking the timer should be
> fairly low if you're already in the DCCP send code for the relevant
> connection.  You have to reset the timer anyway every time you get
> feedback, and checking the timer shouldn't be more expensive than
> resetting it. As you get feedback every RTT under normal conditions,
> you ought to be able to check it at least once per RTT so long as
> you're in the DCCP code anyway.
> 
> Now, do you every need to check the timer when you're not in the DCCP
> send code?  I think not.  It's not like TCP, where you need to send a
> packet on RTO expiry.  I think it should be possible to only ever
> check the nofeedback timer when you're in the send code for that
> particular DCCP connection.  And if so, then surely you can afford to
> check it as often as you reset it - ie around once per RTT?
> 
> You just have to be a bit smart - if you go to send and discover that
> the timer should have expired multiple times, then you have to process
> multiple backoff events, and perhaps reschedule your send for later
> after you've done that.
> 
> - Mark
>