TCP congestion graphs

TCP congestion graphs

[Stephen Hemminger]

I ran some congestion window tests against 2.6.19-rc3.
For congestion window graphs see:
	http://developer.osdl.org/shemminger/tcp/2.6.19-rc3/

The connection was a single flow with a 500ms RTT and a 
100Mbit slowest link speed.

BIC  			OK
CUBIC 			OK (after patch)
HIGHSPEED		BROKEN?
HTCP			OK
RENO			OK (massive overshoot)
SCALEABLE		OK
VEGAS			OK (no better than Reno)
VENO			Same as Reno?
WESTWOOD		Overshoot then low window

Highspeed has something wrong (test glitch?)
Westwood behaves poorly
Veno seems no better than either Reno or Vegas in this
case.

The end systems were from latest git. The netem bridge is using
2.6.18-rt with a patch to netem to use hrtimers.  iperf was modified
to allow easy selection of congestion control.

-- 
Stephen Hemminger <shemminger@osdl.org>

Re: TCP congestion graphs

[Injong Rhee]

Not sure why the slow start for cubic is slower than the others.
We will check on this.

----- Original Message ----- 
From: "Stephen Hemminger" <shemminger@osdl.org>
To: "Douglas Leith" <doug.leith@nuim.ie>; "Sangtae Ha" <sha2@ncsu.edu>
Cc: <netdev@vger.kernel.org>
Sent: Wednesday, October 25, 2006 2:02 PM
Subject: TCP congestion graphs


>I ran some congestion window tests against 2.6.19-rc3.
> For congestion window graphs see:
> http://developer.osdl.org/shemminger/tcp/2.6.19-rc3/
> 
> The connection was a single flow with a 500ms RTT and a 
> 100Mbit slowest link speed.
> 
> BIC  OK
> CUBIC OK (after patch)
> HIGHSPEED BROKEN?
> HTCP OK
> RENO OK (massive overshoot)
> SCALEABLE OK
> VEGAS OK (no better than Reno)
> VENO Same as Reno?
> WESTWOOD Overshoot then low window
> 
> Highspeed has something wrong (test glitch?)
> Westwood behaves poorly
> Veno seems no better than either Reno or Vegas in this
> case.
> 
> The end systems were from latest git. The netem bridge is using
> 2.6.18-rt with a patch to netem to use hrtimers.  iperf was modified
> to allow easy selection of congestion control.
> 
> -- 
> Stephen Hemminger <shemminger@osdl.org>
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: TCP congestion graphs

[Stephen Hemminger]

On Wed, 25 Oct 2006 18:34:17 -0400
"Injong Rhee" <rhee@eos.ncsu.edu> wrote:

> Not sure why the slow start for cubic is slower than the others.
> We will check on this.
> 

I think it is because cubic initializes with a ssthresh of 100,
and others leave ssthresh uninitialized until the first loss.
This causes cubic to change to conservative mode once it crosses
the initial ssthresh.  Other protocols just keep doing additive
increase.

My large BDP test has a 500ms delay (250ms each way) and a real
100mbit link on the last hop.
Because the intermediate emulator has a large queue (10,000 packets)
large growth is possible before the first loss event. There were also
no other flows during this test to fill the intermediate queue.

This may very well be a bogus test, I just was looking for large
BDP behavior quirks. Any test without background traffic as you
have argued is unrealistic.

Re: TCP congestion graphs

[Hagen Paul Pfeifer]

Hi Stephen,

is your rt-patch to netem public available?

Best regards

HGN


-- 
Signed and/or encrypted mails preferd. Key-Id = 0x98350C22
Fingerprint = 490F 557B 6C48 6D7E 5706  2EA2 4A22 8D45 9835 0C22 
Key available under: www.jauu.net/download/gnupg_key 

Re: TCP congestion graphs

[Stephen Hemminger]

On Thu, 26 Oct 2006 20:50:19 +0200
Hagen Paul Pfeifer <hagen@jauu.net> wrote:

> Hi Stephen,
> 
> is your rt-patch to netem public available?
> 
> Best regards
> 
> HGN
> 

The tools are in the tcp directory

http://developer.osdl.org/shemminger/tcp/netem-2.6.18-rt.patch


-- 
Stephen Hemminger <shemminger@osdl.org>

TCP congestion graphs (2.6.19-rc4)

[Stephen Hemminger]

I reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).
	http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dsl

The only obvious problem is Vegas looks broken.

Re: TCP congestion graphs (2.6.19-rc4)

[Xiaoliang (David) Wei]

On 10/31/06, Stephen Hemminger <shemminger@osdl.org> wrote:
> I reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).
>         http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dsl
>
> The only obvious problem is Vegas looks broken.

Thanks, Stephen.

It seems that the default Vegas alpha parameter in the rc4 is 1...

I observed similar situation with the NS2Linux simulator (with 2.6.16
code) and found that if alpha=1, delayed ack will make it broken
(keeping cwnd very low without real congestion)

See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas

(Basically alpha==1 means Vegas seeks to see a delay of about 1 packet
worth. With delayed ack, 1 packet worth of delay is common even with
no congestion.)

To make Vegas work, I'd suggest to raise alpha to at least 2 or 3.
(and beta has to be at least as large as alpha.)

-David

-- 
Xiaoliang (David) Wei      Graduate Student, CS@Caltech
http://davidwei.org
***********************************************

Re: TCP congestion graphs (2.6.19-rc4)

[Stephen Hemminger]

On Tue, 31 Oct 2006 15:25:16 -0800
"Xiaoliang (David) Wei" <davidwei79@gmail.com> wrote:

> On 10/31/06, Stephen Hemminger <shemminger@osdl.org> wrote:
> > I reran the tests, this time using my DSL line to home (80ms 1Mbit/sec).
> >         http://developer.osdl.org/shemminger/tcp/2.6.19-rc4/dsl
> >
> > The only obvious problem is Vegas looks broken.
> 
> Thanks, Stephen.
> 
> It seems that the default Vegas alpha parameter in the rc4 is 1...
> 
> I observed similar situation with the NS2Linux simulator (with 2.6.16
> code) and found that if alpha=1, delayed ack will make it broken
> (keeping cwnd very low without real congestion)
> 
> See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
> 
> (Basically alpha==1 means Vegas seeks to see a delay of about 1 packet
> worth. With delayed ack, 1 packet worth of delay is common even with
> no congestion.)
> 
> To make Vegas work, I'd suggest to raise alpha to at least 2 or 3.
> (and beta has to be at least as large as alpha.)
> 
> -David
> 

I ran with the current default:
	alpha = 1 (scaled 2)
	beta  = 3 (scaled 6)
	gamma = 1 (scaled 2)




-- 
Stephen Hemminger <shemminger@osdl.org>

Re: TCP congestion graphs (2.6.19-rc4)

[David Miller]

From: Stephen Hemminger <shemminger@osdl.org>
Date: Tue, 31 Oct 2006 16:10:07 -0800

> On Tue, 31 Oct 2006 15:25:16 -0800
> "Xiaoliang (David) Wei" <davidwei79@gmail.com> wrote:
> 
> > It seems that the default Vegas alpha parameter in the rc4 is 1...
> > 
> > I observed similar situation with the NS2Linux simulator (with 2.6.16
> > code) and found that if alpha=1, delayed ack will make it broken
> > (keeping cwnd very low without real congestion)
> > 
> > See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
> > 
> > (Basically alpha==1 means Vegas seeks to see a delay of about 1 packet
> > worth. With delayed ack, 1 packet worth of delay is common even with
> > no congestion.)
> > 
> > To make Vegas work, I'd suggest to raise alpha to at least 2 or 3.
> > (and beta has to be at least as large as alpha.)
> > 
> > -David
> > 
> 
> I ran with the current default:
> 	alpha = 1 (scaled 2)
> 	beta  = 3 (scaled 6)
> 	gamma = 1 (scaled 2)

Testing with alpha=2 and beta=4 would be interesting.

Brakmo and Peterson don't see this delayed ACK behavior in their tests
in the original Vegas paper.  I read through it a few times and I see
no explicit mention of whether they had delayed ACKs enabled or not in
any of there tests.

I seem to recall that it was very popular around that time to analyze
TCP changes with delayed ACKs disabled since it made the results
easier to analyze (which in my opinion invalidates any such results,
since in the real world you never run with delayed ACKs off).  So
perhaps the Vegas folks were doing something similar and just didn't
think it was worth mentioning :-)

BTW, Stephen, can you give a short HOWTO on how you generate those
graphs, given a tcpdump trace?  Thanks a lot.

Re: TCP congestion graphs (2.6.19-rc4)

[Evgeniy Polyakov]

On Tue, Oct 31, 2006 at 05:30:28PM -0800, David Miller (davem@davemloft.net) wrote:
> BTW, Stephen, can you give a short HOWTO on how you generate those
> graphs, given a tcpdump trace?  Thanks a lot.

That would be great.
Please also describe your setup so others algos could be tested too.
Thank you.

-- 
	Evgeniy Polyakov

Re: TCP congestion graphs (2.6.19-rc4)

[Stephen Hemminger]

On Wed, 1 Nov 2006 08:25:00 +0300
Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:

> On Tue, Oct 31, 2006 at 05:30:28PM -0800, David Miller (davem@davemloft.net) wrote:
> > BTW, Stephen, can you give a short HOWTO on how you generate those
> > graphs, given a tcpdump trace?  Thanks a lot.
> 
> That would be great.
> Please also describe your setup so others algos could be tested too.
> Thank you.
> 

Some basic stuff is on the parent page
	http://developer.osdl.org/shemminger/tcp

Re: TCP congestion graphs (2.6.19-rc4)

[David Miller]

From: Stephen Hemminger <shemminger@osdl.org>
Date: Tue, 31 Oct 2006 21:58:48 -0800

> Some basic stuff is on the parent page
> 	http://developer.osdl.org/shemminger/tcp

Are you using tcptrace to generate those plots?
If so are you using xplot or gnuplot to build those
image files?

Thanks.

Re: TCP congestion graphs (2.6.19-rc4)

[Stephen Hemminger]

On Wed, 01 Nov 2006 21:44:18 -0800 (PST)
David Miller <davem@davemloft.net> wrote:

> From: Stephen Hemminger <shemminger@osdl.org>
> Date: Tue, 31 Oct 2006 21:58:48 -0800
> 
> > Some basic stuff is on the parent page
> > 	http://developer.osdl.org/shemminger/tcp
> 
> Are you using tcptrace to generate those plots?
> If so are you using xplot or gnuplot to build those
> image files?
> 

Description now up on linux-net
	http://linux-net.osdl.org/index.php/TCP_testing

I use tcpprobe kernel module because you can't directly see congestion
window from packet captures.



-- 
Stephen Hemminger <shemminger@osdl.org>

Re: TCP congestion graphs (2.6.19-rc4)

[David Miller]

From: David Miller <davem@davemloft.net>
Date: Tue, 31 Oct 2006 17:30:28 -0800 (PST)

> From: Stephen Hemminger <shemminger@osdl.org>
> Date: Tue, 31 Oct 2006 16:10:07 -0800
> 
> > On Tue, 31 Oct 2006 15:25:16 -0800
> > "Xiaoliang (David) Wei" <davidwei79@gmail.com> wrote:
> > 
> > > It seems that the default Vegas alpha parameter in the rc4 is 1...
> > > 
> > > I observed similar situation with the NS2Linux simulator (with 2.6.16
> > > code) and found that if alpha=1, delayed ack will make it broken
> > > (keeping cwnd very low without real congestion)
> > > 
> > > See details at http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
> > > 
> > > (Basically alpha==1 means Vegas seeks to see a delay of about 1 packet
> > > worth. With delayed ack, 1 packet worth of delay is common even with
> > > no congestion.)
> > > 
> > > To make Vegas work, I'd suggest to raise alpha to at least 2 or 3.
> > > (and beta has to be at least as large as alpha.)
> > > 
> > > -David
> > > 
> > 
> > I ran with the current default:
> > 	alpha = 1 (scaled 2)
> > 	beta  = 3 (scaled 6)
> > 	gamma = 1 (scaled 2)
> 
> Testing with alpha=2 and beta=4 would be interesting.

Instead of letting this issue rot, I've checked the following into
net-2.6.20

commit cd7f265b9069d8fd66a33d37139821f84ef04f0e
Author: David S. Miller <davem@sunset.davemloft.net>
Date:   Tue Nov 28 14:37:38 2006 -0800

    [TCP] Vegas: Increase default alpha to 2 and beta to 4.
    
    This helps Vegas cope better with delayed ACKs, see
    analysis at:
    
    http://www.cs.caltech.edu/%7Eweixl/technical/ns2linux/known_linux/index.html#vegas
    
    Signed-off-by: David S. Miller <davem@davemloft.net>

diff --git a/net/ipv4/tcp_vegas.c b/net/ipv4/tcp_vegas.c
index a3b7aa0..ddc4bcc 100644
--- a/net/ipv4/tcp_vegas.c
+++ b/net/ipv4/tcp_vegas.c
@@ -42,8 +42,8 @@ #include <net/tcp.h>
  * with V_PARAM_SHIFT bits to the right of the binary point.
  */
 #define V_PARAM_SHIFT 1
-static int alpha = 1<<V_PARAM_SHIFT;
-static int beta  = 3<<V_PARAM_SHIFT;
+static int alpha = 2<<V_PARAM_SHIFT;
+static int beta  = 4<<V_PARAM_SHIFT;
 static int gamma = 1<<V_PARAM_SHIFT;
 
 module_param(alpha, int, 0644);