[LARTC] how to get the latency down on maxed out classes?

[LARTC] how to get the latency down on maxed out classes?

[Abraham van der Merwe]

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

Hi!

I'm using HTB to shape traffic to/from clients, but one of the problems I
have is that once a class utilizes its maximum potential, they latency
spirals out of control.

For example:

     .-----.
     | isp |
     `-----'
        |
   .--------.
   | shaper |
   `--------'
        |
   .--------.
   | client |
   `--------'

lets say I want to limit traffic to/from client to 64kbit. now, client opens
a tcp connection blasting away at full speed.

If client now pings isp, it gets on average around 7 seconds latency. I
tried to improve this by using SFQ on the leaf nodes of my HTB hierarchy,
but that does not really improve the situation, only makes it much worse.
with SFQ I get anything between 250ms and 13 seconds latency.

I then tried fifos. With small packet fifos the packet loss is just
to great to be of any use and even then the latency is quite high (~200ms).
If I make the fifos big enough so that unreasonable numbers of packets isn't
dropped, it just doesn't do much to the latency/or throughput. This
behaviour kind of makes sense, but doesn't help me :P

I'm thinking of using RED, but the number of parameters is daunting and I
have no idea how the HTB rate correlates to packet size and burst rates for
red.

Does anybody have any idea how to get the latency down and still maintain
the correct throughput?

-- 

Regards
 Abraham

Sooner or later you must pay for your sins.
(Those who have already paid may disregard this cookie).

___________________________________________________
 Abraham vd Merwe [ZR1BBQ] - Frogfoot Networks
 P.O. Box 3472, Matieland, Stellenbosch, 7602
 Cell: +27 82 565 4451 Http: http://www.frogfoot.net
 Email: abz@frogfoot.net


[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [LARTC] how to get the latency down on maxed out classes?

[hare ram]

Hi


even i have same problem using CBQ.INIT
let me know if you get any solution
thanks
hare
----- Original Message ----- 
From: "Abraham van der Merwe" <abz@frogfoot.net>
To: "Linux Advanced Routing & Traffic Control list" <lartc@mailman.ds9a.nl>
Sent: Saturday, December 07, 2002 6:45 PM
Subject: [LARTC] how to get the latency down on maxed out classes?



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] how to get the latency down on maxed out classes?

[Stef Coene]

On Saturday 07 December 2002 14:15, Abraham van der Merwe wrote:
> Hi!
>
> I'm using HTB to shape traffic to/from clients, but one of the problems I
> have is that once a class utilizes its maximum potential, they latency
> spirals out of control.
>
> For example:
>
>      .-----.
>
>      | isp |
>
>      `-----'
>
>    .--------.
>
>    | shaper |
>
>    `--------'
>
>    .--------.
>
>    | client |
>
>    `--------'
>
> lets say I want to limit traffic to/from client to 64kbit. now, client
> opens a tcp connection blasting away at full speed.
>
> If client now pings isp, it gets on average around 7 seconds latency. I
> tried to improve this by using SFQ on the leaf nodes of my HTB hierarchy,
> but that does not really improve the situation, only makes it much worse.
> with SFQ I get anything between 250ms and 13 seconds latency.
>
> I then tried fifos. With small packet fifos the packet loss is just
> to great to be of any use and even then the latency is quite high (~200ms).
> If I make the fifos big enough so that unreasonable numbers of packets
> isn't dropped, it just doesn't do much to the latency/or throughput. This
> behaviour kind of makes sense, but doesn't help me :P
>
> I'm thinking of using RED, but the number of parameters is daunting and I
> have no idea how the HTB rate correlates to packet size and burst rates for
> red.
>
> Does anybody have any idea how to get the latency down and still maintain
> the correct throughput?
If you want low latency for some traffic (ping, telnet, ssh), then you can 
create a separate class for it. 
And if you have a 64kbit modem, you have to be sure you never send more data 
then the modem can handle.  If you send more data, the hugh modem queue's 
will be filled so they create a lot of latency.  So for a 64kbit link, try to 
limit ALL traffic to 60kbit so the queue's of the modem are never filled.

Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] how to get the latency down on maxed out classes?

[Abraham van der Merwe]

[-- Attachment #1: Type: text/plain, Size: 1395 bytes --]

Hi Stef!

> > Does anybody have any idea how to get the latency down and still maintain
> > the correct throughput?
> If you want low latency for some traffic (ping, telnet, ssh), then you can 
> create a separate class for it. 
> And if you have a 64kbit modem, you have to be sure you never send more data 
> then the modem can handle.  If you send more data, the hugh modem queue's 
> will be filled so they create a lot of latency.  So for a 64kbit link, try to 
> limit ALL traffic to 60kbit so the queue's of the modem are never filled.

I'm doing all my tests under ideal conditions (over 100mbit lan and shaping
the traffic to something low such as 384kbit). The problem is that the
latency becomes unnatural.

If you have a normal line, e.g. 64kbit and you saturate the line, the
latency still stays within limits, but with HTB, the latency can become very
high if you have multiple concurrent tcp sessions going at full steam in a
class. I can understand why it does this, but I need a way to get the
latency down to acceptable limits.


-- 

Regards
 Abraham

I never killed a man that didn't deserve it.
		-- Mickey Cohen

___________________________________________________
 Abraham vd Merwe [ZR1BBQ] - Frogfoot Networks
 P.O. Box 3472, Matieland, Stellenbosch, 7602
 Cell: +27 82 565 4451 Http: http://www.frogfoot.net
 Email: abz@frogfoot.net


[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [LARTC] how to get the latency down on maxed out classes?

[Stef Coene]

On Saturday 07 December 2002 17:49, Abraham van der Merwe wrote:
> Hi Stef!
>
> > > Does anybody have any idea how to get the latency down and still
> > > maintain the correct throughput?
> >
> > If you want low latency for some traffic (ping, telnet, ssh), then you
> > can create a separate class for it.
> > And if you have a 64kbit modem, you have to be sure you never send more
> > data then the modem can handle.  If you send more data, the hugh modem
> > queue's will be filled so they create a lot of latency.  So for a 64kbit
> > link, try to limit ALL traffic to 60kbit so the queue's of the modem are
> > never filled.
>
> I'm doing all my tests under ideal conditions (over 100mbit lan and shaping
> the traffic to something low such as 384kbit). The problem is that the
> latency becomes unnatural.
>
> If you have a normal line, e.g. 64kbit and you saturate the line, the
> latency still stays within limits, but with HTB, the latency can become
> very high if you have multiple concurrent tcp sessions going at full steam
> in a class. I can understand why it does this, but I need a way to get the
> latency down to acceptable limits.
The only thing where I can think of, is adding a small fifo to each class.  
But you already tried and it created packets loss.  So I can't help you.

Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] how to get the latency down on maxed out classes?

[David Boreham]

> If you have a normal line, e.g. 64kbit and you saturate the line, the
> latency still stays within limits, but with HTB, the latency can become
> very high if you have multiple concurrent tcp sessions going at full steam
> in a class. I can understand why it does this, but I need a way to get the
> latency down to acceptable limits.

Are you sure the latency and packet loss are not the same,
when you have the same queue size as the router has ?

Is the traffic locally generated (on the same machine as the
HTB is running) ? If so then perhaps that's the difference.



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

[LARTC] how to get the latency down on maxed out classes?

[Don Cohen]

 > lets say I want to limit traffic to/from client to 64kbit. now, client opens
 > a tcp connection blasting away at full speed.
 > 
 > If client now pings isp, it gets on average around 7 seconds latency. I
 > tried to improve this by using SFQ on the leaf nodes of my HTB hierarchy,
 > but that does not really improve the situation, only makes it much worse.
 > with SFQ I get anything between 250ms and 13 seconds latency.

You understand what's going on here?
As I recall, both pfifo and sfq default to queues of length 128
packets.  If you fill that with 1500 byte packets you have ~200Kbytes
which is about 1.6Mbits.  At 64Kbit/sec that would take ~30 sec to
send so your latency could be as high as 30 sec.
You can limit this latency by reducing the queue size.

On the other hand, the application that fills the queue evidently
doesn't mind large latency.  Otherwise it wouldn't fill the queue.

I think I posted to this list once a description (maybe even the
code?) of another way to limit latency - drop packets that have been
in the queue for more than a timeout period (I tend to use 3 sec).

SFQ should have the desirable result that one tcp connection won't
slow down another one or a ping.

 > I then tried fifos. With small packet fifos the packet loss is just
 > to great to be of any use and even then the latency is quite high (~200ms).
You consider 200ms high?  One max size packet = 1500 bytes = 12Kbit
which is about 200ms on a 64Kbit link.  You can't expect to do better.

 > I'm thinking of using RED, but the number of parameters is daunting and I
 > have no idea how the HTB rate correlates to packet size and burst rates for
 > red.
RED should be independent of HTB.  
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] how to get the latency down on maxed out classes?

[Abraham van der Merwe]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

Hi Don!

>  > I then tried fifos. With small packet fifos the packet loss is just
>  > to great to be of any use and even then the latency is quite high (~200ms).
> You consider 200ms high?  One max size packet = 1500 bytes = 12Kbit
> which is about 200ms on a 64Kbit link.  You can't expect to do better.

The problem is that with 200ms the packet loss is so much that the link is
effectively useless (90% packet loss). As soon as I make the queue big
enough to not drop significant amounts of packets, the latency goes way up
(>3 secs).


[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [LARTC] how to get the latency down on maxed out classes?

[Don Cohen]

Abraham van der Merwe writes:
 > Hi Don!
 > 
 > >  > I then tried fifos. With small packet fifos the packet loss is just
 > >  > to great to be of any use and even then the latency is quite high (~200ms).

A small detail: what are "small packet fifos"?  You mean fifos that
can only hold a small number of packets?  Or fifos that only hold
packets with small numbers of bytes?

 > > You consider 200ms high?  One max size packet = 1500 bytes = 12Kbit
 > > which is about 200ms on a 64Kbit link.  You can't expect to do better.
 > 
 > The problem is that with 200ms the packet loss is so much that the link is
 > effectively useless (90% packet loss). As soon as I make the queue big
 > enough to not drop significant amounts of packets, the latency goes way up
 > (>3 secs).

I don't understand the connection between 200ms and packet loss.
If you make the queue small (in packet capacity) then worst case
latency decreases.  Packet loss occurs in either case whenever a
packet arrives and the queue is full.

If you try to send at a higher rate than allowed then you will fill
the queue in either case (a small queue more quickly, of course),
and from then on you will lose packets.  If you send packets at twice 
the allowed rate you lose half of them, if you send at 10 times the
allowed rate you lose 90%.

The fact that you're losing lots of packets, though, indicates to me
that you're acting like an attacker, and dropping most of that traffic
is therefore exactly the right thing to do.

If you were using a correctly working tcp it would not continue to
send at 10 times the allowed rate.  It would notice that packets were
being lost and would slow down until the loss rate became very small.

Similarly, I don't understand the latency issue.  An application that
cares about latency will not create a large backlog.

What is this application that is sending faster than the link allows
and wants a low latency, and why is it misbehaving?
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] how to get the latency down on maxed out classes?

[David Boreham]

> Similarly, I don't understand the latency issue.  An application that
> cares about latency will not create a large backlog.

I think the problem may be that a TCP flow will create a queue
of packets, behind which all other packets queue. Thus one sees
200ms latency via 'ping'. Of course this is exactly what you would
see on a real connection of the specified throttled speed (e.g. a 
DSL line). However it seems to surprise folks that there's a relationship
between throughput and latency. They expect to be able to throttle
traffic to 100Kbits, but still see 5ms latency. Without special
measures that isn't going to happen.


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/