[LARTC] CBQ only works 'bounded'

[LARTC] CBQ only works 'bounded'

[Patrick Nagelschmidt]

what i try to achieve with QoS is to divide my outgoing traffic in one high priority class and one low priority class. i have an asymetric connection with 768 kbit/s downstream and 128 kbit/s upstream and want to avoid ack-congestions by sending ack packets through the high priority class and normal outbound traffic through the low priority class. what i've done so far is the following:

#create the root class
tc qdisc add dev ppp0 root handle 1: cbq bandwidth 10Mbit avpkt 1000 mpu 64

#limit outbound traffic to 128kbit/s total
tc class add dev ppp0 parent 1: classid 1:1 cbq bandwidth 10Mbit rate 128kbit allot 1514 weight 12.8kbit prio 8 maxburst 20 avpkt 1000 bounded

#create a high priority class
tc class add dev ppp0 parent 1:1 classid 1:10 cbq bandwidth 10Mbit rate 20kbit allot 1514 weight 2.0kbit prio 1 maxburst 20 avpkt 1000

#create a low priority class
tc class add dev ppp0 parent 1:1 classid 1:20 cbq bandwidth 10Mbit rate 108kbit allot 1514 weight 10.8kbit prio 7 maxburst 20 avpkt 1000 bounded

#use sfq for both
tc qdisc add dev ppp0 parent 1:10 sfq quantum 1514b perturb 10
tc qdisc add dev ppp0 parent 1:20 sfq quantum 1514b perturb 10

#put everything in the low priority class
tc filter add dev ppp0 parent 1: protocol ip prio 20 u32 \
match u8 0x00 0x00 \
flowid 1:20

#except ack-packets (header length 0x05, total l. 0x34, ack set)
tc filter add dev ppp0 parent 1: protocol ip prio 10 u32 \
match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u8 0x34 0xff at 3 \
match u8 0x10 0xff at 33 \
flowid 1:10

the good thing is that this works. of course this doesn't mean its error-free, so please tell me if i've done something terribly wrong.

if i use this configurtion and upload with ~14.5kb/s (ack's not included) i can still download with ~84kb/s. but what i dont understand is that the whole thing stops working if i remove the 'bounded' from classid 1:20. i thought this would release the bandwidth of 1:10 to 1:20 _only_ if 1:10 doesn't need it. i set the prio of 1:10 to the highest value to ensure that if any ack-packets should be sent via 1:10 these packets get send first. but my download speed decreases to ~20kb/s if i upload at the same time. i think this indicates that ack packets have to wait behind some bigger data packets until they get sent. in numbers, i have with 1:20 set to 'bounded':

class cbq 1: root rate 10Mbit (bounded,isolated) prio no-transmit
 Sent 461859 bytes 1189 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 605 undertime 0
class cbq 1:10 parent 1:1 leaf 8146: rate 20Kbit prio 1
 Sent 33372 bytes 828 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 1.57292e+06 undertime 0
class cbq 1:1 parent 1: rate 128Kbit (bounded) prio no-transmit
 Sent 461859 bytes 1189 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 41470 undertime 0
class cbq 1:20 parent 1:1 leaf 8147: rate 108Kbit (bounded) prio 7
 Sent 428487 bytes 361 pkts (dropped 0, overlimits 1860)
 backlog 15p
  borrowed 0 overactions 197 avgidle -82792 undertime 84903

and without 'bounded'

class cbq 1: root rate 10Mbit (bounded,isolated) prio no-transmit
 Sent 1283284 bytes 1397 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 624 undertime 0
class cbq 1:10 parent 1:1 leaf 8148: rate 20Kbit prio 1
 Sent 18244 bytes 454 pkts (dropped 0, overlimits 0)
  borrowed 0 overactions 0 avgidle 2.66114e+06 undertime 0
class cbq 1:1 parent 1: rate 128Kbit (bounded) prio no-transmit
 Sent 1283284 bytes 1397 pkts (dropped 0, overlimits 0)
  borrowed 844 overactions 0 avgidle 219417 undertime 0
class cbq 1:20 parent 1:1 leaf 8149: rate 108Kbit prio 7
 Sent 1265040 bytes 943 pkts (dropped 0, overlimits 2564)
  borrowed 844 overactions 7 avgidle -186045 undertime 265135

now i'm curious what i have to do to make 1:10 share it's bandwidth in a way it doesn't ruin the whole mechanism.

also i would like to know what 'overactions' are and what it means (or at least if it is good or bad) if tc -s shows a 'backlog 15p' or similar in the statistic of e.g. class 1:20.

thanks for your help,

Patrick


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

Re: [LARTC] CBQ only works 'bounded'

[bert hubert]

On Fri, Dec 07, 2001 at 02:29:59AM +0100, Patrick Nagelschmidt wrote:

> what i try to achieve with QoS is to divide my outgoing traffic in one
> high priority class and one low priority class. i have an asymetric
> connection with 768 kbit/s downstream and 128 kbit/s upstream and want to
> avoid ack-congestions by sending ack packets through the high priority
> class and normal outbound traffic through the low priority class. what
> i've done so far is the following:

Can you please try not to send 1000 character long lines? Thanks!

What you try is cool - I was just about to experiment with ACK prioritizing.
> 
> #create the root class
> tc qdisc add dev ppp0 root handle 1: cbq bandwidth 10Mbit avpkt 1000 mpu 64

You talk to your modem over a 10mbit/s interface? 

> #limit outbound traffic to 128kbit/s total
> tc class add dev ppp0 parent 1: classid 1:1 cbq bandwidth 10Mbit rate \
>    128kbit allot 1514 weight 12.8kbit prio 8 maxburst 20 avpkt 1000 \
>    bounded

Looks good.

> #create a high priority class
> tc class add dev ppp0 parent 1:1 classid 1:10 cbq bandwidth 10Mbit \
>    rate 20kbit allot 1514 weight 2.0kbit prio 1 maxburst 20 avpkt 1000

This class is unbounded right now, so it can borrow from 1:20

> #create a low priority class
> tc class add dev ppp0 parent 1:1 classid 1:20 cbq bandwidth 10Mbit \
>    rate 108kbit allot 1514 weight 10.8kbit prio 7 maxburst 20 avpkt 1000 \
>    bounded

Also looks cool.

> #use sfq for both
> tc qdisc add dev ppp0 parent 1:10 sfq quantum 1514b perturb 10
> tc qdisc add dev ppp0 parent 1:20 sfq quantum 1514b perturb 10

Yadda yadda.

> #put everything in the low priority class
> tc filter add dev ppp0 parent 1: protocol ip prio 20 u32 \
>    match u8 0x00 0x00 \
>    flowid 1:20

Fine.

> #except ack-packets (header length 0x05, total l. 0x34, ack set)
> tc filter add dev ppp0 parent 1: protocol ip prio 10 u32 \
> match ip protocol 6 0xff \
> match u8 0x05 0x0f at 0 \
> match u8 0x34 0xff at 3 \
> match u8 0x10 0xff at 33 \
> flowid 1:10

I've seen this command before!

> the good thing is that this works. of course this doesn't mean its
> error-free, so please tell me if i've done something terribly wrong.

Looks fine to me.

> if i use this configurtion and upload with ~14.5kb/s (ack's not included)
> i can still download with ~84kb/s. but what i dont understand is that the

Cool. You weren't possible to do this before?

> whole thing stops working if i remove the 'bounded' from classid 1:20. 

Ok - you can still upload and download, but the positive effect disappears?

> I thought this would release the bandwidth of 1:10 to 1:20 _only_ if 1:10
> doesn't need it. i set the prio of 1:10 to the highest value to ensure
> that if any ack-packets should be sent via 1:10 these packets get send
> first. but my download speed decreases to ~20kb/s if i upload at the same
> time. i think this indicates that ack packets have to wait behind some
> bigger data packets until they get sent. in numbers, i have with 1:20 set
> to 'bounded':

Well, CBQ isn't all *that* precise. Bandwidth is sucked out of your
interactive class for certain chunks of time, again delaying ACKs. In the
long term, bandwidth will probably be available ok for your interactive
class but downloads get hurt even if ACKs get delayed in the short term.

> now i'm curious what i have to do to make 1:10 share it's bandwidth in a
> way it doesn't ruin the whole mechanism.

I've been experimenting with HTB and it appears to be better in this
respect. Even without priorities it managed to keep latency in the
interactive class *very* low.

It may be more estethically pleasing to set 1:10 to 'isolated' instead of
declaring 1:20 'bounded', although it comes down to the same thing.

> also i would like to know what 'overactions' are and what it means (or at
> least if it is good or bad) if tc -s shows a 'backlog 15p' or similar in
> the statistic of e.g. class 1:20.

'backlog 15p' simply means that 15 packets were waiting to be transmitted at
that time. 'overactions' occur when too many packets where queueing up, the
default 'overaction' is drop.

> thanks for your help,

One thing, try reversing the priorities and removing unbounded. Just a
hunch.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

Re: [LARTC] CBQ only works 'bounded'

[Patrick Nagelschmidt]

At 07.12.01, you wrote:
>You talk to your modem over a 10mbit/s interface? 

yes, it's pppoe. and i can recycle my old ne2k cards this way :)

>Cool. You weren't possible to do this before?

no, a normal 16kb/s upload ruined the whole download speed. i loose ~60%
of my downstream if i dont use QoS.

>Ok - you can still upload and download, but the positive effect disappears?

exactly. it's nearly the same as if i wouldn't start the script at all.

>Well, CBQ isn't all *that* precise. Bandwidth is sucked out of your
>interactive class for certain chunks of time, again delaying ACKs. In the
>long term, bandwidth will probably be available ok for your interactive
>class but downloads get hurt even if ACKs get delayed in the short term.

that seems to be the point. the low priority class only borrowed 844 packets
out of ~18000 but that was enough to delay acks remarkably. but now i can be
sure that it is a logistic problem, not a fundamental error in my script.

>I've been experimenting with HTB and it appears to be better in this
>respect. Even without priorities it managed to keep latency in the
>interactive class *very* low.

but i read in your howto, that it will divide remaining bandwidth in a ratio
based on the size of the classes. so this would mean it would devide it in
a 108:20 ratio. if it does the low priority class gets the bigger part and
i might have the same problem again. or do i underestimate HTB? :)

>One thing, try reversing the priorities

i tried it, but this hadn't a positive effect. isn't 1 the highest and 7 the 
lowest? 

regards,

Patrick


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

Re: [LARTC] CBQ only works 'bounded'

[Michael T. Babcock]

On Fri, Dec 07, 2001 at 12:13:33PM +0100, bert hubert wrote:
> > #create the root class
> > tc qdisc add dev ppp0 root handle 1: cbq bandwidth 10Mbit avpkt 1000 mpu 64
> 
> You talk to your modem over a 10mbit/s interface? 

If its over an ethernet card, it would even possibly be 100Mbit ... but a smaller
value for allot, according to the docs, right?  This started making sense to me
at some point in history, so I'm making sure now that you question it.

CBQ (10Mbit, allot 2Mbit) -> CBQ (2Mbit allot ...)
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

Re: [LARTC] CBQ only works 'bounded'

[bert hubert]

On Fri, Dec 07, 2001 at 02:19:20PM +0100, Patrick Nagelschmidt wrote:
> At 07.12.01, you wrote:
> >You talk to your modem over a 10mbit/s interface? 
> 
> yes, it's pppoe. and i can recycle my old ne2k cards this way :)

:-) I use a 486 with ne2k-pci too.

> >I've been experimenting with HTB and it appears to be better in this
> >respect. Even without priorities it managed to keep latency in the
> >interactive class *very* low.
> 
> but i read in your howto, that it will divide remaining bandwidth in a ratio
> based on the size of the classes. so this would mean it would devide it in
> a 108:20 ratio. if it does the low priority class gets the bigger part and
> i might have the same problem again. or do i underestimate HTB? :)

Ok, I tried this with HTB, commands can be found on http://ds9a.nl/shaper
with comments. It turns out that with HTB you need to leave some room for
your interactive class, in my case, I trimmed the bulk class by 20kbit/s.

You could probably also get away with assigning 220kbit/s to your
interactive class and 1kbit/s to the bulk class, but allowing it to borrow
up to 220kbit/s, which would save you from trimming off 10%.

> >One thing, try reversing the priorities
> 
> i tried it, but this hadn't a positive effect. isn't 1 the highest and 7 the 
> lowest? 

Hey, it was just a hunch :-)

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
Trilab                                 The Technology People
Netherlabs BV / Rent-a-Nerd.nl           - Nerd Available -
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

Re: [LARTC] CBQ only works 'bounded'

[Patrick Nagelschmidt]

At 07.12.01, you wrote:
>Ok, I tried this with HTB

the next time i recompile my kernel i'll include it and see if it is more
effective in my scenario.

>You could probably also get away with assigning 220kbit/s to your
>interactive class and 1kbit/s to the bulk class, but allowing it to borrow
>up to 220kbit/s, which would save you from trimming off 10%.

i allready tried that (not with HTB of course), but this resulted in the bulk
class borrowing so much from the interactive class that it rendered the
interactive class non interactive :)

regards,

Patrick


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

Re: [LARTC] CBQ only works 'bounded'

[Martin Devera]

> >I've been experimenting with HTB and it appears to be better in this
> >respect. Even without priorities it managed to keep latency in the
> >interactive class *very* low.
> 
> but i read in your howto, that it will divide remaining bandwidth in a ratio
> based on the size of the classes. so this would mean it would devide it in
> a 108:20 ratio. if it does the low priority class gets the bigger part and
> i might have the same problem again. or do i underestimate HTB? :)

yes you do :) The remaining bw is firts offered to hi prio classes
and only if they don;t want it ot they hit their top rate it will
offer to lower prios. And they divide it by their rate ratios.

The thing is a bit more complicated in presence of more levels
(deeper tree) here offering dscribed above is repeated for each
level to assure that no lo prio can steal from hi prio and testing
only rates for that depth ..
 
devik


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/