Egress bandwidth limiting not working?

Egress bandwidth limiting not working?

[Sebastian Arcus]

I am trying to limit the bandwidth used by the egress leg as per the 
lartc cookbook here: 
http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html

The ingress works fine (well, it is a bit wobbly - but it limits the 
rate at roughly the expected level). However, the egress doesn't appear 
to be working. With no tc active, I get about 300mbs both ways (I'm 
testing in VMs). When it is on, the ingress is down to roughly 500kbs, 
but the egress is down to only about *150mbs*.

Interfaces on both sides use 1500 for MTU. Kernels on both sides are 
3.8.4. Has the syntax changed recently - or is the example at the link 
above incomplete? I appreciate any hints as to what I might be doing wrong.

For the purpose of testing, for egress, I've simplified everything down 
to one root qdisc, one class (to set the rate) and one filter (to catch 
everything):


##############################################
#  egress

# root cbq
tc qdisc add dev eth0 root handle 1: cbq avpkt 1000 \
     bandwidth 1000Mbit

# shape everything at $UPLINK speed - this prevents huge queues in your
# DSL modem which destroy latency:
tc class add dev eth0 parent 1: classid 1:1 cbq rate 500kbit \
     allot 1500 prio 5 bounded isolated

# include everything in the filter
tc filter add dev eth0 protocol ip parent 1:1 prio 3 u32 match \
     ip dst 0.0.0.0/32 flowid 1:1


##############################################
#  ingress

tc qdisc add dev eth0 handle ffff: ingress

tc filter add dev eth0 parent ffff: protocol ip prio 50 u32 \
     match ip src 0.0.0.0/0 police rate 500kbps burst 10k drop flowid :1

Re: Egress bandwidth limiting not working?

[Andrew Beverley]

On Tue, 2013-05-07 at 11:53 +0100, Sebastian Arcus wrote:
> I am trying to limit the bandwidth used by the egress leg as per the 
> lartc cookbook here: 
> http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html
> 
> The ingress works fine (well, it is a bit wobbly - but it limits the 
> rate at roughly the expected level). However, the egress doesn't appear 
> to be working. With no tc active, I get about 300mbs both ways (I'm 
> testing in VMs). When it is on, the ingress is down to roughly 500kbs, 
> but the egress is down to only about *150mbs*.

Have you tried a qdisc other than CBQ, such as HTB? I've never actually
used CBQ myself, but from the LARTC HOWTO:

"CBQ is the most complex qdisc available, the most hyped, the least
understood, and probably the trickiest one to get right. This is not
because the authors are evil or incompetent, far from it, it's just that
the CBQ algorithm isn't all that precise and doesn't really match the
way Linux works."

Are you just trying to achieve simple policing of the interface, or
something more advanced?

Andy

Re: Egress bandwidth limiting not working?

[Sebastian Arcus]

On 08/05/13 07:10, Andrew Beverley wrote:
> On Tue, 2013-05-07 at 11:53 +0100, Sebastian Arcus wrote:
>> I am trying to limit the bandwidth used by the egress leg as per the
>> lartc cookbook here:
>> http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html
>>
>> The ingress works fine (well, it is a bit wobbly - but it limits the
>> rate at roughly the expected level). However, the egress doesn't appear
>> to be working. With no tc active, I get about 300mbs both ways (I'm
>> testing in VMs). When it is on, the ingress is down to roughly 500kbs,
>> but the egress is down to only about *150mbs*.
>
> Have you tried a qdisc other than CBQ, such as HTB? I've never actually
> used CBQ myself, but from the LARTC HOWTO:
>
> "CBQ is the most complex qdisc available, the most hyped, the least
> understood, and probably the trickiest one to get right. This is not
> because the authors are evil or incompetent, far from it, it's just that
> the CBQ algorithm isn't all that precise and doesn't really match the
> way Linux works."
>
> Are you just trying to achieve simple policing of the interface, or
> something more advanced?
>

Thanks Andrew. I'm only trying to achieve bandwidth limitation and QoS 
priority for some VoIP traffic. I did not notice that paragraph above in 
the LARTC HOWTO and I certainly wouldn't want to complicate things 
unnecessarily :-). I'll give HTB a go and report back. The main reason I 
used CBQ is because the Cookbook section of LARTC recommends it in their 
Ultimate Traffic Conditioner chapter. It doesn't mention that CBQ is 
more complex - and the HTB section hints to the need to patch the kernel 
- which put me off at the time.

Re: Egress bandwidth limiting not working?

[Andrew Beverley]

On Wed, 2013-05-08 at 08:03 +0100, Sebastian Arcus wrote:
> > Are you just trying to achieve simple policing of the interface, or
> > something more advanced?
> >
> 
> Thanks Andrew. I'm only trying to achieve bandwidth limitation and QoS 
> priority for some VoIP traffic.

Yeah, I would just try HTB then. It's not the easiest either, but follow
some examples and you should be fine. The one thing to watch out for is
that the sum of the bandwidth for all the leaf classes should add up to
the bandwidth of the root class.

My example is at:

http://andybev.com/index.php/Fair_traffic_shaping_an_ADSL_line_for_a_local_network_using_Linux

[...]

> and the HTB section hints to the need to patch the kernel - which put
> me off at the time.

It's now in the vanilla kernel, so a lot easier :)


Andy

Re: Egress bandwidth limiting not working?

[Sebastian Arcus]

On 08/05/13 18:24, Andrew Beverley wrote:
> On Wed, 2013-05-08 at 08:03 +0100, Sebastian Arcus wrote:
>>> Are you just trying to achieve simple policing of the interface, or
>>> something more advanced?
>>>
>>
>> Thanks Andrew. I'm only trying to achieve bandwidth limitation and QoS
>> priority for some VoIP traffic.
>
> Yeah, I would just try HTB then. It's not the easiest either, but follow
> some examples and you should be fine.

I have tried the HTB example here: 
http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html and it seems 
to work fine - it certainly limits the bandwidth in both directions. 
Thanks again for that.

The one thing to watch out for is
> that the sum of the bandwidth for all the leaf classes should add up to
> the bandwidth of the root class.

In that case is the HTB example in the LARTC cookbook 
(http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html) wrong? It 
assigns to the first leaf class 100% of parent bandwidth, and to the 
second 90% of it.

>
> My example is at:
>
> http://andybev.com/index.php/Fair_traffic_shaping_an_ADSL_line_for_a_local_network_using_Linux
>
> [...]
>
>> and the HTB section hints to the need to patch the kernel - which put
>> me off at the time.
>
> It's now in the vanilla kernel, so a lot easier :)

It seems that HTB was added to the kernel in 2.4.20! - so it would be 
nice if somebody with access to LARTC.org would amend the mention about 
kernel patching on the page above. Also, it would be useful if the same 
page would mention that HTB is preferable and simpler than CBQ.

Re: Egress bandwidth limiting not working?

[Andrew Beverley]

On Wed, 2013-05-08 at 19:29 +0100, Sebastian Arcus wrote:
> > The one thing to watch out for is
> > that the sum of the bandwidth for all the leaf classes should add up to
> > the bandwidth of the root class.
> 
> In that case is the HTB example in the LARTC cookbook 
> (http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html) wrong? It 
> assigns to the first leaf class 100% of parent bandwidth, and to the 
> second 90% of it.

I've seen plenty of examples where people don't seem to have a problem.
However, I found this after a quick search, which seems to make sense:

"Because HTB does not shape in any class except the leaf class, the sum
of the rates of leaf classes should not exceed the ceil of a parent
class. Ideally, the sum of the rates of the children classes would match
the rate of the parent class, allowing the parent class to distribute
leftover bandwidth (ceil - rate) among the children classes"

http://www.tldp.org/HOWTO/Traffic-Control-HOWTO/classful-qdiscs.html

In other words, HTB only looks at each leaf separately when shaping, so
you could potentially have the total of the leaf classes trying to draw
more bandwidth than the total defined.

> It seems that HTB was added to the kernel in 2.4.20! - so it would be 
> nice if somebody with access to LARTC.org would amend the mention about 
> kernel patching on the page above. Also, it would be useful if the same 
> page would mention that HTB is preferable and simpler than CBQ.

Agreed, there is a lot of work to be done updating the documentation.
The LARTC documentation is open to patches though, so if you get a
moment please consider submitting a patch!

Andy