Cake not doing rate limiting in a way it is expected to do

Cake not doing rate limiting in a way it is expected to do

[Akshat Kakkar]

Cake is expected to handle traffic in 2 steps :
First is on the basis of host
Second is within every host, on the basis of flow

So, if I limit traffic to 20Mbps shared across 2 host A & B,
Following are various scenarios, expectation and observations
1. If either A or B is downloading, they will be getting speed of 20Mbps
Observation: Meeting with expectation

2. If both A & B downloads (single download each), each will be
getting speed of 20Mbps
Observation: Meeting with expecation but its very jittery (around
20%), i.e. speed varies from 8Mbps to 12 Mbps. If I use fq_codel speed
is same BUT jitter is very less (around 1%).

3. Now if A starts 3 downloads, and B is still having single download,
A each download should be around 3.3 Mbps and B should be around
10Mbps
Observation: Around 5 Mbps for each download with lot of jitter, i.e.
no advantage of having CAKE!!!

Linux Kernel 4.20
For case 3, output of command : tc -s class show dev eno2

class htb 1:1 root leaf 8003: prio 1 rate 20000Kbit ceil 20000Kbit
burst 200Kb cburst 1600b
 Sent 688474645 bytes 455058 pkt (dropped 0, overlimits 381196 requeues 0)
 rate 19874Kbit 1641pps backlog 21196b 14p requeues 0
 lended: 382532 borrowed: 0 giants: 0
 tokens: 1260573 ctokens: -9427

class cake 8003:44f parent 8003:
 (dropped 3404, overlimits 0 requeues 0)
 backlog 9084b 6p requeues 0
class cake 8003:516 parent 8003:
 (dropped 3565, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
class cake 8003:590 parent 8003:
 (dropped 3023, overlimits 0 requeues 0)
 backlog 4542b 3p requeues 0
class cake 8003:605 parent 8003:
 (dropped 1772, overlimits 0 requeues 0)
 backlog 7570b 5p requeues 0

Re: Cake not doing rate limiting in a way it is expected to do

[Toke Høiland-Jørgensen]

[ adding cake list ]

Akshat Kakkar <akshat.1984@gmail.com> writes:

> Cake is expected to handle traffic in 2 steps :
> First is on the basis of host
> Second is within every host, on the basis of flow
>
> So, if I limit traffic to 20Mbps shared across 2 host A & B,
> Following are various scenarios, expectation and observations
> 1. If either A or B is downloading, they will be getting speed of 20Mbps
> Observation: Meeting with expectation
>
> 2. If both A & B downloads (single download each), each will be
> getting speed of 20Mbps
> Observation: Meeting with expecation but its very jittery (around
> 20%), i.e. speed varies from 8Mbps to 12 Mbps. If I use fq_codel speed
> is same BUT jitter is very less (around 1%).
>
> 3. Now if A starts 3 downloads, and B is still having single download,
> A each download should be around 3.3 Mbps and B should be around
> 10Mbps
> Observation: Around 5 Mbps for each download with lot of jitter, i.e.
> no advantage of having CAKE!!!
>
> Linux Kernel 4.20
> For case 3, output of command : tc -s class show dev eno2
>
> class htb 1:1 root leaf 8003: prio 1 rate 20000Kbit ceil 20000Kbit
> burst 200Kb cburst 1600b
>  Sent 688474645 bytes 455058 pkt (dropped 0, overlimits 381196 requeues 0)
>  rate 19874Kbit 1641pps backlog 21196b 14p requeues 0
>  lended: 382532 borrowed: 0 giants: 0
>  tokens: 1260573 ctokens: -9427
>
> class cake 8003:44f parent 8003:
>  (dropped 3404, overlimits 0 requeues 0)
>  backlog 9084b 6p requeues 0
> class cake 8003:516 parent 8003:
>  (dropped 3565, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
> class cake 8003:590 parent 8003:
>  (dropped 3023, overlimits 0 requeues 0)
>  backlog 4542b 3p requeues 0
> class cake 8003:605 parent 8003:
>  (dropped 1772, overlimits 0 requeues 0)
>  backlog 7570b 5p requeues 0

Could you please share some more details of your setup? The output of
`tc -s qdisc show dev eno2`?

How are you running the download tests? Is this over the internet, or in
a controlled setup? What's your actual line rate? Are you using ingress
shaping as well?

It looks like you are using HTB for rate limiting on top of CAKE? Why?

-Toke

Re: Cake not doing rate limiting in a way it is expected to do

[Akshat Kakkar]

It's a controlled lab setup. Users connected to eno2 and server on eno1.
Link speed 1Gbps.
No ingress shaping.
Simple http download.

I am having multiple rates requirement for multiple user groups, which
I am controlling using various classes and thus using htb.

Just to mention once again, fq_codel is working it's expected way.

Re: Cake not doing rate limiting in a way it is expected to do

[Toke Høiland-Jørgensen]

Akshat Kakkar <akshat.1984@gmail.com> writes:

> It's a controlled lab setup. Users connected to eno2 and server on eno1.
> Link speed 1Gbps.
> No ingress shaping.
> Simple http download.
>
> I am having multiple rates requirement for multiple user groups, which
> I am controlling using various classes and thus using htb.

Well, CAKE has its own built-in shaper, so it hasn't seen much testing
with HTB as a parent. Theoretically it *should* work, though, as long as
CAKE is running in unlimited mode.

Could you please share your full config, and the output of `tc -s qdisc`
after a run?

Thanks,

-Toke

Re: Cake not doing rate limiting in a way it is expected to do

[Akshat Kakkar]

Hi Toke,
I will try to rebuild my setup and then get back to you. I was off due
to an injury.

Thanks,

On Tue, May 28, 2019 at 9:50 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
> Akshat Kakkar <akshat.1984@gmail.com> writes:
>
> > It's a controlled lab setup. Users connected to eno2 and server on eno1.
> > Link speed 1Gbps.
> > No ingress shaping.
> > Simple http download.
> >
> > I am having multiple rates requirement for multiple user groups, which
> > I am controlling using various classes and thus using htb.
>
> Well, CAKE has its own built-in shaper, so it hasn't seen much testing
> with HTB as a parent. Theoretically it *should* work, though, as long as
> CAKE is running in unlimited mode.
>
> Could you please share your full config, and the output of `tc -s qdisc`
> after a run?
>
> Thanks,
>
> -Toke