Weird traffic shaping behavior after kernel upgrade

Weird traffic shaping behavior after kernel upgrade

[Hoggins!]

Hello list,

This is my first post, so please forgive me if my ways are wrong, but
this is an important request for help.

I was running kernel 3.9.10 (Fedora-flavored), and for several reasons,
I needed to upgrade to the last version, 3.12.6. Yeah, it was about time.

Anyway, I really don't know how and where to look for an answer on my
problem, and since the kernel is the only thing that changed on this
machine, I suspect there has to be something with it.

I have the feeling that my traffic shaping is not working exactly as
expected anymore. I use it to "regulate" low-throughput (xDSL)
connections, and it was working great until now, but since I upgraded, I
feel that the experience is not as "fluid" as it used to be.

And I was surprised to see the following output :

---- SNIP ----
$ tc -s class show dev ppp0

class htb 1:11 parent 1:1 prio 0 rate 200000bit ceil 800000bit burst
1492b cburst 1492b
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 932500 ctokens: 233125

class htb 1:1 root rate 800000bit ceil 800000bit burst 1492b cburst 1492b
 Sent 293785084 bytes 341228 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 223981 borrowed: 0 giants: 0
 tokens: 26875 ctokens: 26875

class htb 1:13 parent 1:1 prio 1 rate 200000bit ceil 200000bit burst
1492b cburst 1492b
 Sent 5746052 bytes 25213 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 25213 borrowed: 0 giants: 0
 tokens: 832500 ctokens: 832500

class htb 1:12 parent 1:1 leaf 12: prio 2 rate 100000bit ceil 160000bit
burst 1492b cburst 1492b
 Sent 66792 bytes 1253 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 1253 borrowed: 0 giants: 0
 tokens: 1693898 ctokens: 1067648

class htb 1:15 parent 1:1 prio 1 rate 50000bit ceil 50000bit burst 1492b
cburst 1492b
 Sent 25153220 bytes 32605 pkt (dropped 6, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 32605 borrowed: 0 giants: 0
 tokens: -132903 ctokens: -132903

class htb 1:14 parent 1:1 leaf 14: prio 3 rate 80000bit ceil 400000bit
burst 1492b cburst 1492b
 Sent 262819020 bytes 282157 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 5p requeues 0
 lended: 58178 borrowed: 223981 giants: 0
 tokens: -1062034 ctokens: -412442

---- SNIP ----

What worries me is that the "rate" values are always zeroed, even when
under heavy traffic. It's not the case with the older version of the kernel.

So I was wondering if there was something I missed, and needed to update
in my traffic shaping script. Here it is :

---- SNIP ----

${TC} qdisc del dev ${IF} root
${TC} qdisc add dev ${IF} root handle 1: htb default 14
${TC} class add dev ${IF} parent 1: classid 1:1 htb rate ${MAX}kbps ceil
${MAX}kbps prio 0 mtu ${MTU}

# Vers TheDude
${TC} class add dev ${IF} parent 1:1 classid 1:11 htb rate 200kbit ceil
${MAX}kbps prio 0 mtu ${MTU}

# Paquets ACK
${TC} class add dev ${IF} parent 1:1 classid 1:12 htb rate 100kbit ceil
160kbit prio 2 mtu ${MTU}

# Téléphone
${TC} class add dev ${IF} parent 1:1 classid 1:13 htb rate 200kbit ceil
200kbit prio 1 mtu ${MTU}

# Traffic général
${TC} class add dev ${IF} parent 1:1 classid 1:14 htb rate 80kbit ceil
400kbit prio 3 mtu ${MTU}

# Vers Duplex
${TC} class add dev ${IF} parent 1:1 classid 1:15 htb rate 50kbit ceil
50kbit prio 1 mtu ${MTU}

${TC} qdisc add dev ${IF} parent 1:12 handle 12: sfq perturb 10
${TC} qdisc add dev ${IF} parent 1:14 handle 14: sfq perturb 10

${TC} filter add dev ${IF} protocol ip parent 1:0 handle 2 fw classid 1:11
${TC} filter add dev ${IF} protocol ip parent 1:0 handle 3 fw classid 1:12
${TC} filter add dev ${IF} protocol ip parent 1:0 handle 4 fw classid 1:13
${TC} filter add dev ${IF} protocol ip parent 1:0 handle 5 fw classid 1:14
${TC} filter add dev ${IF} protocol ip parent 1:0 handle 6 fw classid 1:15

---- SNIP ----

The packet MARKing in iptables is working as expected, as the counters
successfully increase over time.

I'm really stuck, and I need your help.

Happy New Year, and thanks in advance !

    Hoggins!

Re: Weird traffic shaping behavior after kernel upgrade

[Remy Mudingay]

Hi Hoggins,

The rate estimator was disabled in kernel 3.12. You can re-enable it
by doing the following :

 echo 1 > /sys/module/sch_htb/parameters/htb_rate_est

I hope that helps.

Cheers,


Remy

On 6 January 2014 00:08, Hoggins! <fuckspam@wheres5.com> wrote:
> Hello list,
>
> This is my first post, so please forgive me if my ways are wrong, but
> this is an important request for help.
>
> I was running kernel 3.9.10 (Fedora-flavored), and for several reasons,
> I needed to upgrade to the last version, 3.12.6. Yeah, it was about time.
>
> Anyway, I really don't know how and where to look for an answer on my
> problem, and since the kernel is the only thing that changed on this
> machine, I suspect there has to be something with it.
>
> I have the feeling that my traffic shaping is not working exactly as
> expected anymore. I use it to "regulate" low-throughput (xDSL)
> connections, and it was working great until now, but since I upgraded, I
> feel that the experience is not as "fluid" as it used to be.
>
> And I was surprised to see the following output :
>
> ---- SNIP ----
> $ tc -s class show dev ppp0
>
> class htb 1:11 parent 1:1 prio 0 rate 200000bit ceil 800000bit burst
> 1492b cburst 1492b
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>  rate 0bit 0pps backlog 0b 0p requeues 0
>  lended: 0 borrowed: 0 giants: 0
>  tokens: 932500 ctokens: 233125
>
> class htb 1:1 root rate 800000bit ceil 800000bit burst 1492b cburst 1492b
>  Sent 293785084 bytes 341228 pkt (dropped 0, overlimits 0 requeues 0)
>  rate 0bit 0pps backlog 0b 0p requeues 0
>  lended: 223981 borrowed: 0 giants: 0
>  tokens: 26875 ctokens: 26875
>
> class htb 1:13 parent 1:1 prio 1 rate 200000bit ceil 200000bit burst
> 1492b cburst 1492b
>  Sent 5746052 bytes 25213 pkt (dropped 0, overlimits 0 requeues 0)
>  rate 0bit 0pps backlog 0b 0p requeues 0
>  lended: 25213 borrowed: 0 giants: 0
>  tokens: 832500 ctokens: 832500
>
> class htb 1:12 parent 1:1 leaf 12: prio 2 rate 100000bit ceil 160000bit
> burst 1492b cburst 1492b
>  Sent 66792 bytes 1253 pkt (dropped 0, overlimits 0 requeues 0)
>  rate 0bit 0pps backlog 0b 0p requeues 0
>  lended: 1253 borrowed: 0 giants: 0
>  tokens: 1693898 ctokens: 1067648
>
> class htb 1:15 parent 1:1 prio 1 rate 50000bit ceil 50000bit burst 1492b
> cburst 1492b
>  Sent 25153220 bytes 32605 pkt (dropped 6, overlimits 0 requeues 0)
>  rate 0bit 0pps backlog 0b 0p requeues 0
>  lended: 32605 borrowed: 0 giants: 0
>  tokens: -132903 ctokens: -132903
>
> class htb 1:14 parent 1:1 leaf 14: prio 3 rate 80000bit ceil 400000bit
> burst 1492b cburst 1492b
>  Sent 262819020 bytes 282157 pkt (dropped 0, overlimits 0 requeues 0)
>  rate 0bit 0pps backlog 0b 5p requeues 0
>  lended: 58178 borrowed: 223981 giants: 0
>  tokens: -1062034 ctokens: -412442
>
> ---- SNIP ----
>
> What worries me is that the "rate" values are always zeroed, even when
> under heavy traffic. It's not the case with the older version of the kernel.
>
> So I was wondering if there was something I missed, and needed to update
> in my traffic shaping script. Here it is :
>
> ---- SNIP ----
>
> ${TC} qdisc del dev ${IF} root
> ${TC} qdisc add dev ${IF} root handle 1: htb default 14
> ${TC} class add dev ${IF} parent 1: classid 1:1 htb rate ${MAX}kbps ceil
> ${MAX}kbps prio 0 mtu ${MTU}
>
> # Vers TheDude
> ${TC} class add dev ${IF} parent 1:1 classid 1:11 htb rate 200kbit ceil
> ${MAX}kbps prio 0 mtu ${MTU}
>
> # Paquets ACK
> ${TC} class add dev ${IF} parent 1:1 classid 1:12 htb rate 100kbit ceil
> 160kbit prio 2 mtu ${MTU}
>
> # Téléphone
> ${TC} class add dev ${IF} parent 1:1 classid 1:13 htb rate 200kbit ceil
> 200kbit prio 1 mtu ${MTU}
>
> # Traffic général
> ${TC} class add dev ${IF} parent 1:1 classid 1:14 htb rate 80kbit ceil
> 400kbit prio 3 mtu ${MTU}
>
> # Vers Duplex
> ${TC} class add dev ${IF} parent 1:1 classid 1:15 htb rate 50kbit ceil
> 50kbit prio 1 mtu ${MTU}
>
> ${TC} qdisc add dev ${IF} parent 1:12 handle 12: sfq perturb 10
> ${TC} qdisc add dev ${IF} parent 1:14 handle 14: sfq perturb 10
>
> ${TC} filter add dev ${IF} protocol ip parent 1:0 handle 2 fw classid 1:11
> ${TC} filter add dev ${IF} protocol ip parent 1:0 handle 3 fw classid 1:12
> ${TC} filter add dev ${IF} protocol ip parent 1:0 handle 4 fw classid 1:13
> ${TC} filter add dev ${IF} protocol ip parent 1:0 handle 5 fw classid 1:14
> ${TC} filter add dev ${IF} protocol ip parent 1:0 handle 6 fw classid 1:15
>
> ---- SNIP ----
>
> The packet MARKing in iptables is working as expected, as the counters
> successfully increase over time.
>
> I'm really stuck, and I need your help.
>
> Happy New Year, and thanks in advance !
>
>     Hoggins!
>
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Weird traffic shaping behavior after kernel upgrade

[Hoggins!]

Hello,

Thanks for the help !
Unfortunately, setting this parameter to 1 does not make the rate
indicators change from zero. Strange.
The "Sent" counters do change over time, so I guess the traffic is
actually shaped, but maybe in a different way that I was used to. There
may also be another variable that I don't master, but I cannot see what
for the moment.

How can I track any other change that would have happened on the Linux
traffic classifier and HTB between these two kernel versions ?

Le 06/01/2014 02:50, Remy Mudingay a écrit :
> Hi Hoggins,
>
> The rate estimator was disabled in kernel 3.12. You can re-enable it
> by doing the following :
>
>  echo 1 > /sys/module/sch_htb/parameters/htb_rate_est
>
> I hope that helps.
>
> Cheers,
>
>
> Remy

Re: Weird traffic shaping behavior after kernel upgrade

[Remy Mudingay]

Hi again,

Disregard my previous email. It seems that you have to reload your htb
qdisc, class and filter rules after updating the kernel htb_rate_est
parameter.

On 6 January 2014 12:17, Remy Mudingay <remy.mudingay@gmail.com> wrote:
> Hi again,
>
> Disregard my previous email. It seems that you have to reload your htb
> qdisc, class and filter rules after updating the kernel htb_rate_est
> parameter.
>
> Remy
>
>
> On 6 January 2014 11:58, Remy Mudingay <remy.mudingay@gmail.com> wrote:
>>
>> Hi Hoggins,
>>
>> Maybe it's a bug but try setting it a couple of times. Do something like
>>
>> For i in 1 2 3 4; do
>> echo 1 > /sys/module/sch_htb/parameters/htb_rate_est
>> done
>>
>> I've just tested it in a 3.12.1 kernel.
>>
>> Ref: http://patchwork.ozlabs.org/patch/249551/
>>
>> Let us know how it goes.
>>
>> Remy
>>
>> Sent from my Phone
>>
>> > On 06 Jan 2014, at 09:59, "Hoggins!" <fuckspam@wheres5.com> wrote:
>> >
>> > Hello,
>> >
>> > Thanks for the help !
>> > Unfortunately, setting this parameter to 1 does not make the rate
>> > indicators change from zero. Strange.
>> > The "Sent" counters do change over time, so I guess the traffic is
>> > actually shaped, but maybe in a different way that I was used to. There
>> > may also be another variable that I don't master, but I cannot see what
>> > for the moment.
>> >
>> > How can I track any other change that would have happened on the Linux
>> > traffic classifier and HTB between these two kernel versions ?
>> >
>> > Le 06/01/2014 02:50, Remy Mudingay a écrit :
>> >> Hi Hoggins,
>> >>
>> >> The rate estimator was disabled in kernel 3.12. You can re-enable it
>> >> by doing the following :
>> >>
>> >> echo 1 > /sys/module/sch_htb/parameters/htb_rate_est
>> >>
>> >> I hope that helps.
>> >>
>> >> Cheers,
>> >>
>> >>
>> >> Remy
>> >
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe lartc" in
>> > the body of a message to majordomo@vger.kernel.org
>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

Re: Weird traffic shaping behavior after kernel upgrade

[Hoggins!]

Hello Dave,

I'll have to try that.
But for the moment, I found a "solution", expecting to use something
"cleaner" : I downgraded to kernel 3.10.25, and everything is working as
usual.
The machine is a production server for a little student radio in France,
so I cannot test a lot without shutting down the entire station.

The downgrade did the trick, so the problem is kinda "solved" for me
(although having the latest kernel would have been a better solution for
me I guess). Anyway, I'm going to tune these policies a bit, as you are
suggesting.

As for the reason I'm not running sfq on all qdiscs, well... as you
understand, with my very poor knowledge, I've been working on this
little script for a long time to have an acceptable level of service. I
had finally found the "perfect" match for the desired performances, and
the results of my "researches" showed that I had to use sfq only on the
qdiscs that did not require a "guaranteed" (although this is not
entirely correct, as we are working on a ADSL connection) throughput :
the classes having sfq are used for TCP operations such as web surfing,
and the returning of ACK packets for example. The other ones have fixed
limited bandwidth (not borrowable) for UDP (RTP) streams directly to
broadcast servers.

Thanks for your help, I'll get back on this list soon.

    Hoggins!


Le 06/01/2014 00:41, Dave Taht a écrit :
> Is there some reason you aren't running sfq on all qdiscs?  That could
> be a cause.
>
> and:
>
> try
>
> fq_codel target 20ms quantum 300  limit 600
>
> instead of sfq.

Re: Weird traffic shaping behavior after kernel upgrade

[Hoggins!]

Yep. That did the trick for a correct display of the rate stats.
But as you see in my previous e-mail, I chose another way to "make
things work".

Thanks a lot for your help !

Le 06/01/2014 12:18, Remy Mudingay a écrit :
> Hi again,
>
> Disregard my previous email. It seems that you have to reload your htb
> qdisc, class and filter rules after updating the kernel htb_rate_est
> parameter.