HTB precision and bursts

HTB precision and bursts

[Ivan Lezhnjov Jr.]

Hello,

I was wondering if the community would be able to help me understand something about HTB and what to expect of its precision.

The configuration that I work with is a Linux laptop using only one wireless interface to communicate with a typical SOHO Cisco Linksys Router.
You can see the actual HTB configuration below and what Linux kernel i use. Let me know if you need any other information to better understand my system.

My Internet connection is asymmetric ADSL, 10Mbit/s down and 1Mbit/s up.

The problem. 

When egress is saturated fully ingress gets sluggish. 
Really annoying, so I try to cap egress according to what appears to be a common knowledge and best practice.

As it is a 1Mbit/s egress, I attempt to limit outgoing traffic at 75kbps. This is what I figured is a good value that doesn't hinder ingress traffic to the point where it start being sluggish.
I know that it gets sluggish because I use a torrent client almost always seeding distro ISOs and whatnot, and I just know from experience that when I use torrent client shaping capabilities capping outgoing traffic at 75kbps there's left enough room for comfortable work. 

So, I'm particularly interested in why transmission can be very precise, and if I limit outbound traffic in it to 75kbps it never goes above that value. However, if use HTB as shown below and tell transmission to get out of the way in aspect of shaping traffic, I can see that on average it is limited to 75kbps indeed but I can see intermittent bursts up to 150kbps.

You could say transmission is stupid and is showing wrong numbers in GUI, but having worked with this particular configuration on a daily basis for years literally I can really tell that when I'm using HTB these bursts can and actually do cripple ingress.

Transmission's internal shaping capabilities proved way more accurate and reliable, so I'd say what it shows in GUI is in fact accurate.

However, limiting egress with the help of transmission isn't cool enough anymore :P Seriously, my home network grows more and more complex with time and there's a real need to use a more advanced shaping solution on this Linux laptop.

So, is there a way to tune HTB to be uncompromisingly precise with no bursts above set value of 75kbps?
If yes, how would I do it?


% uname -a
Linux sega 3.4.2-2-ARCH #1 SMP PREEMPT Tue Jun 12 07:19:42 UTC 2012 i686 GNU/Linux


% tc -s -p qdisc show dev wlan0
qdisc htb 1: root refcnt 5 r2q 10 default 10 direct_packets_stat 0
Sent 55339309 bytes 68604 pkt (dropped 0, overlimits 32068 requeues 83) 
backlog 0b 0p requeues 83 
% tc -s -p class show dev wlan0
class htb 1:11 root prio 0 rate 54000Kbit ceil 54000Kbit burst 1593b cburst 1593b 
Sent 50820347 bytes 49530 pkt (dropped 0, overlimits 0 requeues 0) 
rate 912bit 1pps backlog 0b 0p requeues 0 
lended: 49530 borrowed: 0 giants: 0
tokens: 3469 ctokens: 3469

class htb 1:10 root prio 0 rate 600000bit ceil 600000bit burst 1599b cburst 1599b 
Sent 4520878 bytes 19093 pkt (dropped 0, overlimits 0 requeues 0) 
rate 6648bit 3pps backlog 0b 0p requeues 0 
lended: 19093 borrowed: 0 giants: 0
tokens: 241234 ctokens: 241234

% tc -s -p filter show dev wlan0
filter parent 1: protocol ip pref 1 u32 
filter parent 1: protocol ip pref 1 u32 fh 800: ht divisor 1 
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:11 
 match IP src xxx.xxx.xxx.3/32
filter parent 1: protocol ip pref 1 u32 fh 800::801 order 2049 key ht 800 bkt 0 flowid 1:11 
 match IP dst xxx.xxx.xxx.3/32
filter parent 1: protocol ip pref 1 u32 fh 800::802 order 2050 key ht 800 bkt 0 flowid 1:11 
 match IP src xxx.xxx.xxx.4/32
filter parent 1: protocol ip pref 1 u32 fh 800::803 order 2051 key ht 800 bkt 0 flowid 1:11 
 match IP dst xxx.xxx.xxx.4/32
filter parent 1: protocol ip pref 1 u32 fh 800::804 order 2052 key ht 800 bkt 0 flowid 1:11 
 match IP src xxx.xxx.xxx.5/32
filter parent 1: protocol ip pref 1 u32 fh 800::805 order 2053 key ht 800 bkt 0 flowid 1:11 
 match IP dst xxx.xxx.xxx.5/32
filter parent 1: protocol ip pref 1 u32 fh 800::806 order 2054 key ht 800 bkt 0 flowid 1:11 
 match IP src xxx.xxx.xxx.6/32
filter parent 1: protocol ip pref 1 u32 fh 800::807 order 2055 key ht 800 bkt 0 flowid 1:11 
 match IP dst xxx.xxx.xxx.6/32


and this is how I get it configured via an init script

$tc qdisc del root dev $dev

	$tc qdisc add dev $dev root handle 1: htb default 10

	$tc class add dev $dev parent 1: classid 1:10 htb rate 75kbps
	$tc class add dev $dev parent 1: classid 1:11 htb rate 54mbit

	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip src xxx.xxx.xxx.xxx flowid 1:11
	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip dst xxx.xxx.xxx.xxx flowid 1:11
	
	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip src xxx.xxx.xxx.xxx flowid 1:11
	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip dst xxx.xxx.xxx.xxx flowid 1:11

	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip src xxx.xxx.xxx.xxx flowid 1:11
	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip dst xxx.xxx.xxx.xxx flowid 1:11
	
	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip src xxx.xxx.xxx.xxx flowid 1:11
	$tc filter add dev $dev protocol ip parent 1:0 prio 1 u32 \
	 match ip dst xxx.xxx.xxx.xxx flowid 1:11xxx.xxx.xxx.xxx

I have to conceal IPs, but the idea is that the Linux laptop talks to a few selected machines on the network via class 11 54mbit link, and everything else (the Internet) is capped at 75kbps, which falls into default class 10.

Ivan

Re: HTB precision and bursts

[Andrew Beverley]

On Sat, 2013-02-09 at 08:23 +0200, Ivan Lezhnjov Jr. wrote:
[...]
> The problem. 
> 
> When egress is saturated fully ingress gets sluggish. 
[...]
> As it is a 1Mbit/s egress, I attempt to limit outgoing traffic at
> 75kbps.

If all you want to do is ensure that downloads don't suffer when the
uplink is congested, then you could try prioritising ACK packets within
the uplink, instead of rate limiting the whole link.

I've only ever done this using a high-priority HTB class, but if you
don't want to shape any other type of traffic then a PRIO qdisc might
work for you (filter the ACK packets into the high-priority class and
all other packets into a lower priority one).

> So, is there a way to tune HTB to be uncompromisingly precise with no
> bursts above set value of 75kbps? If yes, how would I do it?

Have you tried playing with the burst/cburst values?

http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm#burst

Andy

Re: HTB precision and bursts

[Andy Furniss]

Ivan Lezhnjov Jr. wrote:
> Hello,

Hi, apologies for lack of examples - I am rusty with this stuff and 
don't have time to test everything so I am just going to make some 
critical points.

>
> I was wondering if the community would be able to help me understand something about HTB and what to expect of its precision.
>
> The configuration that I work with is a Linux laptop using only one wireless interface to communicate with a typical SOHO Cisco Linksys Router.
> You can see the actual HTB configuration below and what Linux kernel i use. Let me know if you need any other information to better understand my system.

It would be much nicer if you could get open wrt on the router or have 
something in front of it so that it saw all network traffic.

>
> My Internet connection is asymmetric ADSL, 10Mbit/s down and 1Mbit/s up.

If you try hard enough you should be able to use almost 100% upstream of 
ADSL as there are ways to allow for overheads, though you need to know 
exact detail of the way you are connected and sync speed (which may of 
course vary a bit).

>
> The problem.
>
> When egress is saturated fully ingress gets sluggish.
> Really annoying, so I try to cap egress according to what appears to be a common knowledge and best practice.
>
> As it is a 1Mbit/s egress, I attempt to limit outgoing traffic at 75kbps. This is what I figured is a good value that doesn't hinder ingress traffic to the point where it start being sluggish.
> I know that it gets sluggish because I use a torrent client almost always seeding distro ISOs and whatnot, and I just know from experience that when I use torrent client shaping capabilities capping outgoing traffic at 75kbps there's left enough room for comfortable work.
>
> So, I'm particularly interested in why transmission can be very precise, and if I limit outbound traffic in it to 75kbps it never goes above that value. However, if use HTB as shown below and tell transmission to get out of the way in aspect of shaping traffic, I can see that on average it is limited to 75kbps indeed but I can see intermittent bursts up to 150kbps.

Transmission doesn't know what's happening on the wire and since 
150kbyte is impossible for you it must be measuring traffic that is 
getting buffered.

>
> You could say transmission is stupid and is showing wrong numbers in GUI, but having worked with this particular configuration on a daily basis for years literally I can really tell that when I'm using HTB these bursts can and actually do cripple ingress.
>

The problem is your htb setup is not any better than nothing as you are 
just sending everything to a possibly large fifo (as you don't specify 
it will get whatever qlen is on the wlan if).

To maximise downloads/interactivity when upstream is saturated needs you 
to give priority to small udp for dns and small tcp for acks over the 
larger bulk data packets.

This involves filtering and a more complicated htb setup - an easier 
option if you are not trying to go for best possible latency for gaming 
would be to put sfq or fq_codel on the internet class.


> Transmission's internal shaping capabilities proved way more accurate and reliable, so I'd say what it shows in GUI is in fact accurate.
>
> However, limiting egress with the help of transmission isn't cool enough anymore :P Seriously, my home network grows more and more complex with time and there's a real need to use a more advanced shaping solution on this Linux laptop.
>
> So, is there a way to tune HTB to be uncompromisingly precise with no bursts above set value of 75kbps?
> If yes, how would I do it?

IME htb won't go over - I shape on adsl at almost 100%, depending on 
setup/use case it may spit out more than one bulk packet occasionally, 
but it still won't go over.


> and this is how I get it configured via an init script
>
> $tc qdisc del root dev $dev
>
> 	$tc qdisc add dev $dev root handle 1: htb default 10

The problem with default is that arp will go here I prefer to filter 
"the rest" with a "protocol ip" filter and not use default. If you have 
ipv6 etc you need to handle this. direct_packets_stat tells you what has 
gone through without shaping.


> 	$tc class add dev $dev parent 1: classid 1:10 htb rate 75kbps
> 	$tc class add dev $dev parent 1: classid 1:11 htb rate 54mbit

Being single duplex and having many overheads mean this 54mbit is not 
going to do anything - you won't get anything like that.

I have no experience trying to shape wireless.