[LARTC] strange htb behaviour

[LARTC] strange htb behaviour

[Tobias Geiger]

Hello List,

I use HTB in a ~80 users serving router with kernel 2.4.20.

eth0 is the internet interface (half-duplex 1Mbit),
eth1 the internal interface (100Mbit full duplex)

cause of the fact that the router also serves as a http-proxy (squid) i
thought setting up the default class on eth1 with the real upload-speed
(100Mbit) would do the job, i.e. shaping normal stuff to 1Mbit except
local generated proxy-traffic which should be served with the full
100Mbit. All other, forwarded, traffic is marked and so filtered to
other classes than the default.
To make it short: It works. when i download s.th. directly from the
server i get the full bw BUT: all other downloads from the internet
absolutly break down and don"t regenerate after some time. i have to
restart the qdiscs! Also i get these strange syslog messages:

Jan  6 06:39:05 Q kernel: htb*c20007 m=2 ty270 cƒ11 pq=0 df@9600
ql=0 pa=0 f:
Jan  6 06:39:05 Q kernel: htb*c20008 m=1 t=-59999999 cf950
pq\x126192148 df\x16359424 ql" pa\x10 f:
Jan  6 06:39:10 Q kernel: NET: 77 messages suppressed.
Jan  6 06:39:10 Q kernel: HTB: mindelayP0, report it please !
Jan  6 06:39:10 Q kernel: htb*g j\x126187316
Jan  6 06:39:10 Q kernel: htb*r7 m=0
Jan  6 06:39:10 Q kernel: htb*r6 m=0

later also:

Jan  6 06:40:05 Q kernel: NET: 518 messages suppressed.
Jan  6 06:40:05 Q kernel: HTB: suspicious delay in wait_tree
d=-1644459092 cl 008 h=1
Jan  6 06:40:10 Q kernel: NET: 518 messages suppressed.
Jan  6 06:40:10 Q kernel: HTB: suspicious delay in wait_tree
d=-1644459092 cl 008 h=1


I can't really imagine what causes this strange behaviour, except
perhaps the r2q/quantum settings with which i played around a little
bit, but just because the defaults caused also warning-messages in
syslog. perhaps these r2q/quantum parameters need tuning ?!
I'll attach my shell-script so you can look at it yourself,
thank you very much for any hint.

Greetings

Tobias




tc qdisc  add dev eth0 root handle 2:0 htb r2q 100 default 3
tc class  add dev eth0 parent 2:0 classid 2:1 htb rate 128kbit ceil
256kbit quantum 1500 burst 30k cburst 50k
tc class  add dev eth0 parent 2:1 classid 2:3 htb rate 1bps ceil 256kbit
prio 3 quantum 1500
tc qdisc  add dev eth0 parent 2:3 handle 3:0 sfq
# then a class for acks , maxiumum prio, but shouldnt eat up more than
1/3 of bw)
tc class  add dev eth0 parent 2:1 classid 2:4 htb rate 12kbit ceil
85kbit prio 0 quantum 1500 burst 5k
tc qdisc  add dev eth0 parent 2:4 handle 4:0 sfq
tc filter add dev eth0 parent 2:0 protocol ip prio 0 handle 0x1869f fw
classid 2:4
# and a class for dns/other stuff which should be served fast
tc class  add dev eth0 parent 2:1 classid 2:5 htb rate 128kbit prio 1
quantum 1500 burst 10k
tc qdisc  add dev eth0 parent 2:5 handle 5:0  sfq
tc filter add dev eth0 parent 2:0 protocol ip prio 1 handle 0x1869d fw
classid 2:5
# games-class: also shouldn exceed 1/3
tc class  add dev eth0 parent 2:1 classid 2:6 htb rate 128kbit prio 2
quantum 1500 burst 30k
tc qdisc  add dev eth0 parent 2:6 handle 6:0  sfq
tc filter add dev eth0 parent 2:0 protocol ip prio 2 handle 0x1869c fw
classid 2:6
# we guarantee here 80%
tc class  add dev eth0 parent 2:1 classid 2:7 htb rate 102kbit ceil
128kbit prio 3 quantum 1500 burst 10k
tc qdisc  add dev eth0 parent 2:7 handle 7:0  sfq
tc filter add dev eth0 parent 2:0 protocol ip prio 3 handle 0x1869b fw
classid 2:7
# bulk-class. lower prio than all others, no reserved bw
tc class  add dev eth0 parent 2:1 classid 2:8 htb rate 1bps ceil 256kbit
prio 4 quantum 1500 burst 5k cburst 10k
tc qdisc  add dev eth0 parent 2:8 handle 8:0  sfq
tc filter add dev eth0 parent 2:0 protocol ip prio 4 handle 0x1869a fw
classid 2:8

tc qdisc  add dev eth1 root handle 2:0 htb r2q 100 default 3
tc class  add dev eth1 parent 2:0 classid 2:1 htb rate 1024kbit ceil
100240kbit quantum 1500 burst 30k cburst 50k
tc class  add dev eth1 parent 2:1 classid 2:3 htb rate 1bps ceil
100240kbit prio 3 quantum 1500
tc qdisc  add dev eth1 parent 2:3 handle 3:0 sfq
# then a class for acks , maxiumum prio, but shouldnt eat up more than
1/3 of bw)
tc class  add dev eth1 parent 2:1 classid 2:4 htb rate 102kbit ceil
512kbit prio 0 quantum 1500 burst 5k
tc qdisc  add dev eth1 parent 2:4 handle 4:0 sfq
tc filter add dev eth1 parent 2:0 protocol ip prio 0 handle 0x1869f fw
classid 2:4
# and a class for dns/other stuff which should be served fast
tc class  add dev eth1 parent 2:1 classid 2:5 htb rate 256kbit ceil
512kbit prio 1 quantum 1500 burst 10k
tc qdisc  add dev eth1 parent 2:5 handle 5:0  sfq
tc filter add dev eth1 parent 2:0 protocol ip prio 1 handle 0x1869d fw
classid 2:5
# games-class: also shouldn exceed 1/3
tc class  add dev eth1 parent 2:1 classid 2:6 htb rate 256kbit ceil
512kbit prio 2 quantum 1500 burst 30k
tc qdisc  add dev eth1 parent 2:6 handle 6:0  sfq
tc filter add dev eth1 parent 2:0 protocol ip prio 2 handle 0x1869c fw
classid 2:6
# we guarantee here 80% of the bw for normal traffic
tc class  add dev eth1 parent 2:1 classid 2:7 htb rate 819kbit ceil
80192kbit prio 3 quantum 1500 burst 10k
tc qdisc  add dev eth1 parent 2:7 handle 7:0  sfq
tc filter add dev eth1 parent 2:0 protocol ip prio 3 handle 0x1869b fw
classid 2:7
# bulk-class. lower prio than all others, no reserved bw
tc class  add dev eth1 parent 2:1 classid 2:8 htb rate 1bps ceil 972kbit
prio 4 quantum 1500 burst 5k cburst 10k
tc qdisc  add dev eth1 parent 2:8 handle 8:0  sfq
tc filter add dev eth1 parent 2:0 protocol ip prio 4 handle 0x1869a fw 
classid 2:8
# now iptables-rules
modprobe ip_conntrack
modprobe ip_conntrack_ftp
iptables -t mangle -A FORWARD -j MARK --set-mark 0x1869b
iptables -t mangle -A FORWARD -p udp --destination-port 53 -j MARK 
--set-mark 0x1869d
iptables -t mangle -A FORWARD -p udp --source-port 53 -j MARK --set-mark 
0x1869d
$iptables -t mangle -A FORWARD -p icmp -m limit --limit 2/second -j MARK 
--set-mark 0x1869d
iptables -t mangle -A FORWARD -p tcp -m state --state
ESTABLISHED,RELATED -m tos --tos 0x10 -j MARK --set-mark
0x1869d
iptables -t mangle -A FORWARD -p tcp -m state --state
ESTABLISHED,RELATED --dport 22 -m tos --tos ! 0x8  -j MARK --set-mark
0x1869d
iptables -t mangle -A FORWARD -p tcp -m state --state
ESTABLISHED,RELATED --sport 22 -m tos --tos ! 0x8  -j MARK --set-mark
0x1869d
iptables -t mangle -A FORWARD -p udp -m state --state
ESTABLISHED,RELATED --dport 14567 -j MARK --set-mark
0x1869c
iptables -t mangle -A FORWARD -p udp -m state --state
ESTABLISHED,RELATED --sport 14567 -j MARK --set-mark
0x1869c
iptables -t mangle -A FORWARD -p tcp -m state --state
ESTABLISHED,RELATED --dport 4000 -j MARK --set-mark
0x1869c
iptables -t mangle -A FORWARD -p udp -m state --state
ESTABLISHED,RELATED --dport 26000:28000 -j MARK --set-mark
0x1869c
iptables -t mangle -A FORWARD -p udp -m state --state
ESTABLISHED,RELATED --sport 26000:28000 -j MARK --set-mark
0x1869c
iptables -t mangle -A FORWARD -m helper --helper ftp -j MARK --set-mark 
0x1869a
iptables -t mangle -A FORWARD -p tcp -m state --state
ESTABLISHED,RELATED --destination-port 25 -j MARK --set-mark
0x1869a
iptables -t mangle -A FORWARD -p tcp --tcp-flags ALL ACK -m length
--length 40:52 -j MARK --set-mark 0x1869f



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] strange htb behaviour

[Stef Coene]

On Monday 06 January 2003 07:01, Tobias Geiger wrote:
> Hello List,
>
> I use HTB in a ~80 users serving router with kernel 2.4.20.
>
> eth0 is the internet interface (half-duplex 1Mbit),
> eth1 the internal interface (100Mbit full duplex)
>
> cause of the fact that the router also serves as a http-proxy (squid) i
> thought setting up the default class on eth1 with the real upload-speed
> (100Mbit) would do the job, i.e. shaping normal stuff to 1Mbit except
> local generated proxy-traffic which should be served with the full
> 100Mbit. All other, forwarded, traffic is marked and so filtered to
> other classes than the default.
> To make it short: It works. when i download s.th. directly from the
> server i get the full bw BUT: all other downloads from the internet
> absolutly break down and don"t regenerate after some time. i have to
> restart the qdiscs! Also i get these strange syslog messages:
>
> Jan  6 06:39:05 Q kernel: htb*c20007 m=2 ty270 cƒ11 pq=0 df@9600
> ql=0 pa=0 f:
> Jan  6 06:39:05 Q kernel: htb*c20008 m=1 t=-59999999 cf950
> pq\x126192148 df\x16359424 ql" pa\x10 f:
> Jan  6 06:39:10 Q kernel: NET: 77 messages suppressed.
> Jan  6 06:39:10 Q kernel: HTB: mindelayP0, report it please !
> Jan  6 06:39:10 Q kernel: htb*g j\x126187316
> Jan  6 06:39:10 Q kernel: htb*r7 m=0
> Jan  6 06:39:10 Q kernel: htb*r6 m=0
>
> later also:
>
> Jan  6 06:40:05 Q kernel: NET: 518 messages suppressed.
> Jan  6 06:40:05 Q kernel: HTB: suspicious delay in wait_tree
> d=-1644459092 cl 008 h=1
> Jan  6 06:40:10 Q kernel: NET: 518 messages suppressed.
> Jan  6 06:40:10 Q kernel: HTB: suspicious delay in wait_tree
> d=-1644459092 cl 008 h=1
>
>
> I can't really imagine what causes this strange behaviour, except
> perhaps the r2q/quantum settings with which i played around a little
> bit, but just because the defaults caused also warning-messages in
> syslog. perhaps these r2q/quantum parameters need tuning ?!
> I'll attach my shell-script so you can look at it yourself,
> thank you very much for any hint.
You are not the first one to notice this.  But I have no solution/answer.  
Maybe Devik does ?

Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/