* [LARTC] Problems with HTB (ceil being overpassed)
@ 2004-01-28 0:21 Gerardo Arceri
2004-01-28 2:20 ` rubens
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Gerardo Arceri @ 2004-01-28 0:21 UTC (permalink / raw)
To: lartc
We run a Hosting farm behind a bridge/iptables firewall setup running
Gentoo with kernel 2.4.20-gentoo-r6, connected to a dual 15Mbps
international internet pipe / , as this:
Net Pipe --------- eth1 Bridge/Firewall eth0 -------- Internal Hosting
Network
lately we have been looking at htb to somehow control excessive usage from
the users behind, but in our implementation there seems to be an error or
something wrong on the setup,
this is the test script i'm using, i know it's very rough but i think it
should do the work.
tc qdisc del dev eth1 root
tc qdisc add dev eth1 root handle 1: htb default 10
tc class add dev eth1 parent 1: classid 1:1 htb rate 98Mbit ceil 98Mbit
tc class add dev eth1 parent 1:1 classid 1:10 htb rate 90Mbit ceil 90Mbit
tc class add dev eth1 parent 1:1 classid 1:11 htb rate 2Mbit ceil 2Mbit
tc class add dev eth1 parent 1:1 classid 1:12 htb rate 4Mbit ceil 4Mbit
tc filter add dev eth1 protocol ip parent 1:0 prio 1 u32 match ip src
$server_ip flowid 1:11
I intend to limit $server_ip to 2Mbit max traffic ow, the problem is after
i run the script htb seems to ignore the limit and traffic for the client
stays in over 3mbit.
but after a while of running with the htb active the server owner
complains that the loading times of pages hosted on the server skyrocket
and that ssh access becomes sluggish.
Normally that server has about 4/5 Mbit/s of outgoing traffic measured by
the iptables/mrtg script, doing a:
#tc -s -d class show dev eth1
shows:
class htb 1:11 parent 1:1 prio 0 quantum 26214 rate 2Mbit ceil 2Mbit burst
2621b/8 mpu 0b cburst 2621b/8 mpu 0b level 0
Sent 23592359 bytes 26524 pkts (dropped 1579, overlimits 0)
rate 315631bps 352pps backlog 96p
lended: 26428 borrowed: 0 giants: 0
tokens: -3 ctokens: -3
class htb 1:1 root rate 98Mbit ceil 98Mbit burst 64212b/8 mpu 0b cburst
64212b/8 mpu 0b level 7
Sent 66766024 bytes 97843 pkts (dropped 0, overlimits 0)
rate 889284bps 1291pps
lended: 0 borrowed: 0 giants: 0
tokens: 1 ctokens: 1
class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 90Mbit ceil 90Mbit
burst 58970b/8 mpu 0b cburst 58970b/8 mpu 0b level 0
Sent 43271713 bytes 71415 pkts (dropped 0, overlimits 0)
rate 573411bps 938pps
lended: 71415 borrowed: 0 giants: 0
tokens: 1 ctokens: 1
class htb 1:12 parent 1:1 prio 0 quantum 52428 rate 4Mbit ceil 4Mbit burst
2620b/8 mpu 0b cburst 2620b/8 mpu 0b level 0
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
lended: 0 borrowed: 0 giants: 0
tokens: 1 ctokens: 1
Showing trafic in excess of 3.5 Mbit/s and coinciding with a mrtg graphic.
From my limited experience i would say that somehow my mrtg is measuring
traffic well before it passes thru htb (which seems imposible from what
i've read). i take the measurement on the
iptables FORWARD chain:
iptables -N $server_ip-in
iptables -N $server_ip-out
iptables -A $server_ip-in -j RETURN
iptables -A $server_ip-out -j RETURN
iptables -A FORWARD -s $server_ip -j $server_ip-out
iptables -A FORWARD -d $server_ip -j $server_ip-in
and to make the actual measurement:
iptables -nvxL $server_ip-in
iptables -nvxL $server_ip-out
Resuming, how can i effectively test if and how well htb it's doing the
job ?
Help will be appreciated.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [LARTC] Problems with HTB (ceil being overpassed)
2004-01-28 0:21 [LARTC] Problems with HTB (ceil being overpassed) Gerardo Arceri
@ 2004-01-28 2:20 ` rubens
2004-01-29 20:41 ` Gerardo Arceri
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: rubens @ 2004-01-28 2:20 UTC (permalink / raw)
To: lartc
It seems you have hit timer innacuracy issues:
http://www.docum.org/stef.coene/qos/faq/cache/40.html
Rubens
On Tue, 27 Jan 2004, Gerardo Arceri wrote:
> We run a Hosting farm behind a bridge/iptables firewall setup running
> Gentoo with kernel 2.4.20-gentoo-r6, connected to a dual 15Mbps
> international internet pipe / , as this:
>
> Net Pipe --------- eth1 Bridge/Firewall eth0 -------- Internal Hosting
> Network
>
> lately we have been looking at htb to somehow control excessive usage from
> the users behind, but in our implementation there seems to be an error or
> something wrong on the setup,
> this is the test script i'm using, i know it's very rough but i think it
> should do the work.
>
> tc qdisc del dev eth1 root
> tc qdisc add dev eth1 root handle 1: htb default 10
> tc class add dev eth1 parent 1: classid 1:1 htb rate 98Mbit ceil 98Mbit
> tc class add dev eth1 parent 1:1 classid 1:10 htb rate 90Mbit ceil 90Mbit
> tc class add dev eth1 parent 1:1 classid 1:11 htb rate 2Mbit ceil 2Mbit
> tc class add dev eth1 parent 1:1 classid 1:12 htb rate 4Mbit ceil 4Mbit
> tc filter add dev eth1 protocol ip parent 1:0 prio 1 u32 match ip src
> $server_ip flowid 1:11
>
> I intend to limit $server_ip to 2Mbit max traffic ow, the problem is after
> i run the script htb seems to ignore the limit and traffic for the client
> stays in over 3mbit.
> but after a while of running with the htb active the server owner
> complains that the loading times of pages hosted on the server skyrocket
> and that ssh access becomes sluggish.
> Normally that server has about 4/5 Mbit/s of outgoing traffic measured by
> the iptables/mrtg script, doing a:
> #tc -s -d class show dev eth1
> shows:
>
> class htb 1:11 parent 1:1 prio 0 quantum 26214 rate 2Mbit ceil 2Mbit burst
> 2621b/8 mpu 0b cburst 2621b/8 mpu 0b level 0
> Sent 23592359 bytes 26524 pkts (dropped 1579, overlimits 0)
> rate 315631bps 352pps backlog 96p
> lended: 26428 borrowed: 0 giants: 0
> tokens: -3 ctokens: -3
>
> class htb 1:1 root rate 98Mbit ceil 98Mbit burst 64212b/8 mpu 0b cburst
> 64212b/8 mpu 0b level 7
> Sent 66766024 bytes 97843 pkts (dropped 0, overlimits 0)
> rate 889284bps 1291pps
> lended: 0 borrowed: 0 giants: 0
> tokens: 1 ctokens: 1
>
> class htb 1:10 parent 1:1 prio 0 quantum 200000 rate 90Mbit ceil 90Mbit
> burst 58970b/8 mpu 0b cburst 58970b/8 mpu 0b level 0
> Sent 43271713 bytes 71415 pkts (dropped 0, overlimits 0)
> rate 573411bps 938pps
> lended: 71415 borrowed: 0 giants: 0
> tokens: 1 ctokens: 1
>
> class htb 1:12 parent 1:1 prio 0 quantum 52428 rate 4Mbit ceil 4Mbit burst
> 2620b/8 mpu 0b cburst 2620b/8 mpu 0b level 0
> Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> lended: 0 borrowed: 0 giants: 0
> tokens: 1 ctokens: 1
>
> Showing trafic in excess of 3.5 Mbit/s and coinciding with a mrtg graphic.
>
> From my limited experience i would say that somehow my mrtg is measuring
> traffic well before it passes thru htb (which seems imposible from what
> i've read). i take the measurement on the
> iptables FORWARD chain:
>
> iptables -N $server_ip-in
> iptables -N $server_ip-out
> iptables -A $server_ip-in -j RETURN
> iptables -A $server_ip-out -j RETURN
> iptables -A FORWARD -s $server_ip -j $server_ip-out
> iptables -A FORWARD -d $server_ip -j $server_ip-in
>
> and to make the actual measurement:
> iptables -nvxL $server_ip-in
> iptables -nvxL $server_ip-out
>
> Resuming, how can i effectively test if and how well htb it's doing the
> job ?
>
>
> Help will be appreciated.
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [LARTC] Problems with HTB (ceil being overpassed)
2004-01-28 0:21 [LARTC] Problems with HTB (ceil being overpassed) Gerardo Arceri
2004-01-28 2:20 ` rubens
@ 2004-01-29 20:41 ` Gerardo Arceri
2004-01-29 21:07 ` Gerardo Arceri
2004-02-03 23:21 ` miller69
3 siblings, 0 replies; 5+ messages in thread
From: Gerardo Arceri @ 2004-01-29 20:41 UTC (permalink / raw)
To: lartc
You were right thanks for the tip, i suspected there was some clock issue
involved, now it works perfectly, 2500Kbit limit is not passed by a single
byte... superb, thanks again.!
On Wed, 28 Jan 2004 00:20:13 -0200 (BRST), <rubens@etica.net> wrote:
>
> It seems you have hit timer innacuracy issues:
> http://www.docum.org/stef.coene/qos/faq/cache/40.html
>
> Rubens
>
>
> On Tue, 27 Jan 2004, Gerardo Arceri wrote:
>
>> We run a Hosting farm behind a bridge/iptables firewall setup running
>> Gentoo with kernel 2.4.20-gentoo-r6, connected to a dual 15Mbps
>> international internet pipe / , as this:
>>
>> Net Pipe --------- eth1 Bridge/Firewall eth0 -------- Internal Hosting
>> Network
>>
>> lately we have been looking at htb to somehow control excessive usage
>> from
>> the users behind, but in our implementation there seems to be an error
>> or
>> something wrong on the setup,
>> this is the test script i'm using, i know it's very rough but i think it
>> should do the work.
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [LARTC] Problems with HTB (ceil being overpassed)
2004-01-28 0:21 [LARTC] Problems with HTB (ceil being overpassed) Gerardo Arceri
2004-01-28 2:20 ` rubens
2004-01-29 20:41 ` Gerardo Arceri
@ 2004-01-29 21:07 ` Gerardo Arceri
2004-02-03 23:21 ` miller69
3 siblings, 0 replies; 5+ messages in thread
From: Gerardo Arceri @ 2004-01-29 21:07 UTC (permalink / raw)
To: lartc
>
> It seems you have hit timer innacuracy issues:
> http://www.docum.org/stef.coene/qos/faq/cache/40.html
>
Recompiled the kernel with the PSCHED_CPU modification and now
tc -s -d class show dev eth1 shows that the server is capped exactly at
312Kpbs (2500Kbit).
BUT... read below quote..
>> From my limited experience i would say that somehow my mrtg is
>> measuring
>> traffic well before it passes thru htb (which seems imposible from what
>> i've read). i take the measurement on the
>> iptables FORWARD chain:
>>
>> iptables -N $server_ip-in
>> iptables -N $server_ip-out
>> iptables -A $server_ip-in -j RETURN
>> iptables -A $server_ip-out -j RETURN
>> iptables -A FORWARD -s $server_ip -j $server_ip-out
>> iptables -A FORWARD -d $server_ip -j $server_ip-in
>>
>> and to make the actual measurement:
>> iptables -nvxL $server_ip-in
>> iptables -nvxL $server_ip-out
>>
That mrtg measurement still shows 412 Kbps, how could it be. doesn't seem
plausible that iptables reads the packets before the packet scheduler,
since it works at a lower level closer to actual hardware.
What's the explanation for this ?
Thanks in advance.!
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [LARTC] Problems with HTB (ceil being overpassed)
2004-01-28 0:21 [LARTC] Problems with HTB (ceil being overpassed) Gerardo Arceri
` (2 preceding siblings ...)
2004-01-29 21:07 ` Gerardo Arceri
@ 2004-02-03 23:21 ` miller69
3 siblings, 0 replies; 5+ messages in thread
From: miller69 @ 2004-02-03 23:21 UTC (permalink / raw)
To: lartc
> It seems you have hit timer innacuracy issues:
> http://www.docum.org/stef.coene/qos/faq/cache/40.html
Well, I've tried this on a vanilla 2.4.24 kernel but was not able to load
sched_htb anymore. The system was a P4 1700MHz - wich should support it. I'm
also experiencing HTB overlimiting as I describe here at the list a while ago.
Regards,
Mike.
--
GMX ProMail (250 MB Mailbox, 50 FreeSMS, Virenschutz, 2,99 EUR/Monat...)
jetzt 3 Monate GRATIS + 3x DER SPIEGEL +++ http://www.gmx.net/derspiegel +++
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2004-02-03 23:21 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-01-28 0:21 [LARTC] Problems with HTB (ceil being overpassed) Gerardo Arceri
2004-01-28 2:20 ` rubens
2004-01-29 20:41 ` Gerardo Arceri
2004-01-29 21:07 ` Gerardo Arceri
2004-02-03 23:21 ` miller69
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.