[LARTC] RE: http bandwidth control

[LARTC] RE: http bandwidth control

[Guillermo Gomez]

Thanx to everyone who gave me the pointers.(Joel, Jason)

I tried several times to get into the HOWTO but it's not easy to follow
and i'm really busy so the squid solution seems to be simpler for me
(delay pools).

Anyway, is there any spanish translation of the HOWTO ? If not i could
try to help on this on my spare time.

My general config is:

lan (eth0) - linux gw - wan (eth1) (internet)

My linux gateway is doing NAT/PPTP VPN/SMTP Proxy (Messagewall)
and general firewalling.

The major issue i have is giving incoming priority on VPN clients and
slowing down incoming email traffic (huge).

Messagewall is limiting the amount of simultaneous smtp sessions but it
does not limit the bandwidth in any sense.

So basically my requirements are
	slow down http incoming traffic (general browsing, achievable with
squid)
	slow down smtp incoming traffic
	prioritize VPN PPTP traffic

Basically my upstream bandwidth is not congested but the downstream is
really congested.

I'm trying the wondershaper as a quick solution also but don't know how
"see if it's working or not"... My internet link is 1Mbps(up/down) and
i'm seen it very, very busy all the time. Also don't know how to list
several low prio src or dst ports... I tried 80,25 or 80:25... nothing
worked... any clue on this ?

Thanx again for the help
-- 
Guillermo Gomez <ggomez@neotechgw.net>
neotech

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] RE: http bandwidth control

[Ed Wildgoose]

>The major issue i have is giving incoming priority on VPN clients and
>slowing down incoming email traffic (huge).
>  
>
...

>I'm trying the wondershaper as a quick solution also but don't know how
>"see if it's working or not"... 
>

I would recommend this script as a better starting point
http://digriz.org.uk/jdg-qos-script/

Andy has some other ideas, that perhaps he will post?  However, in your 
case you want to look at the incoming part.  At the moment there is an 
HTB qdisc with an RED queue on it.  I found good results by copying that 
chunk of code and making a 1:22 queue, changing the iptables stuff to 
filter to that one by default and the original queue only for high 
priority incoming (perhaps you could even go further and setup lots of 
incoming for different priorities). 

You then just tweak the ipfilter stuff below to apply appropriate fwmark 
options and then things will end up in the appropriate buckets and be 
rate limited.

You can use the option "pollbuckets" on this script to see whether it's 
working or not.

The key point is that it's hard to control incoming.  All you can really 
do is drop packets.  However, the *idea* of RED is to proactively drop a 
few to try and slow rates down before queing starts.  It is debatable 
whether it works and some people think it may work better to avoid the 
RED altogether...

Of course you can also only queue on an outgoing interface, so you 
either need to have a bridge/router setup so that stuff for your local 
net is effectively "going out" on the local net card.  Or else you use 
the IMQ device to act as a kind of device in front of your normal 
incoming card so that you now have an outbound interface on that (which 
is effectively inbound to your normal internet facing card) - does that 
make sense?  IMQ is like sticking another device in front of your 
existing device?  Anyway, it does what you want.

Ed W
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] RE: http bandwidth control

[Guillermo Gomez]

THANX Ed

Give me sometime to understand what u said !
I went into the Howto and started reading all over.
Discovered what imq devices are, and remembered what ESFQ was.
Also went to the http://digriz.org.uk/jdg-qos-script/ ans started
studying it.

My kernel is  2.4.18-14 (RH8) and planing to upgrade to FC1 (not yet
confident with FC2).

How can i know if both IMQ and ESFQ is available in my actual kernel?

Regards
Guillermo

On Thu, 2004-06-24 at 07:07, Ed Wildgoose wrote:
> >The major issue i have is giving incoming priority on VPN clients and
> >slowing down incoming email traffic (huge).
> >  
> >
> ...
> 
> >I'm trying the wondershaper as a quick solution also but don't know how
> >"see if it's working or not"... 
> >
> 
> I would recommend this script as a better starting point
> http://digriz.org.uk/jdg-qos-script/
> 
> Andy has some other ideas, that perhaps he will post?  However, in your 
> case you want to look at the incoming part.  At the moment there is an 
> HTB qdisc with an RED queue on it.  I found good results by copying that 
> chunk of code and making a 1:22 queue, changing the iptables stuff to 
> filter to that one by default and the original queue only for high 
> priority incoming (perhaps you could even go further and setup lots of 
> incoming for different priorities). 
> 
> You then just tweak the ipfilter stuff below to apply appropriate fwmark 
> options and then things will end up in the appropriate buckets and be 
> rate limited.
> 
> You can use the option "pollbuckets" on this script to see whether it's 
> working or not.
> 
> The key point is that it's hard to control incoming.  All you can really 
> do is drop packets.  However, the *idea* of RED is to proactively drop a 
> few to try and slow rates down before queing starts.  It is debatable 
> whether it works and some people think it may work better to avoid the 
> RED altogether...
> 
> Of course you can also only queue on an outgoing interface, so you 
> either need to have a bridge/router setup so that stuff for your local 
> net is effectively "going out" on the local net card.  Or else you use 
> the IMQ device to act as a kind of device in front of your normal 
> incoming card so that you now have an outbound interface on that (which 
> is effectively inbound to your normal internet facing card) - does that 
> make sense?  IMQ is like sticking another device in front of your 
> existing device?  Anyway, it does what you want.
> 
> Ed W
-- 
Guillermo Gomez <ggomez@neotechgw.net>
neotech

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] RE: http bandwidth control

[Ed Wildgoose]

>Give me sometime to understand what u said !
>I went into the Howto and started reading all over.
>Discovered what imq devices are, and remembered what ESFQ was.
>Also went to the http://digriz.org.uk/jdg-qos-script/ ans started
>studying it.
>  
>

You just have to plough through the Howto.  Basically though these 
things classify the *outgoing* data into a couple of queues, then they 
release the packets at a programmable maximum rate.  You can either do 
the filtering in "tc", or just do it in iptables (more flexible).

The key thing is that its outgoing stuff only, so you either do it on a 
machine that is routing (cos then the other interface will effectively 
be your "out" one).  Or you use IMQ on the incoming interface, which 
simulates having something just in front of the normal interface (hence 
you can attach outgoing filters to it).

The various qdiscs differ in implementation and number of params, but 
that is the basic idea.  HTB is the simplest because you just create a 
couple of queues and set rates on them - everything else is done in 
iptables.

>My kernel is  2.4.18-14 (RH8) and planing to upgrade to FC1 (not yet
>confident with FC2).
>
>How can i know if both IMQ and ESFQ is available in my actual kernel?
>  
>

Hmm, well the script will error if you haven't.  Redhat has a lot of 
stuff compiled in, but if not then you are going to have a lot of 
compiling ahead.  You need a new kernel, new tc, and each will need 
patches because neither of these things is mainstream yet....  It's not 
hard if you have done this before (takes 20 mins), but if you haven't 
then it's going to need you to set an afternoon aside to learn how to do 
it (there are dozens of sites on how to compile the kernel)

Good luck

Ed W

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] RE: http bandwidth control

[Guillermo Gomez]

Thanx for the wishes :D

Yes i'm used to compile kernels (but don't patching)...

I did not try yet to use the new script since in fact i don't believe i
have IMQ neither ESFQ and my box is doing it's work ok with more than
400 users... don't wan to mess with it. It's congested but it's working.

The good news is that my box is acting as a routing and i could use my
lan (eth0) conn to control the incoming traffic from the internet (eth1)
without the help of IMQ.

So my requirments would be to slow down outgoing http, ftp and smtp.
Also would need to prioritize VPN clients. Affortunately all my VPN
clients have fixed IP addresses so i can mark the packets after the PPTP
de-encapsulation (going out through eth0).

Now it looks easier than before... (until i do the update).

My idea is to use HTBF with some leafs:
	1.- http && ftp  (bulky non priority traffic)
	2.- smtp (bulky but high prio)
	3.- vpn/pptp (bulky and high prio)

I would like to keep the response time low for ssh because of the admin
tasks are majorly achieved through it.

Is it best to use tc filters or iptables packet marking ?
For me it's easier iptables...

Any further suggestion ?

Guillermo

On Thu, 2004-06-24 at 16:40, Ed Wildgoose wrote:
> >Give me sometime to understand what u said !
> >I went into the Howto and started reading all over.
> >Discovered what imq devices are, and remembered what ESFQ was.
> >Also went to the http://digriz.org.uk/jdg-qos-script/ ans started
> >studying it.
> >  
> >
> 
> You just have to plough through the Howto.  Basically though these 
> things classify the *outgoing* data into a couple of queues, then they 
> release the packets at a programmable maximum rate.  You can either do 
> the filtering in "tc", or just do it in iptables (more flexible).
> 
> The key thing is that its outgoing stuff only, so you either do it on a 
> machine that is routing (cos then the other interface will effectively 
> be your "out" one).  Or you use IMQ on the incoming interface, which 
> simulates having something just in front of the normal interface (hence 
> you can attach outgoing filters to it).
> 
> The various qdiscs differ in implementation and number of params, but 
> that is the basic idea.  HTB is the simplest because you just create a 
> couple of queues and set rates on them - everything else is done in 
> iptables.
> 
> >My kernel is  2.4.18-14 (RH8) and planing to upgrade to FC1 (not yet
> >confident with FC2).
> >
> >How can i know if both IMQ and ESFQ is available in my actual kernel?
> >  
> >
> 
> Hmm, well the script will error if you haven't.  Redhat has a lot of 
> stuff compiled in, but if not then you are going to have a lot of 
> compiling ahead.  You need a new kernel, new tc, and each will need 
> patches because neither of these things is mainstream yet....  It's not 
> hard if you have done this before (takes 20 mins), but if you haven't 
> then it's going to need you to set an afternoon aside to learn how to do 
> it (there are dozens of sites on how to compile the kernel)
> 
> Good luck
> 
> Ed W
-- 
Guillermo Gomez <ggomez@neotechgw.net>
neotech

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/