[LARTC] Help with bandwith control in a firewall/bridge machine

["Miguel Ángel Domínguez Durán" <mdominguez@cherrytel.com>]

Hello again,
First, excuse me for my poor english.
I'm trying now to make bandwith control in a firewall machine running 
Shorewall. This machine is also a bridge using bridge-utils 
bridge-utils-devel. It is a mandrake 10. The configuration is something like 
this:

FTP/Webserver ------|   eth0                                    eth1
Mailserver -------------|------BRIDGE/FIREWALL------Router-----Internet
DB App. server -------|

I have installed iptoute2 and all kernel options needed. I have stated 
TC_ENABLED = Yes and copied my own script in the tcstart file so shorewall 
should run it when it gets restarted. I don't get any errors when the script 
is executed, but all the packets go through the default queue in uplink and 
downlink when i analize the queues using .
I use the following script to start the bridge:

#!/bin/sh

set -x

#Activamos el puente:

brctl addbr br0

#Desactivamos el protocolo de spanning tree, posibles loops entre routers,

#como en nuestro caso solo lo vamos a conectar a un router no hace falta:

brctl stp br0 off

#A continuaciÛn aÒadimos las tarjetas de red al puente: (Ojo, una vez hecho

#esto perdemos la conectividad)

brctl addif br0 eth0

brctl addif br0 eth1

#Desactivamos las 2 tarjetas de red:

correo.cherrytel.comifconfig eth0 down

ifconfig eth1 down

#Las volvemos a activar pero sin IP definida

ifconfig eth0 0.0.0.0 up

ifconfig eth1 0.0.0.0 up

#Activamos el puente y le asignamos una IP:

ifconfig br0 213.9.139.6 up

#AÒadimos la ruta por defecto:

route add default gw 213.9.139.1

#Activamos el reenvio:

echo "1" > /proc/sys/net/ipv4/ip_forward

y aÒado la entrada al /etc/rc.local



The script in tcstart is:


#!/bin/bash
#
#

DEV1=eth0 #salida a red interna de Cherrytel
DEV0=eth1 #salida a internet
              # Note that this is significantly lower than the capacity of 
1500.
              # Because of this, you may not want to bother limiting inbound 
traffic
              # until a better implementation such as TCP window 
manipulation can be used.

#
# End Configuration Options
#

TC=/sbin/tc

if [ "$1" = "status" ]
then
        echo "Enlace descendente"
        echo "[qdisc]"
        $TC -s qdisc show dev $DEV1
        echo "[class]"
        $TC -s class show dev $DEV1
        echo "[filter]"
        $TC -s filter show dev $DEV1

        echo "Enlace ascendente"
        echo "[qdisc]"
        $TC -s qdisc show dev $DEV0
        echo "[class]"
        $TC -s class show dev $DEV0
        echo "[filter]"
        $TC -s filter show dev $DEV0

        exit
fi

# Reset everything to a known state (cleared)
$TC qdisc del dev $DEV0 root    2> /dev/null > /dev/null
$TC qdisc del dev $DEV1 root    2> /dev/null > /dev/null
iptables -t mangle -D POSTROUTING -o $DEV1 -j MYSHAPER-IN 2> /dev/null > 
/dev/null
iptables -t mangle -F MYSHAPER-OUT 2> /dev/null > /dev/null
iptables -t mangle -X MYSHAPER-OUT 2> /dev/null > /dev/null
iptables -t mangle -D PREROUTING -i $DEV0 -j MYSHAPER-OUT 2> /dev/null > 
/dev/null
iptables -t mangle -F MYSHAPER-IN 2> /dev/null > /dev/null
iptables -t mangle -X MYSHAPER-IN 2> /dev/null > /dev/null


if [ "$1" = "stop" ]
then
        echo "Shaping removed on $DEV1."
        echo "Shaping removed on $DEV0."
        exit
fi

###########################################################
#
# Inbound Shaping (limits total bandwidth to 850Kbps)
# Este es el enlace descendente, desde internet hacia la red interna de 
Cherrytel

# set queue size to give latency of about 2 seconds on low-prio packets
#ip link set dev $DEV1 qlen 30

# changes mtu on the outbound device.  Lowering the mtu will result
# in lower latency but will also cause slightly lower throughput due
# to IP and TCP protocol overhead.
#ip link set dev $DEV1 mtu 1000

# add HTB root qdisc
$TC qdisc add dev $DEV1 root handle 1: htb default 37

# add main rate limit classes
$TC class add dev $DEV1 parent 1: classid 1:1 htb rate 856kbit

# add leaf classes - We grant each class at LEAST it's "fair share" of 
bandwidth.
#                    this way no class will ever be starved by another 
class.  Each
#                    class is also permitted to consume all of the available 
bandwidth
#                    if no other classes are in use.
$TC class add dev $DEV1 parent 1:1 classid 1:20 htb rate 64kbit ceil 856kbit
$TC class add dev $DEV1 parent 1:1 classid 1:21 htb rate 64kbit ceil 856kbit
$TC class add dev $DEV1 parent 1:1 classid 1:22 htb rate 64kbit ceil 856kbit
$TC class add dev $DEV1 parent 1:1 classid 1:37 htb rate 600kbit ceil 
856kbit   #por defecto

$TC class add dev $DEV1 parent 1:1 classid 1:23 htb rate 64kbit ceil 856kbit 
#oficinas

# attach qdisc to leaf classes - here we at SFQ to each priority class.  SFQ 
insures that
#                                within each class connections will be 
treated (almost) fairly.
$TC qdisc add dev $DEV1 parent 1:20 handle 20: sfq perturb 10
$TC qdisc add dev $DEV1 parent 1:21 handle 21: sfq perturb 10
$TC qdisc add dev $DEV1 parent 1:22 handle 22: sfq perturb 10
$TC qdisc add dev $DEV1 parent 1:37 handle 37: sfq perturb 10

$TC qdisc add dev $DEV1 parent 1:23 handle 23: sfq perturb 10

# filter traffic into classes by fwmark - here we direct traffic into 
priority class according to
#                                         the fwmark set on the packet (we 
set fwmark with iptables
#                                         later).  Note that above we've set 
the default priority
#                                         class to 1:37 so unmarked packets 
(or packets marked with
#                                         unfamiliar IDs) will be defaulted 
to the lowest priority
#                                         class.
$TC filter add dev $DEV1 parent 1:0 prio 0 protocol ip handle 20 fw flowid 
1:20
$TC filter add dev $DEV1 parent 1:0 prio 0 protocol ip handle 21 fw flowid 
1:21
$TC filter add dev $DEV1 parent 1:0 prio 0 protocol ip handle 22 fw flowid 
1:22
$TC filter add dev $DEV1 parent 1:0 prio 0 protocol ip handle 23 fw flowid 
1:23

# Marking the packets.

 iptables -t mangle -N MYSHAPER-IN
 iptables -t mangle -I POSTROUTING -o $DEV1 -j MYSHAPER-IN

 iptables -A MYSHAPER-IN -d 213.9.139.30 -t mangle -j MARK --set-mark 20
 iptables -A MYSHAPER-IN -d 213.9.139.31 -t mangle -j MARK --set-mark 20
 iptables -A MYSHAPER-IN -d 213.9.139.32 -t mangle -j MARK --set-mark 20

 iptables -A MYSHAPER-IN -d 213.9.139.22 -t mangle -j MARK --set-mark 21
 iptables -A MYSHAPER-IN -d 213.9.139.71 -t mangle -j MARK --set-mark 21

 iptables -A MYSHAPER-IN -d 213.9.139.25 -t mangle -j MARK --set-mark 22

 iptables -A MYSHAPER-IN -d 213.9.139.24 -t mangle -j MARK --set-mark 23

# iptables -A MYSHAPER-IN -d 10.9.139.14 -t mangle -j MARK --set-mark 22

# iptables -A MYSHAPER-IN -d 10.9.139.13 -t mangle -j MARK --set-mark 22

#El resto de tráco iríal flujo por defecto, el 2:37.

# Done with inbound shaping
#
####################################################

echo "Control del enlace descendente activado."

#Si solo se desea controlar el enlace descendente, quitar el comentario de 
la siguiente instruccion exit
#exit




###########################################################
#
# Outbound Shaping (limits total bandwidth to 856Kbps)
# Este es el enlace ascendente, desde la red interna de Cherrytel a internet

# set queue size to give latency of about 2 seconds on low-prio packets
#ip link set dev $DEV0 qlen 30

# changes mtu on the outbound device.  Lowering the mtu will result
# in lower latency but will also cause slightly lower throughput due
# to IP and TCP protocol overhead.
#ip link set dev $DEV0 mtu 1000

# add HTB root qdisc
$TC qdisc add dev $DEV0 root handle 2: htb default 87

# add main rate limit classes
$TC class add dev $DEV0 parent 2: classid 2:1 htb rate 856kbit

# add leaf classes - We grant each class at LEAST it's "fair share" of 
bandwidth.
#                    this way no class will ever be starved by another 
class.  Each
#                    class is also permitted to consume all of the available 
bandwidth
#                    if no other classes are in use.
$TC class add dev $DEV0 parent 2:1 classid 2:70 htb rate 64kbit ceil 856kbit
$TC class add dev $DEV0 parent 2:1 classid 2:71 htb rate 64kbit ceil 856kbit
$TC class add dev $DEV0 parent 2:1 classid 2:72 htb rate 64kbit ceil 856kbit
$TC class add dev $DEV0 parent 2:1 classid 2:87 htb rate 600kbit ceil 
856kbit   #por defecto

$TC class add dev $DEV0 parent 2:1 classid 2:73 htb rate 64kbit ceil 856kbit 
#prueba

# attach qdisc to leaf classes - here we at SFQ to each priority class.  SFQ 
insures that
#                                within each class connections will be 
treated (almost) fairly.
$TC qdisc add dev $DEV0 parent 2:70 handle 70: sfq perturb 10
$TC qdisc add dev $DEV0 parent 2:71 handle 71: sfq perturb 10
$TC qdisc add dev $DEV0 parent 2:72 handle 72: sfq perturb 10
$TC qdisc add dev $DEV0 parent 2:87 handle 87: sfq perturb 10

$TC qdisc add dev $DEV0 parent 2:73 handle 73: sfq perturb 10

# filter traffic into classes by fwmark - here we direct traffic into 
priority class according to
#                                         the fwmark set on the packet (we 
set fwmark with iptables
#                                         later).  Note that above we've set 
the default priority
#                                         class to 1:87 so unmarked packets 
(or packets marked with
#                                         unfamiliar IDs) will be defaulted 
to the lowest priority
#                                         class.
$TC filter add dev $DEV0 parent 2:0 prio 0 protocol ip handle 70 fw flowid 
2:70
$TC filter add dev $DEV0 parent 2:0 prio 0 protocol ip handle 71 fw flowid 
2:71
$TC filter add dev $DEV0 parent 2:0 prio 0 protocol ip handle 72 fw flowid 
2:72
$TC filter add dev $DEV0 parent 2:0 prio 0 protocol ip handle 73 fw flowid 
2:73

# Marking the packets.

 iptables -t mangle -N MYSHAPER-OUT
 iptables -t mangle -I PREROUTING -i $DEV0 -j MYSHAPER-OUT

 iptables -A MYSHAPER-OUT -s 213.9.139.30 -t mangle -j MARK --set-mark 70
 iptables -A MYSHAPER-OUT -s 213.9.139.31 -t mangle -j MARK --set-mark 70
 iptables -A MYSHAPER-OUT -s 213.9.139.32 -t mangle -j MARK --set-mark 70

 iptables -A MYSHAPER-OUT -s 213.9.139.22 -t mangle -j MARK --set-mark 71
 iptables -A MYSHAPER-OUT -s 213.9.139.71 -t mangle -j MARK --set-mark 71

 iptables -A MYSHAPER-OUT -s 213.9.139.25 -t mangle -j MARK --set-mark 72

 iptables -A MYSHAPER-OUT -s 213.9.139.24 -t mangle -j MARK --set-mark 73

# iptables -A MYSHAPER-OUT -s 10.9.139.13 -t mangle -j MARK --set-mark 72

# iptables -A MYSHAPER-OUT -s 10.9.139.14 -t mangle -j MARK --set-mark 72


#El resto de tráco iríal flujo por defecto, el 2:87.


# Done with outbound shaping
#
####################################################

echo "Control del enlace ascendente activado."

exit

Thank you very much

UN CORDIAL SALUDO

Miguel ¡ngel DomÌnguez Dur·n.
Departamento TÈcnico.
Cherrytel Comunicaciones, S.L.
mdominguez@cherrytel.com
http://www.cherrytel.com/
Tlf. 902 115 673
Fax 952218170 

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/