[LARTC] Shaping traffic over a linux bridge

[LARTC] Shaping traffic over a linux bridge

[Hugh Buchanan]

Greetings,

I am using tc/cbq to do some traffic shaping over a linux bridge.  My
system is running the 2.4.20 kernel with the latest bridge-nf and
ebtables patches.  I am also running ebtables 2.0.3 and iptables 1.2.8.

Last week, since I had yet to discover ebtables/bridging I was testing
the shaping with a basic NAT setup using totally seperate networks on
each ethernet interface.

I now have a fully functional bridge.  I was amazed to see how easy it
is!  I love it!

But now I am trying to add tc back into the mix.  I have looked at the
simple and real life examples, and a typical mark line would be
something like
ebtables -A FORWARD -p ipv4 -i eth0 -j mark --set-mark 2 --mark-target
CONTINUE

So now I have two questions.

How many bits are available for marking?  On docum.org, I saw examples
for mark values up to '5', which leads me to assume that there are at
least 3 bits, meaning I can mark up to '7' (or is it '8'?).  What is the
numeric range here?

And what I am wanting to do is possible, right?  I haven't found any
real life examples for traffic shaping over a bridge yet (links
appreciated).  I have found this though:
"When you create a bridge with the bridge-utils, you get a new device :
br0. You can shape traffic on this device, but you can not use iptables
to mark packets and the fw filter to use that mark. But you can use the
u32 filter."

Does this mean I need to use some other process besides normal mangling?

What it comes down to are two issues (that are almost the same).  I
don't know how to mark packets coming in from the LAN, and I don't know
which interface to bind tc to.

Here is a diagram of my setup:

64.119.201.0/24 ---- eth1 [bridge, br0] eth0 ---- 64.119.201.1 (router)

And here is the script I am playing with.  I have tried binding tc and
ebtables and iptables to all three interfaces (eth0,eth1,br0) and it
doesn't seem to change anything.. but then again I don't know if my
packets are even being marked.  Should I be using a 2.5.x kernel instead
of the patched 2.4.20?  I need this for production use.. stability is
important.

#!/bin/sh
OUTSIDE="eth0"
INSIDE="eth1"
BRIDGE="br0"
LAN="64.119.201.0/24"
OPTION="allot 1514 maxburst 20 avpkt 1000 prio 3"

ebtables -P INPUT ACCEPT
ebtables -P OUTPUT ACCEPT
ebtables -P FORWARD ACCEPT
ebtables -F
ebtables -t nat -F

iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -F
iptables -t nat -F

echo "Loading ebtables rules"
ebtables -A FORWARD -p ipv4 -i eth1 --ip-source 64.119.201.114 -j mark
--set-mark 2 --mark-target CONTINUE
ebtables -A FORWARD -p ipv4 -j mark --set-mark 1 --mark-target CONTINUE

echo "Loading iptables rules"
iptables -A PREROUTING -i eth1 -s $LAN -t mangle -j MARK --set-mark 1
iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2

###############################################################################
# i have tried $INSIDE, $OUTSIDE, and $BRIDGE here
DEV="dev $INSIDE"
RATE_TOT\x10kbit
SERVERSP0kbit
SERVERS_WEIGHTPkbit
GENPkbit
GEN_WEIGHT=5kbit

echo "Clearing qdiscs"
tc qdisc del dev br0 root
tc qdisc del dev eth0 root
tc qdisc del dev eth1 root

echo "Inserting qdiscs"
tc qdisc add $DEV root handle 10: cbq bandwidth $RATE_TOT avpkt 1000
tc class add $DEV parent 10:0 classid 10:2 cbq bandwidth $RATE_TOT rate
$RATE_TOT $OPTION bounded

tc qdisc add $DEV parent 10:2 handle 20: cbq bandwidth $RATE_TOT allot
1514 avpkt 1000
tc class add $DEV parent 20: classid 20:2 cbq bandwidth $RATE_TOT rate
$RATE_TOT $OPTION prio 3
tc class add $DEV parent 20:2 classid 20:10 cbq bandwidth $RATE_TOT rate
$SERVERS $OPTION weight $SERVERS_WEIGHT bounded
tc class add $DEV parent 20:2 classid 20:20 cbq bandwidth $RATE_TOT rate
$GEN     $OPTION weight $GEN_WEIGHT bounded

echo "Adding tc filters"
tc filter add $DEV parent 10: protocol ip prio 3 handle 1 fw classid 10:2
tc filter add $DEV parent 10: protocol ip prio 3 handle 2 fw classid 10:2

tc filter add $DEV parent 20: protocol ip prio 3 handle 1 fw classid 20:20
tc filter add $DEV parent 20: protocol ip prio 3 handle 2 fw classid 20:10

echo "eth0:"
tc -s qdisc ls dev eth0
echo "eth1:"
tc -s qdisc ls dev eth1
echo "br0:"
tc -s qdisc ls dev br0

Hugh Buchanan
Userfriendly.com



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Shaping traffic over a linux bridge

[Hugh Buchanan]

I should add some additional comments.

I have gone through most of the LARTC archives dealing with tc.. it 
seems a lot of people have attempted this, but no one ever posts 
solutions to these things.

There are a bunch of archive posts I found somewhat helpful.
http://lists.netfilter.org/pipermail/netfilter/2002-May/034041.html
http://mailman.ds9a.nl/pipermail/lartc/2003q1/007571.html

Using those two posts, I decided to give u32 filtering a try to avoid 
the mangling confusion I have.. and it doesn't seem to change much.

Shaping works perfectly using NAT/MASQUERADE and iptables mangling.  
When I throw bridging into the mix it stops working.

I have yet to try a 2.5.x kernel.. I would prefer not to, but I suppose 
since I have no ideas right now on how to proceed, I better try 2.5.x.

If anyone figures this out, send me your address.. i'll send you a box 
of cookies or something.

Hugh

Hugh Buchanan wrote:

> Greetings,
>
> I am using tc/cbq to do some traffic shaping over a linux bridge.  My
> system is running the 2.4.20 kernel with the latest bridge-nf and
> ebtables patches.  I am also running ebtables 2.0.3 and iptables 1.2.8.
>
> Last week, since I had yet to discover ebtables/bridging I was testing
> the shaping with a basic NAT setup using totally seperate networks on
> each ethernet interface.
>
> I now have a fully functional bridge.  I was amazed to see how easy it
> is!  I love it!
>
> But now I am trying to add tc back into the mix.  I have looked at the
> simple and real life examples, and a typical mark line would be
> something like
> ebtables -A FORWARD -p ipv4 -i eth0 -j mark --set-mark 2 --mark-target
> CONTINUE
>
> So now I have two questions.
>
> How many bits are available for marking?  On docum.org, I saw examples
> for mark values up to '5', which leads me to assume that there are at
> least 3 bits, meaning I can mark up to '7' (or is it '8'?).  What is the
> numeric range here?
>
> And what I am wanting to do is possible, right?  I haven't found any
> real life examples for traffic shaping over a bridge yet (links
> appreciated).  I have found this though:
> "When you create a bridge with the bridge-utils, you get a new device :
> br0. You can shape traffic on this device, but you can not use iptables
> to mark packets and the fw filter to use that mark. But you can use the
> u32 filter."
>
> Does this mean I need to use some other process besides normal mangling?
>
> What it comes down to are two issues (that are almost the same).  I
> don't know how to mark packets coming in from the LAN, and I don't know
> which interface to bind tc to.
>
> Here is a diagram of my setup:
>
> 64.119.201.0/24 ---- eth1 [bridge, br0] eth0 ---- 64.119.201.1 (router)
>
> And here is the script I am playing with.  I have tried binding tc and
> ebtables and iptables to all three interfaces (eth0,eth1,br0) and it
> doesn't seem to change anything.. but then again I don't know if my
> packets are even being marked.  Should I be using a 2.5.x kernel instead
> of the patched 2.4.20?  I need this for production use.. stability is
> important.
>
> #!/bin/sh
> OUTSIDE="eth0"
> INSIDE="eth1"
> BRIDGE="br0"
> LAN="64.119.201.0/24"
> OPTION="allot 1514 maxburst 20 avpkt 1000 prio 3"
>
> ebtables -P INPUT ACCEPT
> ebtables -P OUTPUT ACCEPT
> ebtables -P FORWARD ACCEPT
> ebtables -F
> ebtables -t nat -F
>
> iptables -P INPUT ACCEPT
> iptables -P OUTPUT ACCEPT
> iptables -P FORWARD ACCEPT
> iptables -F
> iptables -t nat -F
>
> echo "Loading ebtables rules"
> ebtables -A FORWARD -p ipv4 -i eth1 --ip-source 64.119.201.114 -j mark
> --set-mark 2 --mark-target CONTINUE
> ebtables -A FORWARD -p ipv4 -j mark --set-mark 1 --mark-target CONTINUE
>
> echo "Loading iptables rules"
> iptables -A PREROUTING -i eth1 -s $LAN -t mangle -j MARK --set-mark 1
> iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2
>
> ###############################################################################
> # i have tried $INSIDE, $OUTSIDE, and $BRIDGE here
> DEV="dev $INSIDE"
> RATE_TOT\x10kbit
> SERVERSP0kbit
> SERVERS_WEIGHTPkbit
> GENPkbit
> GEN_WEIGHT=5kbit
>
> echo "Clearing qdiscs"
> tc qdisc del dev br0 root
> tc qdisc del dev eth0 root
> tc qdisc del dev eth1 root
>
> echo "Inserting qdiscs"
> tc qdisc add $DEV root handle 10: cbq bandwidth $RATE_TOT avpkt 1000
> tc class add $DEV parent 10:0 classid 10:2 cbq bandwidth $RATE_TOT rate
> $RATE_TOT $OPTION bounded
>
> tc qdisc add $DEV parent 10:2 handle 20: cbq bandwidth $RATE_TOT allot
> 1514 avpkt 1000
> tc class add $DEV parent 20: classid 20:2 cbq bandwidth $RATE_TOT rate
> $RATE_TOT $OPTION prio 3
> tc class add $DEV parent 20:2 classid 20:10 cbq bandwidth $RATE_TOT rate
> $SERVERS $OPTION weight $SERVERS_WEIGHT bounded
> tc class add $DEV parent 20:2 classid 20:20 cbq bandwidth $RATE_TOT rate
> $GEN     $OPTION weight $GEN_WEIGHT bounded
>
> echo "Adding tc filters"
> tc filter add $DEV parent 10: protocol ip prio 3 handle 1 fw classid 10:2
> tc filter add $DEV parent 10: protocol ip prio 3 handle 2 fw classid 10:2
>
> tc filter add $DEV parent 20: protocol ip prio 3 handle 1 fw classid 
> 20:20
> tc filter add $DEV parent 20: protocol ip prio 3 handle 2 fw classid 
> 20:10
>
> echo "eth0:"
> tc -s qdisc ls dev eth0
> echo "eth1:"
> tc -s qdisc ls dev eth1
> echo "br0:"
> tc -s qdisc ls dev br0
>
> Hugh Buchanan
> Userfriendly.com
>
>
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Shaping traffic over a linux bridge

[Hugh Buchanan]

I have since fixed my problem.  I am not sure if it's useful to anyone, 
but I'll briefly describe what I did to get it working.

In the end, I am not using the 'iptables' program at all.  I am using 
ebtables to mangle the packets on eth1 as they come in.  What I was 
doing before with ebtables was mostly right.. I have pretty simple 
ebtable lines that mark the packets.  Packet marks can be 32 bits.

And my shaping is very different now too.. I shape on the outside (eth0) 
interface.. I can shape incoming traffic going over the eth1 interface 
as well, although I don't need it.

I also find that cbq doesn't work too happily.. HTB works like a charm, 
and really is more useful for my application.  I ended up needing to use 
the 'tc' binary supplied by the HTB people.

I am not completely done yet.. but at the very least it works.. it is 
now just a matter of setting up all of the classes/rules for the 200+ 
MAC addresses I'll be shaping.

Hugh

Hugh Buchanan wrote:

> I should add some additional comments.
>
> I have gone through most of the LARTC archives dealing with tc.. it 
> seems a lot of people have attempted this, but no one ever posts 
> solutions to these things.
>
> There are a bunch of archive posts I found somewhat helpful.
> http://lists.netfilter.org/pipermail/netfilter/2002-May/034041.html
> http://mailman.ds9a.nl/pipermail/lartc/2003q1/007571.html
>
> Using those two posts, I decided to give u32 filtering a try to avoid 
> the mangling confusion I have.. and it doesn't seem to change much.
>
> Shaping works perfectly using NAT/MASQUERADE and iptables mangling.  
> When I throw bridging into the mix it stops working.
>
> I have yet to try a 2.5.x kernel.. I would prefer not to, but I 
> suppose since I have no ideas right now on how to proceed, I better 
> try 2.5.x.
>
> If anyone figures this out, send me your address.. i'll send you a box 
> of cookies or something.
>
> Hugh
>
> Hugh Buchanan wrote:
>
>> Greetings,
>>
>> I am using tc/cbq to do some traffic shaping over a linux bridge.  My
>> system is running the 2.4.20 kernel with the latest bridge-nf and
>> ebtables patches.  I am also running ebtables 2.0.3 and iptables 1.2.8.
>>
>> Last week, since I had yet to discover ebtables/bridging I was testing
>> the shaping with a basic NAT setup using totally seperate networks on
>> each ethernet interface.
>>
>> I now have a fully functional bridge.  I was amazed to see how easy it
>> is!  I love it!
>>
>> But now I am trying to add tc back into the mix.  I have looked at the
>> simple and real life examples, and a typical mark line would be
>> something like
>> ebtables -A FORWARD -p ipv4 -i eth0 -j mark --set-mark 2 --mark-target
>> CONTINUE
>>
>> So now I have two questions.
>>
>> How many bits are available for marking?  On docum.org, I saw examples
>> for mark values up to '5', which leads me to assume that there are at
>> least 3 bits, meaning I can mark up to '7' (or is it '8'?).  What is the
>> numeric range here?
>>
>> And what I am wanting to do is possible, right?  I haven't found any
>> real life examples for traffic shaping over a bridge yet (links
>> appreciated).  I have found this though:
>> "When you create a bridge with the bridge-utils, you get a new device :
>> br0. You can shape traffic on this device, but you can not use iptables
>> to mark packets and the fw filter to use that mark. But you can use the
>> u32 filter."
>>
>> Does this mean I need to use some other process besides normal mangling?
>>
>> What it comes down to are two issues (that are almost the same).  I
>> don't know how to mark packets coming in from the LAN, and I don't know
>> which interface to bind tc to.
>>
>> Here is a diagram of my setup:
>>
>> 64.119.201.0/24 ---- eth1 [bridge, br0] eth0 ---- 64.119.201.1 (router)
>>
>> And here is the script I am playing with.  I have tried binding tc and
>> ebtables and iptables to all three interfaces (eth0,eth1,br0) and it
>> doesn't seem to change anything.. but then again I don't know if my
>> packets are even being marked.  Should I be using a 2.5.x kernel instead
>> of the patched 2.4.20?  I need this for production use.. stability is
>> important.
>>
>> #!/bin/sh
>> OUTSIDE="eth0"
>> INSIDE="eth1"
>> BRIDGE="br0"
>> LAN="64.119.201.0/24"
>> OPTION="allot 1514 maxburst 20 avpkt 1000 prio 3"
>>
>> ebtables -P INPUT ACCEPT
>> ebtables -P OUTPUT ACCEPT
>> ebtables -P FORWARD ACCEPT
>> ebtables -F
>> ebtables -t nat -F
>>
>> iptables -P INPUT ACCEPT
>> iptables -P OUTPUT ACCEPT
>> iptables -P FORWARD ACCEPT
>> iptables -F
>> iptables -t nat -F
>>
>> echo "Loading ebtables rules"
>> ebtables -A FORWARD -p ipv4 -i eth1 --ip-source 64.119.201.114 -j mark
>> --set-mark 2 --mark-target CONTINUE
>> ebtables -A FORWARD -p ipv4 -j mark --set-mark 1 --mark-target CONTINUE
>>
>> echo "Loading iptables rules"
>> iptables -A PREROUTING -i eth1 -s $LAN -t mangle -j MARK --set-mark 1
>> iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2
>>
>> ###############################################################################
>> # i have tried $INSIDE, $OUTSIDE, and $BRIDGE here
>> DEV="dev $INSIDE"
>> RATE_TOT\x10kbit
>> SERVERSP0kbit
>> SERVERS_WEIGHTPkbit
>> GENPkbit
>> GEN_WEIGHT=5kbit
>>
>> echo "Clearing qdiscs"
>> tc qdisc del dev br0 root
>> tc qdisc del dev eth0 root
>> tc qdisc del dev eth1 root
>>
>> echo "Inserting qdiscs"
>> tc qdisc add $DEV root handle 10: cbq bandwidth $RATE_TOT avpkt 1000
>> tc class add $DEV parent 10:0 classid 10:2 cbq bandwidth $RATE_TOT rate
>> $RATE_TOT $OPTION bounded
>>
>> tc qdisc add $DEV parent 10:2 handle 20: cbq bandwidth $RATE_TOT allot
>> 1514 avpkt 1000
>> tc class add $DEV parent 20: classid 20:2 cbq bandwidth $RATE_TOT rate
>> $RATE_TOT $OPTION prio 3
>> tc class add $DEV parent 20:2 classid 20:10 cbq bandwidth $RATE_TOT rate
>> $SERVERS $OPTION weight $SERVERS_WEIGHT bounded
>> tc class add $DEV parent 20:2 classid 20:20 cbq bandwidth $RATE_TOT rate
>> $GEN     $OPTION weight $GEN_WEIGHT bounded
>>
>> echo "Adding tc filters"
>> tc filter add $DEV parent 10: protocol ip prio 3 handle 1 fw classid 
>> 10:2
>> tc filter add $DEV parent 10: protocol ip prio 3 handle 2 fw classid 
>> 10:2
>>
>> tc filter add $DEV parent 20: protocol ip prio 3 handle 1 fw classid 
>> 20:20
>> tc filter add $DEV parent 20: protocol ip prio 3 handle 2 fw classid 
>> 20:10
>>
>> echo "eth0:"
>> tc -s qdisc ls dev eth0
>> echo "eth1:"
>> tc -s qdisc ls dev eth1
>> echo "br0:"
>> tc -s qdisc ls dev br0
>>
>> Hugh Buchanan
>> Userfriendly.com
>


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/