[LARTC] Bandwith thinking error

[LARTC] Bandwith thinking error

[Lars Oeschey]

Hi,

I found that I had some thinking error with the wshaper script. I assigned
the bandwith of my DSL connection to it, but the machine where it runs is
normally connected to the LAN with 100Mbit behind a separate
Hardware-Router.Obviously, the complete connection of the machine was slowed down to 384k
because I told it so.I guess, since wshaper takes only one card as argument, I can't use that
script for shaping, as I just want to slow done specific services on that
card, but I can't enter the correct value for Bandwith.
So what is the correct way to go? I think I need two paths, one with
100MBit for the local network (where actually there'll happen no shaping)
and another path with 384k where I need to shape Internet services after
priority...
Lars


-- 
visit the C.O.R.E. http://www.the-core.net


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Andreas Klauer]

Am Wednesday 12 May 2004 16:08 schrieb Lars Oeschey:

> I found that I had some thinking error with the wshaper script. I
> assigned the bandwith of my DSL connection to it, but the machine where
> it runs is normally connected to the LAN with 100Mbit behind a separate
> Hardware-Router.

WShaper reduces the complete bandwidth of a device to a given rate, so it's 
to be attached directly to the internet device. So if you use the same 
device for communicating with the router and with other machines in the 
LAN, there is a problem.

> So what is the correct way to go? 

You have the same problem if you attach HTB filters (for incoming NATed 
bandwidth) to the LAN device. It can be solved by creating one fat parent 
class which has the full LAN rates. This fat class gets two children: a 
DSL class which gets the DSL rates and a LAN class which gets (LAN minus 
DSL) rate. The DSL class then gets further children for DSL traffic 
classification for example on a per user or interactive/http/protocol 
basis.
        
You have to add your filters then to the DSL class instead of parent qdisc,
and a filter in the parent qdisc that puts packets that go to the Router IP
into the DSL class. Or modify your filters so that they only apply to 
Router packets. Especially if you're using ingress, you have to modify the 
policy filters so that they only apply to packets coming from the router.

As a simplified ascii graphic:

   HTB qdisc
    |
    \--- HTB fat class (LAN rate)
          |
          \--- HTB DSL class (DSL rate; only packets to the router go here)
          \--- HTB LAN class (LAN-DSL rate; all other packets go here)
        
A problem with this design would be if you have additional local traffic 
that goes to the router (e.g. a ssh session to the router that does not 
actually go to the internet). This would be classified as DSL traffic too. 
I don't know if filters can be designed in a way that they only match on 
gateway'ed traffic.

Shaping this way won't work particularly well especially if there are other 
users in your LAN using the router. You should do the shaping directly on 
the router in any case.

HTH
Andreas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Lars Oeschey]

<Zitiere wer="Andreas Klauer">

> You have to add your filters then to the DSL class instead of
> parent qdisc, and a filter in the parent qdisc that puts packets
> that go to the Router IP into the DSL class. Or modify your filters
> so that they only apply to  Router packets. Especially if you're
> using ingress, you have to modify the  policy filters so that they
> only apply to packets coming from the router.

err, what's ingress?

> As a simplified ascii graphic:
>
>   HTB qdisc
>    |
>    \--- HTB fat class (LAN rate)
>          |
>          \--- HTB DSL class (DSL rate; only packets to the router
>          go here) \--- HTB LAN class (LAN-DSL rate; all other
>          packets go here)
>
> A problem with this design would be if you have additional local
> traffic  that goes to the router (e.g. a ssh session to the router
> that does not  actually go to the internet). This would be
> classified as DSL traffic too.  I don't know if filters can be
> designed in a way that they only match on  gateway'ed traffic.

There's the occasional http traffic to the router for configuration
(it's the current t-online WLan device), but that's quite rare and I
don't care if it's a bit slow then (hey, I already worked with imap
over 384k in my LAN ;)).
> Shaping this way won't work particularly well especially if there
> are other  users in your LAN using the router. You should do the
> shaping directly on  the router in any case.

There's just me and my wife, and she only uses http over the proxy on
the shaped machine. I will transfer her pop3 access to the linux-box
w/fetchmail some day, so every traffic goes through the linux-box.Um. Forgot about ftp, it's currently direct... perhaps I should set up
a ftp proxy too then... But it's also quite rare.
Are there any tools to define the shaping? Or do I really have to
write it from scratch?
Lars


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Andreas Klauer]

Am Thursday 13 May 2004 09:06 schrieb Lars Oeschey:

> err, what's ingress?

Read in the LARTC Howto (www.lartc.org) about it.
Wondershaper does it for limiting incoming traffic.
The real classful shaping can only be done for outgoing traffic.
(Well, probably unless you're using IMQ, but Wondershaper doesn't).

> There's the occasional http traffic to the router for configuration
> (it's the current t-online WLan device), but that's quite rare and I
> don't care if it's a bit slow then (hey, I already worked with imap
> over 384k in my LAN ;)).

Never mind that, I figured it should be easy to get the filters right.

> There's just me and my wife, 

You need shaping so badly if there are only two users?

> I will transfer her pop3 access to the linux-box w/fetchmail some day, so
> every traffic goes through the linux-box.

If you want every traffic to go through your linux box, remove the router 
from the LAN, connect it directly to your box and let your box do the 
routing for your wife. In case your box can go online directly, sell the 
router.

> Are there any tools to define the shaping? Or do I really have to
> write it from scratch?

I just modified the wondershaper script a little. You could test if it
actually works. I couldn't test it, since I have a dedicated linux box 
doing the routing.

The modified wondershaper is here:
    http://www.metamorpher.de/files/wshaper-over-lan.htb

Here's an image of the class structure it creates:
    http://www.metamorpher.de/files/wshaper-over-lan.png

This image was created with Graphviz. I just love this program :-)
I used a hacked version of Stef Coene's show.pl to generate the graph.
Still have problems parsing the filter output, though...

The blue boxes are qdiscs, the orange house is a root class, the green eggs 
are normal classes. Green arrows go to the root class, red arrows go to 
the leafs. :-) It doesn't show filters though, except for mark handles... 
tc filter output isn't easy to parse.

The LAN traffic should go into the fat egg on the right. The internet
traffic should go into the smaller egg on the left (and into it's 
children).

Use "wshaper-over-lan.htb status" to see if it actually works.
If you produce both traffic in the LAN and on the internet,
the LAN traffic should show up in class 1:3, the internet traffic in class 
1:1 and it's children.

Andreas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Lars Oeschey]

<Zitiere wer="Andreas Klauer">
> Am Thursday 13 May 2004 09:06 schrieb Lars Oeschey:

> Read in the LARTC Howto (www.lartc.org) about it.

just did ;)

>> There's just me and my wife,
> You need shaping so badly if there are only two users?

err, not really, it's just that http/mail etc. wins over p2p ;)

> If you want every traffic to go through your linux box, remove the
> router  from the LAN, connect it directly to your box and let your
> box do the  routing for your wife. In case your box can go online
> directly, sell the  router.

mh, been there, done that... I come from fli4l, and was glad that I
could switch to that tiny little router-device ;)I think when I can just priorise squid over p2p stuff, everything
should be fine
> I just modified the wondershaper script a little. You could test if
> it actually works. I couldn't test it, since I have a dedicated
> linux box  doing the routing.
> The modified wondershaper is here:

cool, thanks a lot for that... I'll check it out when I'm home today!

Lars

-- 
visit the C.O.R.E. http://www.the-core.net


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Andreas Klauer]

Am Thursday 13 May 2004 15:48 schrieb Lars Oeschey:
> err, not really, it's just that http/mail etc. wins over p2p ;)

If you want to detect P2P traffic, have a look into this:
http://rnvs.informatik.uni-leipzig.de/ipp2p/index_en.html

My Fair NAT script supports it. It works well for me. :-)

HTH
Andreas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Lars Oeschey]

Andreas Klauer wrote:

>The modified wondershaper is here:
>    http://www.metamorpher.de/files/wshaper-over-lan.htb
>  
>
I tested the script now, it works good so far in that LAN traffic isn't 
slowed down anymore . But when p2p has full Bandwidth, http over the 
proxy is very slow. I would like that p2p just gets the bandwidth thats 
left over after http, is that possible? What I tried with the original 
script was this, could that work?

tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
   match ip dport 3128 0xffff flowid 1:10

tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
   match ip sport 3128 0xffff flowid 1:10

tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
   match ip dport 80 0xffff flowid 1:10

tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
   match ip sport 80 0xffff flowid 1:10

Lars
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Andreas Klauer]

Am Thursday 13 May 2004 20:59 schrieb Lars Oeschey:
> Andreas Klauer wrote:
> >The modified wondershaper is here:
> >    http://www.metamorpher.de/files/wshaper-over-lan.htb
>
> I would like that p2p just gets the bandwidth thats
> left over after http, is that possible?

It's hard to get the P2P stuff right. First of all, it's usually
download traffic - but Wondershaper only shapes upload traffic.
Second of all, what is P2P traffic? Matching http protocol probably won't 
work, since some P2P programs also use HTTP. (IIRC BitTorrent uses http to 
communicate with the tracker). If you're only prioritizing specific port, 
you'll find many other forms of communications (chats, ftp, whatever) be 
slow because that's put into same class as P2P too.

> What I tried with the original 
> script was this, could that work?
>
> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip dport 3128 0xffff flowid 1:10
>
> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip sport 3128 0xffff flowid 1:10

3128 is a http port? I didn't know >_>

> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip dport 80 0xffff flowid 1:10
>
> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip sport 80 0xffff flowid 1:10

That'd be bad for interactive connections... change default to 30,
lower rate for 30 (low rate, high ceil, large prio, so the class has to
borrow everything). Put http traffic into 20. Wondershaper's rate settings
are weird anyway - sum of child rates is much larger than parent rate.

HTH
Andreas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Jason Boxman]

On Thursday 13 May 2004 14:59, Lars Oeschey wrote:
> Andreas Klauer wrote:
> >The modified wondershaper is here:
> >    http://www.metamorpher.de/files/wshaper-over-lan.htb
>
> I tested the script now, it works good so far in that LAN traffic isn't
> slowed down anymore . But when p2p has full Bandwidth, http over the
> proxy is very slow. I would like that p2p just gets the bandwidth thats
> left over after http, is that possible? What I tried with the original
> script was this, could that work?

I imagine you set $DEV to ethX, right?  (In the original script it was ppp0.)

Anyway, as to the p2p traffic, it is usually pretty pervasive.  You will need 
to use something like IPP2P or L7-Filter to catch it and stick it in a p2p or 
bulk class.  I found that for edonkey, for example, matching on port 4662 
only caught 80% of the traffic in the best case scenario and 50% in the 
worst, making things like HTTP virtually unusable.

You'll need to patch your kernel to use either one and they both have IPTables 
components you will need to use to match traffic.  IPP2P needs CONNMARK, 
which is available in patch-o-matic for Netfilter.  Be advised that you need 
the CVS version of IPTables to use CONNMARK with 2.6.  IPP2P and CONNMARK 
work well on my 2.4.24 kernel with the CONNMARK patch from patch-o-matic.  
(The non -ng variant.)  I also recommend the CLASSIFY patch if you are going 
to be using IPTables anyway.

I have found that for the simple case of traffic shaping, it is actually easy 
to write your own basic script to handle it.  Wondershaper is a nice template 
to start with.  The usual procedure is 1) decide on classes and leafs and 
define them with `tc` and 2) decide how best to match traffic destined for 
your classes, using either `tc` or IPTables, and define matching rules.

> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip dport 3128 0xffff flowid 1:10
>
> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip sport 3128 0xffff flowid 1:10
>
> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip dport 80 0xffff flowid 1:10
>
> tc filter add dev $DEV parent 1: protocol ip prio 10 u32 \
>    match ip sport 80 0xffff flowid 1:10
>
> Lars

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Bandwith thinking error

[Lars Oeschey]

Jason Boxman wrote:

>I imagine you set $DEV to ethX, right?  (In the original script it was ppp0.)
>
>  
>
yep...

> <>You'll need to patch your kernel to use either one and they both 
> have IPTables
> components you will need to use to match traffic. IPP2P needs CONNMARK,
> which is available in patch-o-matic for Netfilter. Be advised that you 
> need
> the CVS version of IPTables to use CONNMARK with 2.6. IPP2P and CONNMARK
> work well on my 2.4.24 kernel with the CONNMARK patch from patch-o-matic.
> (The non -ng variant.) I also recommend the CLASSIFY patch if you are 
> going
> to be using IPTables anyway.
>
ah, I wanted to avoid kernel compiling since I run a specialized debian 
that's mainly intended as vdr server... I'm a bit afraid of  destryoing 
something with vdr ;) I blew the system up a bit to make it a universal 
server, which was easy with debian, but I still got away without kernel 
compiling...

Lars

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/