[LARTC] Suggestions for a new shaper

[LARTC] Suggestions for a new shaper

[Kenneth Kalmer]

Guys

Reading through my lartc mail, and thinking on how to tackle my next
shaping solution, gave me an idea. I'd like to propose this idea for
scrutiny and if it sounds viable I'll definitely need to start
developing in due time.

For me in my day to day work with developing network application
specifically geared for middle-tier ISP's, and reading on other
peoples issues I gather that the following would be the ultimate
solution to traffic jams.

First the case study. An untrusted network with over-subscribed users
(and abusers). Common at universities and bigger enterprises. People
want good speed at any time, but abusers should be detected and
clamped down automatically. This system needs intelligence. Let's say
a good combo between HTB & WRR, through in a pacemaker and the
configurability of XML files.

My thinking is Python (I only code interpreted) doing the
configurations and "live" htb manipulation to simulate wrr, but still
offer the gaurentees of htb. The account for data and detect abusers,
something like pmacctd would be easy to integrate with and provides
powerful options.

Extra things should be the ability to handle multiple subnets, not
just a single one, and while doing Python we might as well have
options to produce rrd's to make sure that everything is running
perfectly.

Any advice, tips or suggestions? I know it won't be an easy feat, but
it is worth a shot...

Best

--

Kenneth Kalmer
kenneth.kalmer@gmail.com

Folding@home stats
http://fah-web.stanford.edu/cgi-bin/main.py?qtype=userpage&username=kenneth%2Ekalmer
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Suggestions for a new shaper

[Peter Surda]

Kenneth Kalmer wrote:
> Guys
Hi,

> First the case study. An untrusted network with over-subscribed users
> (and abusers). Common at universities and bigger enterprises. People
> want good speed at any time, but abusers should be detected and
> clamped down automatically. This system needs intelligence. Let's say
> a good combo between HTB & WRR, through in a pacemaker and the
> configurability of XML files.
I have a lot of experience in networks like this, so I'll post some 
comments:

> My thinking is Python (I only code interpreted) doing the
> configurations and "live" htb manipulation to simulate wrr, but still
> offer the gaurentees of htb.
I have heard that HTB doesn't scale well when you have it arranged in 
this way (search "Ostrochovsky" in the archives, or perhaps it was 
another list => use google). This would have to be analysed. Usually I 
avoid it by creating more levels (think binary tree), but I don't have 
any measurements on high load.

WRR on the other hand has many advantages: it scales very well, 
automatically penalizes abusers, and behaves predictably (I performed a 
statistical-mathematical analysis of one such case last week, got great 
results). There is one problem though: what it distributes is actually 
sending frequency, not bandwidth. Therefore, you can't set limits to 
traffic of individuals, you can't give anyone "guaranteed" bandwith, or 
give someone "twice as much bandwidth as the others". You can set it up 
so that certain IPs are penalized less or more, but there is no way to 
give them specific bandwidth.

In my experience (my customers) sometimes want to set specific 
bandwidths, but after using WRR for some time, due to its fairness 
characteristics they decide it's not necessary. A commercial ISP may 
have different requirements though, so that they can sell different 
bandwidths for different prices.

Although I don't have a mathematical proof :-), I think that these two 
approaches are mutually exclusive. If you want predictable behaviour (no 
latency peaks), you can either give individual people specific 
bandwidth, or you can utilize the bandwidth fully. You can't utilize the 
whole bandwidth while giving people specific amounts of it.

Nevertheless, I have some ideas how to bring certain advantages of HTB 
to WRR:
- if you want to cap someone, you may be able to use a htb inside one 
wrr-subclass
- if you want to guarantee someone's bandwidth, give him a permanently 
higher weight (cap others to a lower weight), put htb there and set its rate

(For comparison, I use ESFQ inside the wrr-subclasses).

I'd like to stress however that you lose a lot of WRR's predictability 
if you manually fiddle with the settings. This would have to be tested 
extensively.

> Extra things should be the ability to handle multiple subnets, not
> just a single one,
WRR handles multiple subnets without problems and without the need to 
set anything (classes are assigned dynamically as the IPs appear).

 > and while doing Python we might as well have
> options to produce rrd's to make sure that everything is running
> perfectly.
Sorry for advertisement (again). Shurdix already does this, most of it 
automatically (and uses perl, but the scripts are very simple). I use 
ipt_ACCOUNT for accounting because I want it to be independent from 
traffic control, but WRR has it's own accounting method (look on their 
website, it is also more precise, because iptables-based accounting 
doesn't know about packets that are dropped while doing egress).

> Any advice, tips or suggestions? I know it won't be an easy feat, but
> it is worth a shot...
As I said, the first thing you should do is to set your priorities:
- if you want to utilise the bandwidth fully, go WRR
- if you want to set specific parameters, go HTB

And you have to test, test, test, because if you lose predictability 
you're screwed :-).

> Best
> Kenneth Kalmer
Yours sincerely,
Peter

-- 
http://www.shurdix.org - Linux distribution for routers and firewalls
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Suggestions for a new shaper

[Kenneth Kalmer]

On 1/30/06, Peter Surda <surda@shurdix.com> wrote:
> Kenneth Kalmer wrote:
> > Guys
> Hi,

Hi Peter

> > First the case study. An untrusted network with over-subscribed users
> > (and abusers). Common at universities and bigger enterprises. People
> > want good speed at any time, but abusers should be detected and
> > clamped down automatically. This system needs intelligence. Let's say
> > a good combo between HTB & WRR, through in a pacemaker and the
> > configurability of XML files.
> I have a lot of experience in networks like this, so I'll post some
> comments:

I know, I've learned a lot about WRR from your postings and hacking
shurdix to sh*t... :)

> > My thinking is Python (I only code interpreted) doing the
> > configurations and "live" htb manipulation to simulate wrr, but still
> > offer the gaurentees of htb.
> I have heard that HTB doesn't scale well when you have it arranged in
> this way (search "Ostrochovsky" in the archives, or perhaps it was
> another list => use google). This would have to be analysed. Usually I
> avoid it by creating more levels (think binary tree), but I don't have
> any measurements on high load.

Understood

> WRR on the other hand has many advantages: it scales very well,
> automatically penalizes abusers, and behaves predictably (I performed a
> statistical-mathematical analysis of one such case last week, got great
> results). There is one problem though: what it distributes is actually
> sending frequency, not bandwidth. Therefore, you can't set limits to
> traffic of individuals, you can't give anyone "guaranteed" bandwith, or
> give someone "twice as much bandwidth as the others". You can set it up
> so that certain IPs are penalized less or more, but there is no way to
> give them specific bandwidth.

 I got this under control, I just have to automate this in my current setup.

> In my experience (my customers) sometimes want to set specific
> bandwidths, but after using WRR for some time, due to its fairness
> characteristics they decide it's not necessary. A commercial ISP may
> have different requirements though, so that they can sell different
> bandwidths for different prices.

This is true, but a product of ours needs to offer a best of both
world solution... I also think a lot of other people would benefit
from such a setup if, and when, it works as advertised. Imagine that
everyone can use the internet at full speed, but as soon as the CEO
needs to view a webcast everyone is slowed down to accommodate him. I
know it's great in theory, but some people will demand this...

> Although I don't have a mathematical proof :-), I think that these two
> approaches are mutually exclusive. If you want predictable behaviour (no
> latency peaks), you can either give individual people specific
> bandwidth, or you can utilize the bandwidth fully. You can't utilize the
> whole bandwidth while giving people specific amounts of it.

Here I tend to disagree... Unless tc and the kernel can't handle it
properly, why can't wrr be partially simulated by constantly adjusting
the rates of HTB classes. I use wrr myself and love it, but there are
things in htb that I desire in wrr (and vice versa).

> Nevertheless, I have some ideas how to bring certain advantages of HTB
> to WRR:
> - if you want to cap someone, you may be able to use a htb inside one
> wrr-subclass
> - if you want to guarantee someone's bandwidth, give him a permanently
> higher weight (cap others to a lower weight), put htb there and set its rate
>
> (For comparison, I use ESFQ inside the wrr-subclasses).

So do I, but I never even thought about attaching an htb to a wrr. I
still want to try creating two htb's siblings and attaching a wrr to
each of them, but IIRC in one of your postings to me that either wrr
or the kernel doesn't like this very much and it will eventually
panic.

> I'd like to stress however that you lose a lot of WRR's predictability
> if you manually fiddle with the settings. This would have to be tested
> extensively.

Bumped my head several times with the insane wrr params...

> > Extra things should be the ability to handle multiple subnets, not
> > just a single one,
> WRR handles multiple subnets without problems and without the need to
> set anything (classes are assigned dynamically as the IPs appear).

This is very cool, I know... The same can be simulated by a script...

>  > and while doing Python we might as well have
> > options to produce rrd's to make sure that everything is running
> > perfectly.
> Sorry for advertisement (again). Shurdix already does this, most of it
> automatically (and uses perl, but the scripts are very simple). I use
> ipt_ACCOUNT for accounting because I want it to be independent from
> traffic control, but WRR has it's own accounting method (look on their
> website, it is also more precise, because iptables-based accounting
> doesn't know about packets that are dropped while doing egress).

Advertise as much as you like. As stated earlier I've hacked shurdix
to sh*t to see how you achieve things and you to get a working wrr
config. The problem is that a lot of people, like us, tend to roll our
'all-in-one' solutions that need a host of other software and services
running over several boxes, all very tightly integrated to deliver an
in-house ISP (for the lack of a better word). Everything we build
starts on a clean RHEL and then mutates to a fully working product
(while keeping the base firmly intact).

> > Any advice, tips or suggestions? I know it won't be an easy feat, but
> > it is worth a shot...
> As I said, the first thing you should do is to set your priorities:
> - if you want to utilise the bandwidth fully, go WRR
> - if you want to set specific parameters, go HTB
>
> And you have to test, test, test, because if you lose predictability
> you're screwed :-).

Definitely. Everyone must understand that I'm not biased to either one
of the algorithms, something is just telling me to try and fine the
middle man. Something that is both firm and fair, and can keep a
constant eye on things, set guarantees and stick to them.

It can be done, I'm sure, it is only a matter of time!

> > Best
> > Kenneth Kalmer
> Yours sincerely,
> Peter

As always, thanks Peter! Very much appreciated!

Regards

--

Kenneth Kalmer
kenneth.kalmer@gmail.com

Folding@home stats
http://fah-web.stanford.edu/cgi-bin/main.py?qtype=userpage&username=kenneth%2Ekalmer
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc