From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Barry Rooney" Subject: Performance Monitoring Date: Fri, 2 Jan 2004 22:03:43 -0000 Sender: netfilter-admin@lists.netfilter.org Message-ID: <004401c3d17c$4baa7cc0$0a01000a@xcom1> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0041_01C3D17C.4996B7F0" Return-path: Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: To: netfilter@lists.netfilter.org This is a multi-part message in MIME format. ------=_NextPart_000_0041_01C3D17C.4996B7F0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi All, Can anyone recommend an opensource bandwidth monitoring tool that can = plot throughtput and breakdown into sockets/services for proving the performance of my qdiscs and IPTables? Many thanks Barry. --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.555 / Virus Database: 347 - Release Date: 23/12/2003 ------=_NextPart_000_0041_01C3D17C.4996B7F0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi All,

Can anyone recommend an opensource = bandwidth=20 monitoring tool that can plot throughtput and breakdown into=20 sockets/services

for proving the performance of my = qdiscs and=20 IPTables?

Many thanks

Barry.

---
Outgoing mail is certified = Virus=20 Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: = 6.0.555 /=20 Virus Database: 347 - Release Date:=20 23/12/2003

------=_NextPart_000_0041_01C3D17C.4996B7F0-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Satrapa Subject: Re: Performance Monitoring Date: Tue, 06 Jan 2004 09:27:03 +1100 Sender: netfilter-admin@lists.netfilter.org Message-ID: <3FF9E4B7.8010109@lintelsys.com.au> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <004401c3d17c$4baa7cc0$0a01000a@xcom1> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Barry Rooney Cc: netfilter@lists.netfilter.org Barry Rooney wrote: > .. recommend an opensource bandwidth monitoring tool ... Is IPAC-NG what you're looking for? http://ipac-ng.sourceforge.net/ From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Lawrence Tang" Subject: Re: Performance Monitoring Date: Tue, 6 Jan 2004 11:57:15 +1000 Sender: netfilter-admin@lists.netfilter.org Message-ID: <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii" To: Alex Satrapa , Barry Rooney Cc: netfilter@lists.netfilter.org Does this will help to calculate each PC on LAN MB usage report ?? Lawrence ----- Original Message ----- From: "Alex Satrapa" To: "Barry Rooney" Cc: Sent: Tuesday, January 06, 2004 8:27 AM Subject: Re: Performance Monitoring > Barry Rooney wrote: > > .. recommend an opensource bandwidth monitoring tool ... > > Is IPAC-NG what you're looking for? > > http://ipac-ng.sourceforge.net/ > > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Satrapa Subject: Re: Performance Monitoring Date: Tue, 06 Jan 2004 15:04:23 +1100 Sender: netfilter-admin@lists.netfilter.org Message-ID: <3FFA33C7.9010806@lintelsys.com.au> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> Reply-To: netfilter@lists.netfilter.org Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: netfilter@lists.netfilter.org Lawrence Tang wrote: > Does this will help to calculate each PC on LAN MB usage report ?? You should be able to configure it to do so. IPAC-NG uses separate accounting rules for every item that you want to report on. Thus if you want individual accounting per PC, you can set it up to do so. Install it and fiddle. That's my recommendation. Alex Satrapa From mboxrd@z Thu Jan 1 00:00:00 1970 From: "bino" Subject: Re: Performance Monitoring Date: Tue, 06 Jan 2004 10:38:02 +0700 Sender: netfilter-admin@lists.netfilter.org Message-ID: <20040106033802.30955.qmail@paus.pesat.net.id> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> <3FFA33C7.9010806@lintelsys.com.au> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <3FFA33C7.9010806@lintelsys.com.au> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; format="flowed"; charset="us-ascii" To: netfilter@lists.netfilter.org I my self don't familiar with IPAC-NG. The basic logic block is : 1. use the feature of iptables -N to create per ip-addr IN and Out chain 2. jump every traffic per ip addr, to respective chain use cron to run the bash-script that do : 1. iptables -L -vnx 2. Parse the data from each respective chain 3. stor it to remote MySQL using MySql client tool 4. reset (zero ?) the value of each chain That way you can have a traffic record per station (ip addr) If you just need monitoring like MRTG (in bps, no detailed history record), it'll more simple ... you only need to hack NetSNMPD and use MRTG to do the rest, no SQL hasle. Sincerely -bino- Alex Satrapa writes: > Lawrence Tang wrote: >> Does this will help to calculate each PC on LAN MB usage report ?? > > You should be able to configure it to do so. IPAC-NG uses separate > accounting rules for every item that you want to report on. Thus if you > want individual accounting per PC, you can set it up to do so. > > Install it and fiddle. That's my recommendation. > > Alex Satrapa > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Gale Subject: Re: Performance Monitoring Date: Mon, 5 Jan 2004 22:58:38 -0700 Sender: netfilter-admin@lists.netfilter.org Message-ID: <20040105225838.68cdadc5.michael@bluesuperman.com> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> <3FFA33C7.9010806@lintelsys.com.au> <20040106033802.30955.qmail@paus.pesat.net.id> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20040106033802.30955.qmail@paus.pesat.net.id> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii" To: netfilter@lists.netfilter.org Wait a minute here ... you want a rule for each IP ? Depending on the stats you need I suggest you strongly look into the following: ntop -- provides a web GUI for real time monitoring. Using it now on a firewall box to monitoring traffic on each interface. Adv .. provides great states , very detailed Dis .. seems to be some over header ... uses a DDR db :( You can use curl to pull the stats nightly and save them to a text file. Then create a little PHP scritp to provide you with the numbers. Now you will have stats for as long as you want. iptraf -- not bad ... detail is low. Adv ... NO over head and works great on a work station or 1 interface machine. It takes a bit to setup because you have to create all the filters your self. Dis ... out is simple ... a php script to produce a nice web GUI is needed. Nagios -- http://www.nagios.org/ Could be over kill depending on what you want ... this is more of a network monitoring tool. Really not designed to be run with one machine in mind. IPFM -- not bad .. very simple: example: HOST IN OUT TOTAL host1.domain.com 12345 6666684 6679029 MRTG for total traffic accounts only Bandwidthd -- not bad ... currently testing it. Seems to provide web png files much like MRTG but does provide host info. I do not believe you are able to save the data though :( Michael. On Tue, 06 Jan 2004 10:38:02 +0700 "bino" wrote: > I my self don't familiar with IPAC-NG. > The basic logic block is : > 1. use the feature of iptables -N to create per ip-addr IN and Out > chain 2. jump every traffic per ip addr, to respective chain > > use cron to run the bash-script that do : > 1. iptables -L -vnx > 2. Parse the data from each respective chain > 3. stor it to remote MySQL using MySql client tool > 4. reset (zero ?) the value of each chain > > That way you can have a traffic record per station (ip addr) > > If you just need monitoring like MRTG (in bps, no detailed history > record), it'll more simple ... you only need to hack NetSNMPD and use > MRTG to do the rest, no SQL hasle. > > Sincerely > -bino- > > Alex Satrapa writes: > > > Lawrence Tang wrote: > >> Does this will help to calculate each PC on LAN MB usage report ?? > > > > You should be able to configure it to do so. IPAC-NG uses separate > > accounting rules for every item that you want to report on. Thus if > > you want individual accounting per PC, you can set it up to do so. > > > > Install it and fiddle. That's my recommendation. > > > > Alex Satrapa > > > > > > > > > > > -- Hand over the Slackware CD's and back AWAY from the computer, your geek rights have been revoked !!! Michael Gale Slackware user :) Bluesuperman.com From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Gale Subject: Re: Performance Monitoring Date: Mon, 5 Jan 2004 23:01:15 -0700 Sender: netfilter-admin@lists.netfilter.org Message-ID: <20040105230115.687bded9.michael@bluesuperman.com> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> <3FFA33C7.9010806@lintelsys.com.au> <20040106033802.30955.qmail@paus.pesat.net.id> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20040106033802.30955.qmail@paus.pesat.net.id> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii" To: netfilter@lists.netfilter.org Hello, Yesterday I was a reply on Performance Monitoring on the netfilter mail list, it suggested the user use IPAC-NG. The admin then have to create a chain for each IP they want to monitor. I did not think this is a good idea ... so for those of you who want to do bandwidth monitoring I suggest you check out the following. Here is a list of ones I have tried. ntop -- provides a web GUI for real time monitoring. Using it now on a firewall box to monitoring traffic on each interface. Adv .. provides great states , very detailed Dis .. seems to be some over header ... uses a DDR db :( You can use curl to pull the stats nightly and save them to a text file. Then create a little PHP scritp to provide you with the numbers. Now you will have stats for as long as you want. iptraf -- not bad ... detail is low. Adv ... NO over head and works great on a work station or 1 interface machine. It takes a bit to setup because you have to create all the filters your self. Dis ... out is simple ... a php script to produce a nice web GUI is needed. Nagios -- http://www.nagios.org/ Could be over kill depending on what you want ... this is more of a network monitoring tool. Really not designed to be run with one machine in mind. IPFM -- not bad .. very simple: example: HOST IN OUT TOTAL host1.domain.com 12345 6666684 6679029 MRTG for total traffic accounts only Bandwidthd -- not bad ... currently testing it. Seems to provide web png files much like MRTG but does provide host info. I do not believe you are able to save the data though :( Michael. On Tue, 06 Jan 2004 10:38:02 +0700 "bino" wrote: > I my self don't familiar with IPAC-NG. > The basic logic block is : > 1. use the feature of iptables -N to create per ip-addr IN and Out > chain 2. jump every traffic per ip addr, to respective chain > > use cron to run the bash-script that do : > 1. iptables -L -vnx > 2. Parse the data from each respective chain > 3. stor it to remote MySQL using MySql client tool > 4. reset (zero ?) the value of each chain > > That way you can have a traffic record per station (ip addr) > > If you just need monitoring like MRTG (in bps, no detailed history > record), it'll more simple ... you only need to hack NetSNMPD and use > MRTG to do the rest, no SQL hasle. > > Sincerely > -bino- > > Alex Satrapa writes: > > > Lawrence Tang wrote: > >> Does this will help to calculate each PC on LAN MB usage report ?? > > > > You should be able to configure it to do so. IPAC-NG uses separate > > accounting rules for every item that you want to report on. Thus if > > you want individual accounting per PC, you can set it up to do so. > > > > Install it and fiddle. That's my recommendation. > > > > Alex Satrapa > > > > > > > > > > > -- Hand over the Slackware CD's and back AWAY from the computer, your geek rights have been revoked !!! Michael Gale Slackware user :) Bluesuperman.com From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Gale Subject: Re: Performance Monitoring Date: Mon, 5 Jan 2004 23:02:33 -0700 Sender: netfilter-admin@lists.netfilter.org Message-ID: <20040105230233.27d84a49.michael@bluesuperman.com> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> <3FFA33C7.9010806@lintelsys.com.au> <20040106033802.30955.qmail@paus.pesat.net.id> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20040106033802.30955.qmail@paus.pesat.net.id> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii" To: netfilter@lists.netfilter.org Hello, Even if you have a script that creates the chains for each IP .. if you use all the IP's from .1 to .250. Then a packet will have to be compared to 249 chains before if matches a chain if it is from or to IP .250. This is not good. Michael. On Tue, 06 Jan 2004 10:38:02 +0700 "bino" wrote: > I my self don't familiar with IPAC-NG. > The basic logic block is : > 1. use the feature of iptables -N to create per ip-addr IN and Out > chain 2. jump every traffic per ip addr, to respective chain > > use cron to run the bash-script that do : > 1. iptables -L -vnx > 2. Parse the data from each respective chain > 3. stor it to remote MySQL using MySql client tool > 4. reset (zero ?) the value of each chain > > That way you can have a traffic record per station (ip addr) > > If you just need monitoring like MRTG (in bps, no detailed history > record), it'll more simple ... you only need to hack NetSNMPD and use > MRTG to do the rest, no SQL hasle. > > Sincerely > -bino- > > Alex Satrapa writes: > > > Lawrence Tang wrote: > >> Does this will help to calculate each PC on LAN MB usage report ?? > > > > You should be able to configure it to do so. IPAC-NG uses separate > > accounting rules for every item that you want to report on. Thus if > > you want individual accounting per PC, you can set it up to do so. > > > > Install it and fiddle. That's my recommendation. > > > > Alex Satrapa > > > > > > > > > > > -- Hand over the Slackware CD's and back AWAY from the computer, your geek rights have been revoked !!! Michael Gale Slackware user :) Bluesuperman.com From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ramin Dousti Subject: Re: Performance Monitoring Date: Fri, 9 Jan 2004 19:04:19 -0500 Sender: netfilter-admin@lists.netfilter.org Message-ID: <20040110000419.GA25191@cannon.eng.us.uu.net> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> <3FFA33C7.9010806@lintelsys.com.au> <20040106033802.30955.qmail@paus.pesat.net.id> <20040105230233.27d84a49.michael@bluesuperman.com> Mime-Version: 1.0 Return-path: Content-Disposition: inline In-Reply-To: <20040105230233.27d84a49.michael@bluesuperman.com> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Michael Gale Cc: netfilter@lists.netfilter.org On Mon, Jan 05, 2004 at 11:02:33PM -0700, Michael Gale wrote: > > Hello, > > Even if you have a script that creates the chains for each IP .. if you > use all the IP's from .1 to .250. Then a packet will have to be compared > to 249 chains before if matches a chain if it is from or to IP .250. One can come up with a btree which should reduce the worst case lookup to a max of 8 lookups for a /24. Ramin > > This is not good. > > Michael. From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Thhoep" Subject: Re: Performance Monitoring Date: Sat, 10 Jan 2004 09:54:04 +0100 Sender: netfilter-admin@lists.netfilter.org Message-ID: <000901c3d757$4d37e060$1684188d@Kiste> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> <3FFA33C7.9010806@lintelsys.com.au> <20040106033802.30955.qmail@paus.pesat.net.id> <20040105230233.27d84a49.michael@bluesuperman.com> <20040110000419.GA25191@cannon.eng.us.uu.net> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii" To: Ramin Dousti , Michael Gale Cc: netfilter@lists.netfilter.org > One can come up with a btree which should reduce the worst case lookup to a max > of 8 lookups for a /24. i did this once with a traffic counting system. works nice, still not perfect, but nice. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Satrapa Subject: Re: Performance Monitoring Date: Mon, 12 Jan 2004 10:26:16 +1100 Sender: netfilter-admin@lists.netfilter.org Message-ID: <4001DB98.9090607@lintelsys.com.au> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <3FF9E4B7.8010109@lintelsys.com.au> <011001c3d3f8$6a6a7e20$7700000a@lawrencewin2k> <3FFA33C7.9010806@lintelsys.com.au> <20040106033802.30955.qmail@paus.pesat.net.id> <20040105230233.27d84a49.michael@bluesuperman.com> <20040110000419.GA25191@cannon.eng.us.uu.net> Reply-To: netfilter@lists.netfilter.org Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20040110000419.GA25191@cannon.eng.us.uu.net> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: netfilter@lists.netfilter.org Ramin Dousti wrote: > One can come up with a btree which should reduce the worst case lookup to a max > of 8 lookups for a /24. It'd be better if netfilter supported some way of either binding rules to an interface, or allowing a hashtable-lookup for a "jump" based on IP address. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Antony Stone Subject: Re: Performance Monitoring Date: Sun, 11 Jan 2004 23:32:40 +0000 Sender: netfilter-admin@lists.netfilter.org Message-ID: <200401112332.40921.Antony@Soft-Solutions.co.uk> References: <004401c3d17c$4baa7cc0$0a01000a@xcom1> <20040110000419.GA25191@cannon.eng.us.uu.net> <4001DB98.9090607@lintelsys.com.au> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Return-path: In-Reply-To: <4001DB98.9090607@lintelsys.com.au> Errors-To: netfilter-admin@lists.netfilter.org List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Content-Type: text/plain; charset="us-ascii" To: netfilter@lists.netfilter.org On Sunday 11 January 2004 11:26 pm, Alex Satrapa wrote: > Ramin Dousti wrote: > > One can come up with a btree which should reduce the worst case lookup to > > a max of 8 lookups for a /24. > > It'd be better if netfilter supported some way of either binding rules > to an interface, or allowing a hashtable-lookup for a "jump" based on IP > address. It normally isn't much of a problem, because for most people, using the state match means that only the first packet of a new connection has to go through the ruleset looking for a rule to fnd out whether it's ACCEPTed or not - all future packets for the connection (assuming it gets ESTABLISHED) match on the very first rule and the whole system is quite efficient. Of course, if you're not using state matching then the above does not apply, but this is why statefulness is one of the good bits about netfilter. Antony. -- These clients are often infected by viruses or other malware and need to be fixed. If not, the user at that client needs to be fixed... - Henrik Nordstrom, on Squid user's mailing list Please reply to the list; please don't CC me.