Re: Bandwidth Monitoring

Re: Bandwidth Monitoring

[Arnt Karlsen]

On Fri, 29 Aug 2003 16:01:11 +0530, 
"Sathi" <sathiyan@gmx.net> wrote in message 
<04a901c36e18$ad2d6650$2a0110ac@SAMHP>:

> Hello All,
> 
> Is there any way to findout the bandwidth used by a client(LAN user),
> from Linux gateway server.
> 
> I have to take monthly statistics of bandwidth used by a client (LAN
> user).

..several, 'iptables -vnL' shows how much traffic etc per rule, 
setting up rules per user and 'iptables -vnL' every month etc 
is one way, there is MRTG, ntop, RRD etc.
 
> FYI i am using CBQ for bandwidth control

...for each user?  Try 'http://fmb.no/ipcop/setup-cbq-0.0.4.tar.bz2', 
I'm open for ideas etc on -0.0.5.  ;-)

-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;-)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.

Re: Bandwidth Monitoring

[Ray Leach]

[-- Attachment #1: Type: text/plain, Size: 662 bytes --]

On Fri, 2003-08-29 at 12:31, Sathi wrote:
> Hello All,
> 
> Is there any way to findout the bandwidth used by a client(LAN user), from
> Linux gateway server.
> 
Have a look at bwacct. It's real easy to setup and works well.
www.bwacct.org

> I have to take monthly statistics of bandwidth used by a client (LAN user).
> 
> FYI i am using CBQ for bandwidth control
> 
> Regards
> Sathi
-- 
--
Raymond Leach <raymondl@knowledgefactory.co.za>
Network Support Specialist
http://www.knowledgefactory.co.za
"lynx -source http://www.rchq.co.za/raymondl.asc | gpg --import"
Key fingerprint = 7209 A695 9EE0 E971 A9AD  00EE 8757 EE47 F06F FB28
--

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Bandwidth Monitoring

[Dharmendra.T]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

Hi,

But to use this we have to install other packages like apache, mysql
etc. Which will not be recommended to do on a firewall machine. If
bwacct can sniff the data from any other machine and give me the
statistics then that works out great for me.


Regards
Dharmendra.T
dharmu@nsecure.net




On Mon, 2003-09-01 at 13:03, Ray Leach wrote:

    On Fri, 2003-08-29 at 12:31, Sathi wrote:
    > Hello All,
    > 
    > Is there any way to findout the bandwidth used by a client(LAN user), from
    > Linux gateway server.
    > 
    Have a look at bwacct. It's real easy to setup and works well.
    www.bwacct.org
    
    > I have to take monthly statistics of bandwidth used by a client (LAN user).
    > 
    > FYI i am using CBQ for bandwidth control
    > 
    > Regards
    > Sathi
    -- 
    --
    Raymond Leach <raymondl@knowledgefactory.co.za>
    Network Support Specialist
    http://www.knowledgefactory.co.za
    "lynx -source http://www.rchq.co.za/raymondl.asc | gpg --import"
    Key fingerprint = 7209 A695 9EE0 E971 A9AD  00EE 8757 EE47 F06F FB28
    --

-- 
This message is intended for the addressee only. It may contain
privileged or Confidential information. If you have received this
message in error,please notify the sender and destroy the message
immediately.Unauthorised use or reproduction of this message is strictly
prohibited.

[-- Attachment #2: Type: text/html, Size: 3054 bytes --]

bandwidth monitoring

[patrick.leduc]

Hello

Does somebody know a program for monitoring bandwidth by ip? I have one 
internet interface and I must monitor many ips adresses on this interface. I 
tried Ipac-ng and, I worked a lot to do this config but it seems not working 
this way.

thanx

Re: bandwidth monitoring

[J. Nerius]

Hello,

I have had good luck with a program called bandwidthd, available at
http://bandwidthd.sourceforge.net/

I know it's not netfilter related specifically but I found it met my
needs where others failed. 

J.N.

On Thu, 2005-01-06 at 11:28 -0500, patrick.leduc@novipro.com wrote:
> Hello
> 
> Does somebody know a program for monitoring bandwidth by ip? I have one 
> internet interface and I must monitor many ips adresses on this interface. I 
> tried Ipac-ng and, I worked a lot to do this config but it seems not working 
> this way.
> 
> thanx
> 

Re: bandwidth monitoring

[Les Mikesell]

On Thu, 2005-01-06 at 10:28, patrick.leduc@novipro.com wrote:

> Does somebody know a program for monitoring bandwidth by ip? I have one 
> internet interface and I must monitor many ips adresses on this interface. I 
> tried Ipac-ng and, I worked a lot to do this config but it seems not working 
> this way.

Ntop might do what you need.  http://www.ntop.org.  It can summarize and
sort traffic by address/port/protocol, etc.  I don't use it continuously
but fire it up for a while if I think something is hogging the network.

-- 
  Les Mikesell
   les@futuresource.com

Re: bandwidth monitoring

[J. Nerius]

ntop is great for *short term* monitoring. Generally it will become
unmanageable if run for too long. If you want to monitor and keep stats
over a longer period of time, bandwidthd will probably work better for
you.

J.N.

On Thu, 2005-01-06 at 11:42 -0600, Les Mikesell wrote:
> On Thu, 2005-01-06 at 10:28, patrick.leduc@novipro.com wrote:
> 
> > Does somebody know a program for monitoring bandwidth by ip? I have one 
> > internet interface and I must monitor many ips adresses on this interface. I 
> > tried Ipac-ng and, I worked a lot to do this config but it seems not working 
> > this way.
> 
> Ntop might do what you need.  http://www.ntop.org.  It can summarize and
> sort traffic by address/port/protocol, etc.  I don't use it continuously
> but fire it up for a while if I think something is hogging the network.
> 

Re: bandwidth monitoring

[Michael Gale]

Hello,

	To run NTOP permanitly you just have to clear the stats nightly. I have 
a php script that makes a HTTP connection, downloads the stats from NTOP 
and saves them in a mysql database. It then makes another connection and 
tells NTOP to reset the stats.

That keeps the memory and CPU usage low ... so you could run it 
continuously.

Michael.


patrick.leduc@novipro.com wrote:
> Hello
> 
> Does somebody know a program for monitoring bandwidth by ip? I have one 
> internet interface and I must monitor many ips adresses on this interface. I 
> tried Ipac-ng and, I worked a lot to do this config but it seems not working 
> this way.
> 
> thanx
> 

-- 
Michael Gale
Lan Administrator
Utilitran Corp.

I make better friends with those who think for them selves

Re: bandwidth monitoring

[J. Nerius]

How many hosts and how much traffic are you running through it? I've
wanted to come up with a solution similar to the one you've described to
replace my current bandwidthd setup but I'm thinking that my network may
be too large with too much traffic to support something like that
without building a monster box just to capture the stats. 

J.N.

On Thu, 2005-01-06 at 13:09 -0700, Michael Gale wrote:
> Hello,
> 
> 	To run NTOP permanitly you just have to clear the stats nightly. I have 
> a php script that makes a HTTP connection, downloads the stats from NTOP 
> and saves them in a mysql database. It then makes another connection and 
> tells NTOP to reset the stats.
> 
> That keeps the memory and CPU usage low ... so you could run it 
> continuously.
> 
> Michael.
> 
> 
> patrick.leduc@novipro.com wrote:
> > Hello
> > 
> > Does somebody know a program for monitoring bandwidth by ip? I have one 
> > internet interface and I must monitor many ips adresses on this interface. I 
> > tried Ipac-ng and, I worked a lot to do this config but it seems not working 
> > this way.
> > 
> > thanx
> > 
> 

Re: bandwidth monitoring

[Michael Gale]

Hello,

	I am not sure I follow ... NTOP takes care of providing the stats, so 
if you firewall box or ntop server can handle your network.

The storage box can be what and where ever ... for example you could 
have a low end box somewhere on the network that makes the http 
connection, downloads the stats and stores them in a database.

Then you only need a simple method to display them when needed. If you 
are a large company it might be easier to setup multiple ntop servers 
and have all the stats stored in a central location.

Michael.



J. Nerius wrote:
> How many hosts and how much traffic are you running through it? I've
> wanted to come up with a solution similar to the one you've described to
> replace my current bandwidthd setup but I'm thinking that my network may
> be too large with too much traffic to support something like that
> without building a monster box just to capture the stats. 
> 
> J.N.
> 
> On Thu, 2005-01-06 at 13:09 -0700, Michael Gale wrote:
> 
>>Hello,
>>
>>	To run NTOP permanitly you just have to clear the stats nightly. I have 
>>a php script that makes a HTTP connection, downloads the stats from NTOP 
>>and saves them in a mysql database. It then makes another connection and 
>>tells NTOP to reset the stats.
>>
>>That keeps the memory and CPU usage low ... so you could run it 
>>continuously.
>>
>>Michael.
>>
>>
>>patrick.leduc@novipro.com wrote:
>>
>>>Hello
>>>
>>>Does somebody know a program for monitoring bandwidth by ip? I have one 
>>>internet interface and I must monitor many ips adresses on this interface. I 
>>>tried Ipac-ng and, I worked a lot to do this config but it seems not working 
>>>this way.
>>>
>>>thanx
>>>
>>
> 

-- 
Michael Gale
Lan Administrator
Utilitran Corp.

I make better friends with those who think for them selves

Re: bandwidth monitoring

[J. Nerius]

Hello, 

My concern was for the amount of resources NTOP would end up using with
the currently volume of traffic we have going through any one link. 

Basically, there will be at least 3000 hosts totaling around 25 megabits
of traffic at any given time. In my experience with ntop for short-term
monitoring, I found that in these circumstances, the process size and
cpu utilization were quite high.

I've been using bandwidthd to collect data and then just rsyncing that
data to a different server that actually graphs the data. I'd like to
use ntop for it's detailed tracking capabilities, but given the resource
utilization issues I've seen in the past, am not sure how feasible this
would be. 

How many hosts and how much traffic is currently going through your
setup? It's very possible that I may just need to tweak settings
somewhere to improve performance. 

J.N.


On Thu, 2005-01-06 at 14:28 -0700, Michael Gale wrote:
> Hello,
> 
> 	I am not sure I follow ... NTOP takes care of providing the stats, so 
> if you firewall box or ntop server can handle your network.
> 
> The storage box can be what and where ever ... for example you could 
> have a low end box somewhere on the network that makes the http 
> connection, downloads the stats and stores them in a database.
> 
> Then you only need a simple method to display them when needed. If you 
> are a large company it might be easier to setup multiple ntop servers 
> and have all the stats stored in a central location.
> 
> Michael.
> 
> 
> 
> J. Nerius wrote:
> > How many hosts and how much traffic are you running through it? I've
> > wanted to come up with a solution similar to the one you've described to
> > replace my current bandwidthd setup but I'm thinking that my network may
> > be too large with too much traffic to support something like that
> > without building a monster box just to capture the stats. 
> > 
> > J.N.
> > 
> > On Thu, 2005-01-06 at 13:09 -0700, Michael Gale wrote:

Re: bandwidth monitoring

[Michael Gale]

Ahhh .. I see now. Our setup is rather small and we only have about 10 
megabits max at any given time -- usually much less.

Here is the command I use to start ntop:
ntop -4 -a /home/ntopwatch/webaccess --daemon -i eth1,eth2 -o -n -p 
userprolist -s -u ntopwatch -w 192.168.7.1:3000 -M -P /home/ntopwatch/

My userprolist:
FTP=ftp|ftp-data
HTTP=http|www|https|3128
DIFFHTTP=8080|8888
RDP=3389
DNS=name|domain
NBios-IP=netbios-ns|netbios-dgm|netbios-ssn
Mail=pop-2|pop-3|pop3|kpop|smtp|imap|imap2
DHCP-BOOTP=67-68
SNMP=snmp|snmp-trap
NNTP=nntp
SSH=22
Gnutella=6346|6347|6348
Kazaa=1214
WinMX=6699|7730
DirectConnect=0
eDonkey=4661-4665
Messenger=1863|5000|5001|5190-5193

But the box which is running ntop is:
Dual PIII 1.4Ghz with 1GB of RAM.

Michael.

J. Nerius wrote:
> Hello, 
> 
> My concern was for the amount of resources NTOP would end up using with
> the currently volume of traffic we have going through any one link. 
> 
> Basically, there will be at least 3000 hosts totaling around 25 megabits
> of traffic at any given time. In my experience with ntop for short-term
> monitoring, I found that in these circumstances, the process size and
> cpu utilization were quite high.
> 
> I've been using bandwidthd to collect data and then just rsyncing that
> data to a different server that actually graphs the data. I'd like to
> use ntop for it's detailed tracking capabilities, but given the resource
> utilization issues I've seen in the past, am not sure how feasible this
> would be. 
> 
> How many hosts and how much traffic is currently going through your
> setup? It's very possible that I may just need to tweak settings
> somewhere to improve performance. 
> 
> J.N.
> 
> 
> On Thu, 2005-01-06 at 14:28 -0700, Michael Gale wrote:
> 
>>Hello,
>>
>>	I am not sure I follow ... NTOP takes care of providing the stats, so 
>>if you firewall box or ntop server can handle your network.
>>
>>The storage box can be what and where ever ... for example you could 
>>have a low end box somewhere on the network that makes the http 
>>connection, downloads the stats and stores them in a database.
>>
>>Then you only need a simple method to display them when needed. If you 
>>are a large company it might be easier to setup multiple ntop servers 
>>and have all the stats stored in a central location.
>>
>>Michael.
>>
>>
>>
>>J. Nerius wrote:
>>
>>>How many hosts and how much traffic are you running through it? I've
>>>wanted to come up with a solution similar to the one you've described to
>>>replace my current bandwidthd setup but I'm thinking that my network may
>>>be too large with too much traffic to support something like that
>>>without building a monster box just to capture the stats. 
>>>
>>>J.N.
>>>
>>>On Thu, 2005-01-06 at 13:09 -0700, Michael Gale wrote:
> 
> 
> 

-- 
Michael Gale
Lan Administrator
Utilitran Corp.

I make better friends with those who think for them selves

Re: bandwidth monitoring

[Michael Gale]

No problem ...

	I just checked and actually I am using two scripts.

The first script:
--snip-- runntop -- executed by cron.
#!/bin/bash

/usr/bin/curl -k -s -s --digest -u ntopUP:ntopUP 
https://mis.utilitran.com/uploadscripts/ntopget.php > /dev/null

if [ "$?" == "0" ]; then
echo "File loaded" >> /dev/null
else
~/email "Error with NTOP update" netops@utilitran.com < /dev/null
fi

sleep 5

/usr/bin/curl --basic -u admin:ntopadmin 
http://ntop.utilitran.com:3000/ntop/resetStats.html

if [ "$?" == "0" ]; then
echo "File loaded" >> /dev/null
else
~/email "Error with NTOP reset" netops@utilitran.com < /dev/null
fi
--snip--

The first script makes a http request to a php web page that display no 
data. Then once that is successful it makes a http connections to ntop 
and resets the stats.

The second script ... that the first script calls, uses curl to make a 
http connection to the ntop server, download the stats and store them in 
a database. The scripts are kind of messy ... really my first attempt. 
You may notice that data_total is not being used .. that is because I am 
working on storing the total amounts somewhere .. but that is not done.


The second script:
--snip-- ntopget.php -- executed by http request
<?php
//
// The PHP curl module supports the received page to be returned in a 
variable
// if told.
//

$dat=date('m:d:y-U');
$startphp="<?php\n";
$endphp="?>";

echo 'Preparing to download file<BR>';
$fp = fopen('/home/mis/htdocs/datafiles/ntop/data/' . $dat, 'w');
fwrite($fp,$startphp,strlen($startphp));
fclose($fp);

$fp = fopen('/home/mis/htdocs/datafiles/ntop/data/' . $dat, 'a');
$ch = 
curl_init("http://ntop.utilitran.com:3000/ntop/dumpData.html?language=php&view=long");

curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);

fclose($fp);

$fp = fopen('/home/mis/htdocs/datafiles/ntop/data/' . $dat, 'a');
fwrite($fp,$endphp,strlen($endphp));
fclose($fp);

echo 'Done downloading file<BR>';

include '/home/mis/htdocs/datafiles/ntop/data/' . $dat;

// Start of using array

               $db = mysql_connect("127.0.0.1", "ntop", "ntop") or 
die("Error ... check with network team");
               mysql_select_db("ntop",$db);

foreach ($ntopHash as $key => $value) {

         $addme = strpos($key,"192.168.7");
         $data = array();

         if ( $addme !== FALSE )
         {

array_push($data,$key);
array_push($data,$ntopHash[$key]['hostResolvedName']);
array_push($data,$dat);
array_push($data,$ntopHash[$key]['ethAddressString']);
array_push($data,$ntopHash[$key]['firstSeen']);
array_push($data,$ntopHash[$key]['lastSeen']);
array_push($data,$ntopHash[$key]['averageRcvdThpt']);
array_push($data,$ntopHash[$key]['averageSentThpt']);
array_push($data,$ntopHash[$key]['averageTThpt']);
array_push($data,$ntopHash[$key]['icmpSent']);
array_push($data,$ntopHash[$key]['icmpRcvd']);
array_push($data,$ntopHash[$key]['IP']['FTP']['sendLoc']);
array_push($data,$ntopHash[$key]['IP']['FTP']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['FTP']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['FTP']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['HTTP']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['HTTP']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['HTTP']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['HTTP']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['DIFFHTTP']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['DIFFHTTP']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['DIFFHTTP']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['DIFFHTTP']['rcvdFromRem']);

array_push($data,$ntopHash[$key]['IP']['RDP']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['RDP']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['RDP']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['RDP']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['DNS']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['DNS']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['DNS']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['DNS']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['Mail']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['Mail']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['Mail']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['Mail']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['DHCP-BOOTP']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['DHCP-BOOTP']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['DHCP-BOOTP']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['DHCP-BOOTP']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['SNMP']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['SNMP']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['SNMP']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['SNMP']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['NNTP']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['NNTP']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['NNTP']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['NNTP']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['SSH']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['SSH']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['SSH']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['SSH']['rcvdFromRem']);
array_push($data,$ntopHash[$key]['IP']['Messenger']['sentLoc']);
array_push($data,$ntopHash[$key]['IP']['Messenger']['rcvdLoc']);
array_push($data,$ntopHash[$key]['IP']['Messenger']['sentRem']);
array_push($data,$ntopHash[$key]['IP']['Messenger']['rcvdFromRem']);


         foreach ( $data as $key => $value ) {

         $data_table = $data_table . '\'' . $value . '\',';

         }

         for ($i=0;$i<6;$i++) {

         $data_total = $data_total . '\'' . $data_table[$i] . '\',';

         }


         $data_table = substr($data_table,0,(strlen($data_table)-1));
         $data_total = substr($data_total,0,(strlen($data_total)-1));

                 // Here we upload the data into the daily table

                 $sql = "INSERT INTO ntop_daily VALUES ($data_table)";

                 $result = mysql_query($sql) or die("Error ..... check 
with network team " . mysql_error());

                 if ( $result === FALSE )
                 {
                 exec('/home/mis/email "NTOP Nightly Update FAILED" "The 
NTOP nightly update failed" \&/> /dev/null');
                 }

         }
unset($data_table);
unset($data_total);
unset($data);
unset($result);

}

               mysql_close($db);

?>
--snip--




Marek Dohojda wrote:
> Can I see that script?
> 
> 
> 
> 
> 
> ---
> 01001101011000010111001001100101011010110010000001000100010011110110100001101111011010100110010001100001 
> 
> 
> Michael Gale wrote:
> 
>> Hello,
>>
>>     To run NTOP permanitly you just have to clear the stats nightly. I 
>> have a php script that makes a HTTP connection, downloads the stats 
>> from NTOP and saves them in a mysql database. It then makes another 
>> connection and tells NTOP to reset the stats.
>>
>> That keeps the memory and CPU usage low ... so you could run it 
>> continuously.
>>
>> Michael.
>>
>>
>> patrick.leduc@novipro.com wrote:
>>
>>> Hello
>>>
>>> Does somebody know a program for monitoring bandwidth by ip? I have 
>>> one internet interface and I must monitor many ips adresses on this 
>>> interface. I tried Ipac-ng and, I worked a lot to do this config but 
>>> it seems not working this way.
>>>
>>> thanx
>>>
>>

-- 
Michael Gale
Lan Administrator
Utilitran Corp.

I make better friends with those who think for them selves

RE: bandwidth monitoring

[Mark E. Donaldson]

 
http://bandwidthd.sourceforge.net/


-----Original Message-----
From: netfilter-bounces@lists.netfilter.org
[mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of
patrick.leduc@novipro.com
Sent: Thursday, January 06, 2005 8:28 AM
To: netfilter@lists.netfilter.org
Subject: bandwidth monitoring

Hello

Does somebody know a program for monitoring bandwidth by ip? I have one
internet interface and I must monitor many ips adresses on this interface. I
tried Ipac-ng and, I worked a lot to do this config but it seems not working
this way.

thanx


########################################################
This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.

postmaster@bandwidthco.com
MailScanner at bandwidthco.com is for your absolute protection.
########################################################




########################################################
This message has been scanned for viruses and dangerous 
content by MailScanner, and is believed to be clean.

postmaster@bandwidthco.com
MailScanner at bandwidthco.com is for your absolute protection.
########################################################

RE: bandwidth monitoring

[Fabiano Reis]

I´m acctualy using hotsanic to make this. 

http://hotsanic.sourceforge.net/

it have features to monitore other things as well...



-----Original Message-----
From: netfilter-bounces@lists.netfilter.org
[mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of
patrick.leduc@novipro.com
Sent: Thursday, January 06, 2005 2:28 PM
To: netfilter@lists.netfilter.org
Subject: bandwidth monitoring

Hello

Does somebody know a program for monitoring bandwidth by ip? I have one 
internet interface and I must monitor many ips adresses on this interface. I

tried Ipac-ng and, I worked a lot to do this config but it seems not working

this way.

thanx

Re: bandwidth monitoring

[Ranjeet Shetye]

* patrick.leduc@novipro.com (patrick.leduc@novipro.com) wrote:
> Hello
> 
> Does somebody know a program for monitoring bandwidth by ip? I have one 
> internet interface and I must monitor many ips adresses on this interface. I 
> tried Ipac-ng and, I worked a lot to do this config but it seems not working 
> this way.
> 
> thanx
> 

Long term bandwidth usage can be tracked using MRTG. (stored data +
daemon + http server + web based gui)

Instantaneous bandwidth usage can be tracked using iftop. (ncurses)

-- 
Ranjeet Shetye
Senior Software Engineer
Zultys Technologies
Ranjeet dot Shetye at Zultys dot com
http://www.zultys.com/
 
The views, opinions, and judgements expressed in this message are solely those of
the author. The message contents have not been reviewed or approved by Zultys.

Re: bandwidth monitoring

[Jose Maria Lopez]

El mié, 26 de 01 de 2005 a las 19:33, Ranjeet Shetye escribió:
> * patrick.leduc@novipro.com (patrick.leduc@novipro.com) wrote:
> > Hello
> > 
> > Does somebody know a program for monitoring bandwidth by ip? I have one 
> > internet interface and I must monitor many ips adresses on this interface. I 
> > tried Ipac-ng and, I worked a lot to do this config but it seems not working 
> > this way.
> > 
> > thanx
> > 
> 
> Long term bandwidth usage can be tracked using MRTG. (stored data +
> daemon + http server + web based gui)
> 
> Instantaneous bandwidth usage can be tracked using iftop. (ncurses)

You can try our bastion-firewall-stats-addon, it's GPL and it uses
rrdtool to do the graphical stats. It's very easy to use, but you
have to use bastion-firewall to use it.

Another good option it's using something like Cacti to have an
interface to Rrdtool. It has scripts to monitor interfaces and
can do all kind of graphs.

Regards.

-- 
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
jkerouac@bgsec.com
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑA

The only people for me are the mad ones -- the ones who are mad to live,
mad to talk, mad to be saved, desirous of everything at the same time,
the ones who never yawn or say a commonplace thing, but burn, burn, burn
like fabulous yellow Roman candles.
                -- Jack Kerouac, "On the Road"

RE: bandwidth monitoring

[Daniel Chemko]

J. Nerius wrote:
> ntop is great for *short term* monitoring. Generally it will become
> unmanageable if run for too long. If you want to monitor and keep
> stats over a longer period of time, bandwidthd will probably work
> better for you.
> 
> J.N.
> 
> On Thu, 2005-01-06 at 11:42 -0600, Les Mikesell wrote:
>> On Thu, 2005-01-06 at 10:28, patrick.leduc@novipro.com wrote:
>> 
>>> Does somebody know a program for monitoring bandwidth by ip? I have
>>> one internet interface and I must monitor many ips adresses on this
>>> interface. I tried Ipac-ng and, I worked a lot to do this config
>>> but it seems not working this way.
>> 
>> Ntop might do what you need.  http://www.ntop.org.  It can summarize
>> and sort traffic by address/port/protocol, etc.  I don't use it
>> continuously but fire it up for a while if I think something is
>> hogging the network. 

I'll address this as well. Ntop is fantastic at giving you snapshot data
of a network, but it is inanely heavy at long term monitoring of
services. It got to the point that monitoring traffic from the firewall
filled memory and CPU usage if run long enough. It doesn't work for
continuous operations.

The one really good thing about iptables is that every rule has a
counter fo the number of hits that you run through it, so it  is
possible to create custom counters for your software. This is not a
'simple' process, but it'll give you accurate traffic flows with
filtering, etc. that a normal libcap based tool can't give you. PS:
/proc/net/dev data is incorrect when netfilter & NAT are enabled. I
believe its because NAT (return?) traffic bypass this counter, so any
management tool that uses this technique for monitoring bandwith will
also be flawed on a netfilter router.

RE: bandwidth monitoring

[Les Mikesell]

On Thu, 2005-01-06 at 13:04, Daniel Chemko wrote:

> I'll address this as well. Ntop is fantastic at giving you snapshot data
> of a network, but it is inanely heavy at long term monitoring of
> services. It got to the point that monitoring traffic from the firewall
> filled memory and CPU usage if run long enough. It doesn't work for
> continuous operations.

The thing to keep in mind is that it is keeping a database that
includes all of the remote IP addresses and ports as well as the
local ones, so of course this will grow quickly.  I haven't needed
to deal with long-term histories so I haven't investigated the
possibilities but I do recall something about ntop being able
to start new logs and archive the old ones as needed to limit
the active size.  There are tools to monitor and summarize
netflows between each pair of addresses, but again I haven't
used that part of the package.  There is also some support for
RRD databases which would probably be the way to go for longer
histories.

-- 
  Les Mikesell
   les@futuresource.com

RE: bandwidth monitoring

[Daniel Chemko]

J. Nerius wrote:
> How many hosts and how much traffic are you running through it? I've
> wanted to come up with a solution similar to the one you've described
> to replace my current bandwidthd setup but I'm thinking that my
> network may be too large with too much traffic to support something
> like that without building a monster box just to capture the stats.
> 

If you have a small static number of hosts in/out of your system, you
may want to use netfilter blank rule counters since the penalty of
passing each counter is very very low (entirely kernel side).

To put this in perspective, there've been a lot of performance issues
with people running 10000+ rule sites with adverse effects on their
network setup. Lower than that, and the impact is pretty low. Plus,
blank rules don't do anything but increment the counter, so the actual
CPU utilization of these rules are even lower. This is to give maxumum
accounting of an existing kernel. I'm sure there have been a few
in-kernel accounting packages made, but I can't recall any at the
moment. Maybe someone here can refresh our memory.

Of course the problem with this approach is that you have to know what
IP's that are generating traffic before setting this thing up since the
iptables rules are static. Its good if you want to monitor internal
user's traffic to the net and the amount of traffic a server is getting,
but to actually track the internet endpoints, you're better off using a
dynamic traffic tracking tools like ntop or bandwidthd.

Re: bandwidth monitoring

[Patrich Björklund]

Hi, there. I dont really know if you just wanna know what ips gets
to/through your interfaces? I have a prog called tcpick. It show like
this:

root # tcpick -i eth0 -C
Starting tcpick 0.1.19
tcpick: listening on eth0
SYN-SENT       192.168.0.2:45190 > 217.215.148.17:pop3
SYN-RECEIVED   192.168.0.2:45190 > 217.215.148.17:pop3
ESTABLISHED    192.168.0.2:45190 > 217.215.148.17:pop3
FIN-WAIT-1     192.168.0.2:45190 > 217.215.148.17:pop3
FIN-WAIT-2     192.168.0.2:45190 > 217.215.148.17:pop3

>Hello
>
>Does somebody know a program for monitoring bandwidth by ip? I have
>one 
>internet interface and I must monitor many ips adresses on this
>interface. I 
>tried Ipac-ng and, I worked a lot to do this config but it seems not
>working 
>this way.
>
>thanx



------------------------------