IP Traffic Accounting

IP Traffic Accounting

[yomega]

Hi List,

i want to set up IP Traffic Accounting. I wanna to measure the Traffic which
comes in and goes out on specified Ports.

After reading some IPTABLES Manuals and testing some with my Linux
Fileserver, i thought of doing exactly this by that way:
I create Rules to Log the specified Ports: iptables ..... -j
log --log-prefix [name] <- this one :) Because of my syslog Settings, the
Packets are logged into /var/log/firewall
Ok now i make a Cron with a little Python or PHP Script that analyses the
logged packages, and flushes the Log File empty :). The Cron is executed
every 5 minutes.

OK this should work, but i still got a question:

First i start to realize that my log file will become very big with even
little outbound traffic. Now the cron is exectued and all the data written
on the hd is analysed and written on the hd. I'm afraid that this will
injure the health of the HD. Is there any other maybe more "clean"
possibility to do that loggin? or have i made some mistakes in my thoughts?

Maybe everthing happens in the RAM (syslog and analysing)? So this would not
be a hd health prob?

Thanx 4 all suggestions :)

Greetz,
Stephan

Re: IP Traffic Accounting

[David B Harris]

[-- Attachment #1: Type: text/plain, Size: 1273 bytes --]

On Sat, 22 Jun 2002 16:02:06 +0200
"yomega" <yomega@wahooo.net> wrote:
> OK this should work, but i still got a question:
> 
> First i start to realize that my log file will become very big with
> even little outbound traffic. Now the cron is exectued and all the
> data written on the hd is analysed and written on the hd. I'm afraid
> that this will injure the health of the HD. Is there any other maybe
> more "clean" possibility to do that loggin? or have i made some
> mistakes in my thoughts?

I'd suggest you use iptable's byte-counting instead. 'iptables -L -n -v
-x' will list the bytes which have _crossed_ each given rule. (So it
won't just count which packets have matched.)

Probably be a lot faster ;)

P.S.: In production, don't forget to use -Z to zero the counters, so you
aren't counting the same packets twice.

-- 
________________________________________________________________________
\ David B. Harris, Systems administrator   |   http://www.terrabox.com /
/  eelf@sympatico.ca, elf@terrabox.com     |     http://eelf.ddts.net  \
\======================================================================/
/ Clan Barclay motto: Aut agere, aut mori.  (Either action, or death.) \
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: IP Traffic Accounting

[Antony Stone]

On Saturday 22 June 2002 3:00 pm, David B Harris wrote:

> I'd suggest you use iptable's byte-counting instead. 'iptables -L -n -v -x'

Thanks, David - I forgot the '-x' in my version, and this makes the numbers 
an awful lot easier to process :-)

> will list the bytes which have _crossed_ each given rule. (So it
> won't just count which packets have matched.)

I'm not sure I agree with this - I believe the byte / packet counters only 
count packets matched by the rule, so that if you have the rules:

iptables -A FORWARD -p tcp --dport 25 -j ACCEPT
iptables -A FORWARD -p tcp --dport 110 -j ACCEPT

the counters for the first one will only show you SMTP traffic, and the 
counters for the second one will only show you POP3 traffic.

 

Antony.

Re: IP Traffic Accounting

[Leonardo Rodrigues]

    Usually, when you need to read traffic counters on iptables/ipchains,
you create what we called an 'accounting rule', that means, a rule that
matchs what you need it to match but it has NO action.

    example: iptables -A forward -p tcp --dport 25

this will create a rule that will only count packets/bytes and does not with
it. Of course if you need to allow/deny it, you'll need to deny/allow it
latter on another rule.

    So, if person has one rule for accounting and other for allow/deny the
packet, it can be counted YES in two different places. But this will happen
ONLY IF we have an accounting rule for it, that means, a rule that does
nothing but count the packet.

    If you're getting the counters directly on the ACCEPT/DENY rule, you'll
really dont have problems on packets being counted twice, because after
reaching the accept/deny rule packet will stop trying to be matched on other
rules.

    Sincerily,
    Leonardo Rodrigues

----- Original Message -----
From: "Antony Stone" <Antony@Soft-Solutions.co.uk>
To: <netfilter@lists.samba.org>
Sent: Saturday, June 22, 2002 11:09 AM
Subject: Re: IP Traffic Accounting


>
> I'm not sure I agree with this - I believe the byte / packet counters only
> count packets matched by the rule, so that if you have the rules:
>
> iptables -A FORWARD -p tcp --dport 25 -j ACCEPT
> iptables -A FORWARD -p tcp --dport 110 -j ACCEPT
>

AW: IP Traffic Accounting / Own Chains

[yomega]

Hi List,

Thank you really much for all the help :) Now everthing is workin fine...not
really everything *G*

OK IP Counting is no problem :) It was a problem but because of your nice
and workin answeres the silly one writing this mail got it :)

Now i plan to set up accounting for an CS Server. First aof all i wanna make
an own chain for this Server, that would be much easier to count packets for
different servers and things.

OK so i do:
iptables -N [CHAIN NAME]
and then add rules for example this one:
iptables -A [CHAIN NAME] -p tcp --dport 80 -j ACCEPT
so now i've made some traffic on my webbie and typed in that:

iptables -L -n -v -x

now the list is printed :) INPUT and the other Chains are correct but this
chain (my own one) didn count anything.

Maybe i just haven't understand what i'm doin :) or i'am silly *G*

So it be real nice if you help this silly man writing this mail *G*

Greetings and a really nice day :)

Stephan

> -----Ursprüngliche Nachricht-----
> Von: netfilter-admin@lists.samba.org
> [mailto:netfilter-admin@lists.samba.org]Im Auftrag von Antony Stone
>
>
> On Saturday 22 June 2002 3:00 pm, David B Harris wrote:
>
> > I'd suggest you use iptable's byte-counting instead. 'iptables
> -L -n -v -x'
>
> Thanks, David - I forgot the '-x' in my version, and this makes
> the numbers
> an awful lot easier to process :-)
>
> > will list the bytes which have _crossed_ each given rule. (So it
> > won't just count which packets have matched.)
>
> I'm not sure I agree with this - I believe the byte / packet
> counters only
> count packets matched by the rule, so that if you have the rules:
>
> iptables -A FORWARD -p tcp --dport 25 -j ACCEPT
> iptables -A FORWARD -p tcp --dport 110 -j ACCEPT
>
> the counters for the first one will only show you SMTP traffic, and the
> counters for the second one will only show you POP3 traffic.
>
>
>
> Antony.
>
>

Re: AW: IP Traffic Accounting / Own Chains

[Antony Stone]

On Saturday 22 June 2002 4:18 pm, yomega wrote:

> OK so i do:
> iptables -N [CHAIN NAME]
> and then add rules for example this one:
> iptables -A [CHAIN NAME] -p tcp --dport 80 -j ACCEPT
> so now i've made some traffic on my webbie and typed in that:
>
> iptables -L -n -v -x
>
> now the list is printed :) INPUT and the other Chains are correct but this
> chain (my own one) didn count anything.

Did you put a rule anywhere which says

iptables -A FORWARD ....... -j [CHAIN NAME]

so that packets get fed into your new chain ?

 

Antony.

AW: AW: IP Traffic Accounting / Own Chains

[yomega]

Hi PPL,

WAHOOOOOoo *startsdancing*

now its doin fine :)

THX THX THX THX THX :)

Greetz,
stephan

> -----Ursprüngliche Nachricht-----
> Von: netfilter-admin@lists.samba.org
> [mailto:netfilter-admin@lists.samba.org]Im Auftrag von Antony Stone
> Gesendet: Samstag, 22. Juni 2002 17:18
> An: netfilter@lists.samba.org
> Betreff: Re: AW: IP Traffic Accounting / Own Chains
>
>
> On Saturday 22 June 2002 4:18 pm, yomega wrote:
>
> > OK so i do:
> > iptables -N [CHAIN NAME]
> > and then add rules for example this one:
> > iptables -A [CHAIN NAME] -p tcp --dport 80 -j ACCEPT
> > so now i've made some traffic on my webbie and typed in that:
> >
> > iptables -L -n -v -x
> >
> > now the list is printed :) INPUT and the other Chains are
> correct but this
> > chain (my own one) didn count anything.
>
> Did you put a rule anywhere which says
>
> iptables -A FORWARD ....... -j [CHAIN NAME]
>
> so that packets get fed into your new chain ?
>
>
>
> Antony.
>
>

Re: IP Traffic Accounting

[Antony Stone]

On Saturday 22 June 2002 3:02 pm, yomega wrote:

> Hi List,
>
> i want to set up IP Traffic Accounting. I wanna to measure the Traffic
> which comes in and goes out on specified Ports.
>
> After reading some IPTABLES Manuals and testing some with my Linux
> Fileserver, i thought of doing exactly this by that way:
> I create Rules to Log the specified Ports: iptables ..... -j
> log --log-prefix [name] <- this one :) Because of my syslog Settings, the
> Packets are logged into /var/log/firewall
> Ok now i make a Cron with a little Python or PHP Script that analyses the
> logged packages, and flushes the Log File empty :). The Cron is executed
> every 5 minutes.
>
> OK this should work, but i still got a question:
>
> First i start to realize that my log file will become very big with even
> little outbound traffic. Now the cron is exectued and all the data written
> on the hd is analysed and written on the hd. I'm afraid that this will
> injure the health of the HD. Is there any other maybe more "clean"
> possibility to do that loggin? or have i made some mistakes in my thoughts?

I *really* wouldn't worry about the health of your hard disk - reading & 
writing data is what hard disks are made for :-)

However, if you want to keep the size of the log files down, you could try 
just creating some rules whcih match the packets you're interested in (maybe 
you have some already ?) which simply ACCEPT / DROP / REJECT the packets, and 
then you can see how many packets & bytes got matched by those rules with

iptables -L -n -v

That way you can see the numbers whenever you want to, and calculate traffic 
over a given time period by subtraction.

 

Antony.

Re: IP Traffic Accounting

[Jason R. Martin]

I think most of the other posts answered your questions with respect to
netfilter.  I just wanted to offer an alternative for IP accounting.  Check
out ipaudit (http://ipaudit.sourceforge.net).  Assuming you have the pcap
libraries installed on your system, it will keep track of all connections
to/from your system in a nice compact format.  Since it keeps track of
connections instead of individual packets, it will likely tell you all the
accounting info you want (packet counts, byte counts, host IPs, ports, etc).
There's even a nifty web interface that you can use to view the data.

Just a thought.

Jason

On Sat, Jun 22, 2002 at 04:02:06PM +0200, yomega wrote:
> Hi List,
> 
> i want to set up IP Traffic Accounting. I wanna to measure the Traffic which
> comes in and goes out on specified Ports.
> 
> After reading some IPTABLES Manuals and testing some with my Linux
> Fileserver, i thought of doing exactly this by that way:
> I create Rules to Log the specified Ports: iptables ..... -j
> log --log-prefix [name] <- this one :) Because of my syslog Settings, the
> Packets are logged into /var/log/firewall
> Ok now i make a Cron with a little Python or PHP Script that analyses the
> logged packages, and flushes the Log File empty :). The Cron is executed
> every 5 minutes.
> 
> OK this should work, but i still got a question:
> 
> First i start to realize that my log file will become very big with even
> little outbound traffic. Now the cron is exectued and all the data written
> on the hd is analysed and written on the hd. I'm afraid that this will
> injure the health of the HD. Is there any other maybe more "clean"
> possibility to do that loggin? or have i made some mistakes in my thoughts?
> 
> Maybe everthing happens in the RAM (syslog and analysing)? So this would not
> be a hd health prob?
> 
> Thanx 4 all suggestions :)
> 
> Greetz,
> Stephan

Re: IP Traffic Accounting

[Michael]

Jason R. Martin wrote:

>I think most of the other posts answered your questions with respect to
>netfilter.  I just wanted to offer an alternative for IP accounting.  Check
>out ipaudit (http://ipaudit.sourceforge.net).  Assuming you have the pcap
>libraries installed on your system, it will keep track of all connections
>to/from your system in a nice compact format.  Since it keeps track of
>connections instead of individual packets, it will likely tell you all the
>accounting info you want (packet counts, byte counts, host IPs, ports, etc).
>There's even a nifty web interface that you can use to view the data.
>
>Just a thought.
>
That's a good suggestion , Jason.

I have had experience with similar programs, for example ipmeter 
(http://www.ipmeter.org).

The main problems with this way of doing things are:

- If the host you run the IPaudit program on is also a firewall, by 
virtue of the  fact that the program  requires promiscuous mode on the 
interface it listens on introduces a (theoretical) security risk.

- An alternative to running the IPaudit on the firewalling host is to 
have a seperate standalone machine running IPaudit, to sniff packets 
passing by on an ethernet segment. This requires either a bridge device 
or a non-switch Hub to connect the IPaudit machine to the same ethernet 
segment as the interface you want to audit. So additional costs, and 
additional points of failure.

- If the IPaudit runs on an additional machine as above, and uses a 
plain garden variety non-switched Hub to allow the audit machine to 
"see" the packets going to and from the internal interface of the 
firewall, this introduces a performance bottle neck, especially when you 
consider that there is no such thing as a full duplex non-switching hub. 
So the use of the IPaudit standalone monitoring machine causes less than 
ideal performance.

IPmeter runs on a standalone machine, and I have observed performance 
problems when using a non-switch hub to access the ethernet segment. 
Collisions -many- with as little as 50 users.

IPaudit will probably be quite ok for a Workstation providing Internet 
connections to other home users or small office, if security on the 
Internal interface is not a concern. Also, a standalone machine to 
monitor traffic may be a simple setup, and performance may not be such 
an issue for a small number of hosts when using a no-switch Hub. 
Performance will not be hampered at all if you can get hold of an 
ethernet bridging device.

BTW, I have a complex setup of iptables firewall, a pam_iptables module, 
PHP scripts, C coded programs and  postgressql DB running right now to 
account for Internet access traffic. It's a monstrosity of 
interdependent programs, very messy. But it is currently working, and 
provides Internet access for over 50 users. Users have to loggin to the 
server , this triggers iptables rules to be added to allow them access, 
and starts a process that stores their byte counts in a PostgreSQL DB. 
Users can also access a local web page to check their usage details for 
month etc...

When I get the time, I'll clean it all up, document it and put it on a 
website as an example of how to account for packets/bytecounts using 
iptables, *and* provide access control as well (Thats what pam_iptables 
provides).

I am saying it's a definitive, and good example, just "an example"...

Cheers,
Michael

Re: IP Traffic Accounting

[Antony Stone]

On Monday 24 June 2002 6:42 am, Michael wrote:

> I have had experience with similar programs, for example ipmeter
> (http://www.ipmeter.org).

Hi Michael.

Could you check and confirm that URL please ?

Doesn't work from where I am.....

 

Antony.

Re: IP Traffic Accounting

[Michael]

Antony Stone wrote:

>On Monday 24 June 2002 6:42 am, Michael wrote
>
>I have had experience with similar programs, for example ipmeter
>
>>(http://www.ipmeter.org).
>>
>
>Hi Michael.
>
>Could you check and confirm that URL please ?
>
>Doesn't work from where I am.....
>
Oops, it's a dot com, not dot org. Much appologies.
Do a dig on www.ipmeter.com....it was up once upon a time!!

In any case the web page must have been pulled down or is offline for 
some reason. Here's a link to a mirror that carries it.

http://www.mirrors.wiretapped.net/security/network-monitoring/ipmeter/

In a nutshell, the thing is designed from ground up to run on FreeBSD, 
and it's not really a program, but a collection of scripts to use 
NetraMet. Two ways to get it going, download the install cdrom image, 
and burn CD, or download the source and run some scripts to set it up on 
a fresh FreeBSD installation.

The Install CDrom is easiest, but uses an old BSD version. I found the 
old FreeBSD version very flakey on the hardware we had, with it running 
out of system resorces. I managed to get it all going on a later FreeBSD 
4.4, with a bit of tweaking and it has been rock solid. Due to the way 
it's been implemented, it won't run anylater than PostgresSQL 7.0.2 
without some big changes (It has some custom SQL functions that clash 
with new ones in later versions).

Ours is actually still running now, 73 days uptime. Last downtime due to 
power supply failure...

If you are really curious about how it looks, I can make the admin front 
end available for you to browse. It's in service right now, monitoring 
about 70 users, but sometime in the near future it will be de-comissioned.

Some free advice, avoid it !!! It's BSD, it is slightly clunky, very 
resource hungry etc. It's good for monitoring servers, not so much users 
accessing Internet. Some excellent ideas there though, so for some 
inspiration, it's worth a look.

Cheers,
Michael

Re: IP Traffic Accounting

[Michael]

Antony Stone wrote:

>On Monday 24 June 2002 6:42 am, Michael wrote
>
>I have had experience with similar programs, for example ipmeter
>
>>(http://www.ipmeter.org).
>>
>
>Hi Michael.
>
>Could you check and confirm that URL please ?
>
>Doesn't work from where I am.....
>
Oops, it's a dot com, not dot org. Much appologies.
Do a dig on www.ipmeter.com....it was up once upon a time!!

In any case the web page must have been pulled down or is offline for 
some reason. Here's a link to a mirror that carries it.

http://www.mirrors.wiretapped.net/security/network-monitoring/ipmeter/

In a nutshell, the thing is designed from ground up to run on FreeBSD, 
and it's not really a program, but a collection of scripts to use 
NetraMet. Two ways to get it going, download the install cdrom image, 
and burn CD, or download the source and run some scripts to set it up on 
a fresh FreeBSD installation.

The Install CDrom is easiest, but uses an old BSD version. I found the 
old FreeBSD version very flakey on the hardware we had, with it running 
out of system resorces. I managed to get it all going on a later FreeBSD 
4.4, with a bit of tweaking and it has been rock solid. Due to the way 
it's been implemented, it won't run anylater than PostgresSQL 7.0.2 
without some big changes (It has some custom SQL functions that clash 
with new ones in later versions).

Ours is actually still running now, 73 days uptime. Last downtime due to 
power supply failure...

If you are really curious about how it looks, I can make the admin front 
end available for you to browse. It's in service right now, monitoring 
about 70 users, but sometime in the near future it will be de-comissioned.

Some free advice, avoid it !!! It's BSD, it is slightly clunky, very 
resource hungry etc. It's good for monitoring servers, not so much users 
accessing Internet. Some excellent ideas there though, so for some 
inspiration, it's worth a look.

Cheers,
Michael

Re: IP Traffic Accounting

[Jason R. Martin]

> The main problems with this way of doing things are:
> 
> - If the host you run the IPaudit program on is also a firewall, by 
> virtue of the  fact that the program  requires promiscuous mode on the 
> interface it listens on introduces a (theoretical) security risk.

I agree, there is a security risk, as the recent tcpdump vulnerabilities
show.  However, the need for accounting was greater than the slim chance
of an attack directed at our setup.  I don't use the setup in the same way
you do, as I don't need to know exactly what each user is doing, but I 
am able to see host traffic useage so I can follow high bandwidth patterns.

> - An alternative to running the IPaudit on the firewalling host is to 
> have a seperate standalone machine running IPaudit, to sniff packets 
> passing by on an ethernet segment. This requires either a bridge device 
> or a non-switch Hub to connect the IPaudit machine to the same ethernet 
> segment as the interface you want to audit. So additional costs, and 
> additional points of failure.
> 
> - If the IPaudit runs on an additional machine as above, and uses a 
> plain garden variety non-switched Hub to allow the audit machine to 
> "see" the packets going to and from the internal interface of the 
> firewall, this introduces a performance bottle neck, especially when you 
> consider that there is no such thing as a full duplex non-switching hub. 
> So the use of the IPaudit standalone monitoring machine causes less than 
> ideal performance.
>
> IPmeter runs on a standalone machine, and I have observed performance 
> problems when using a non-switch hub to access the ethernet segment. 
> Collisions -many- with as little as 50 users.
> 
> IPaudit will probably be quite ok for a Workstation providing Internet 
> connections to other home users or small office, if security on the 
> Internal interface is not a concern. Also, a standalone machine to 
> monitor traffic may be a simple setup, and performance may not be such 
> an issue for a small number of hosts when using a no-switch Hub. 
> Performance will not be hampered at all if you can get hold of an 
> ethernet bridging device.

Yeah, using a hub would definitely not work for me.  I run ipaudit directly
on the firewall, where I have yet to see any sort of performance hit, even
with roughly 800-1000 users.  Every 30min I get a ~300k file.  I'm working
on ways of using the information through a MySQL database instead of the
flat text files that ipaudit produces.  We'll see if anything fun comes of
that :-)

Jason

> BTW, I have a complex setup of iptables firewall, a pam_iptables module, 
> PHP scripts, C coded programs and  postgressql DB running right now to 
> account for Internet access traffic. It's a monstrosity of 
> interdependent programs, very messy. But it is currently working, and 
> provides Internet access for over 50 users. Users have to loggin to the 
> server , this triggers iptables rules to be added to allow them access, 
> and starts a process that stores their byte counts in a PostgreSQL DB. 
> Users can also access a local web page to check their usage details for 
> month etc...
> 
> When I get the time, I'll clean it all up, document it and put it on a 
> website as an example of how to account for packets/bytecounts using 
> iptables, *and* provide access control as well (Thats what pam_iptables 
> provides).
> 
> I am saying it's a definitive, and good example, just "an example"...
> 
> Cheers,
> Michael

Re: IP Traffic Accounting

[Ramin Alidousti]

On Mon, Jun 24, 2002 at 10:33:00AM -0500, Jason R. Martin wrote:

> > IPaudit will probably be quite ok for a Workstation providing Internet 
> > connections to other home users or small office, if security on the 
> > Internal interface is not a concern. Also, a standalone machine to 
> > monitor traffic may be a simple setup, and performance may not be such 
> > an issue for a small number of hosts when using a no-switch Hub. 

You, of course, can use a switch and mirror all traffic to a port
where your NIDS or accounting application hangs off of.

Ramin

> > Performance will not be hampered at all if you can get hold of an 
> > ethernet bridging device.
> 
> Yeah, using a hub would definitely not work for me.  I run ipaudit directly
> on the firewall, where I have yet to see any sort of performance hit, even
> with roughly 800-1000 users.  Every 30min I get a ~300k file.  I'm working
> on ways of using the information through a MySQL database instead of the
> flat text files that ipaudit produces.  We'll see if anything fun comes of
> that :-)
> 
> Jason