transfer Bytes Counting

transfer Bytes Counting

[HareRam]

Hi all

iam using IPtables since long
now i have new requirement in my network
i sould like to count tranfer bytes In/Out for individual IP or MAC address
could any give me small examples, how can i do this rather i achive this

thanks in advance

hare

RE: transfer Bytes Counting

[Stewart Thompson]

Hi Hare:

Here is one way that was originally suggested by Antony Stone on the list.

<Begin Quote>

In my FORWARD chain, instead of ACCEPTing packets which are ESTABLISHED or
RELATED, I send them to a user-defined chain called for example PKTCOUNT

Then the PKTCOUNT chain contains rules like this:

iptables -A PKTCOUNT -s 11.22.33.44 -j ACCEPT
iptables -A PKTCOUNT -s 11.22.33.55 - j ACCEPT
iptables -A PKTCOUNT -s 11.22.33.66 -j ACCEPT

and so on, for each of the IP addresses you're interested in.   You could of
course use -d if you're more interested in destination addresses, or use
both.

Then the command iptables -L PKTCOUNT -n -v will show you the number of
packets and the number of bytes which have matched on each rule in this
chain
- ie the number which matched each IP address.

I actually have a cron job to do this once a minute and record all the
numbers to an IP log file, which I can then parse with a Perl program to
produce some pretty graphs. I'm sure mrtg could do this if you wanted to
use that instead.

I guess if you've already created a set of SNAT or DNAT rules to do the
translations you want, then you probably don't even need to create the
PKTCOUNT chain - just try doing iptables -L PREROUTING -n -v -t nat and it
will tell you how many packets and bytes got translated by each rule.

<End Quote>

Hope that helps.

Stu....






-----Original Message-----
From: netfilter-admin@lists.netfilter.org
[mailto:netfilter-admin@lists.netfilter.org]On Behalf Of HareRam
Sent: September 30, 2002 10:12 PM
To: netfilter@lists.netfilter.org
Subject: transfer Bytes Counting

Hi all

iam using IPtables since long
now i have new requirement in my network
i sould like to count tranfer bytes In/Out for individual IP or MAC address
could any give me small examples, how can i do this rather i achive this

thanks in advance

hare

RE: transfer Bytes Counting

[Stewart Thompson]

Hi Hare:

	Always CC the list so that other people can
help you out as well. I don't do redirect with any of the machines
that I Administer. However, you seem to be loading a lot of modules
for the simple rules you are using. Perhaps you have plans for them
in the future. Hopefully Antony will jump in here and add to this advice.

	Make a user defined chain for each on of your subnets.
Also, if your looking for security, which you should be if this accesses
the Internet. Flush all your chains, and set your policies to DROP.
Anyway, getting back to your question. I was thinking of something like.

/sbin/iptables -N NET1CHAIN
/sbin/iptables -A NET1CHAIN -s 192.168.20.11 -j REDIRECT --to-port 3129
/sbin/iptables -A NET1CHAIN -s 192.168..20.22 - j REDIRECT --to-port 3129
/sbin/iptables -A NET1CHAIN -s 192.168.20.33 -j REDIRECT --to-port 3129

sbin/iptables -N NET2CHAIN
/sbin/iptables -A NET2CHAIN -s 192.168.3.11 -j REDIRECT --to-port 3129
/sbin/iptables -A NET2CHAIN -s 192.168.3.22 - j REDIRECT --to-port 3129
/sbin/iptables -A NET2CHAIN -s 192.168.3.33 -j REDIRECT --to-port 3129

/sbin/iptables -t nat -A PREROUTING -s 192.168.20.0/24 -p tcp --dport 80 -j
\
NET1CHAIN
/sbin/iptables -t nat -A PREROUTING -s 192.168.3.0/24 -p tcp --dport 80 -j \
NET2CHAIN
/sbin/iptables -t nat -A POSTROUTING -s 192.168.20.0/24 -o eth0 -j
MASQUERADE
/sbin/iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o eth0 -j MASQUERADE
-------------------

	If this is going to be involved, there are applications that might
be better suited for keeping track of packets. Since it appears you are
redirecting
to a proxy. It may be a better place to do the packet counting. Hope that
helps.
	I am sure some other people will jump in with better suggestions.

Stu.........



-----Original Message-----
From: HareRam [mailto:hareram@sol.net.in]
Sent: October 1, 2002 2:14 AM
To: stewart.thompson@shaw.ca
Subject: Re: transfer Bytes Counting

Hi Stewart

thanks for ur responce, it really help full
but iam using like follow config
------------------------
m=modprobe
$m ip_conntrack_ftp
$m ip_conntrack_irc
$m ip_conntrack
$m ip_nat_ftp
$m ip_nat_irc
$m ip_queue
$m iptable_filter
$m iptable_mangle
$m iptable_nat
$m ip_tables
$m ipt_limit
$m ipt_LOG
$m ipt_mac
$m ipt_mark
$m ipt_MARK
$m ipt_MASQUERADE
$m ipt_MIRROR
$m ipt_multiport
$m ipt_owner
$m ipt_REDIRECT
$m ipt_REJECT
$m ipt_state
$m ipt_tcpmss
$m ipt_TCPMSS
$m ipt_tos
$m ipt_TOS
$m ipt_unclean
/sbin/iptables -F
/sbin/iptables -F -t nat
/sbin/iptables -t nat -A PREROUTING -s 192.168.20.0/24 -p tcp --dport 80 -j
REDIRECT --to-port 3129
/sbin/iptables -t nat -A PREROUTING -s 192.168.3.0/24 -p tcp --dport 80 -j
REDIRECT --to-port 3129
/sbin/iptables -t nat -A POSTROUTING -s 192.168.20.0/24 -o eth0 -j
MASQUERADE
/sbin/iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o eth0 -j MASQUERADE
-------------------

[root@catcable root]# iptables -L PREROUTING -n -v -t nat
Chain PREROUTING (policy ACCEPT 976 packets, 114K bytes)
 pkts bytes target     prot opt in     out     source
destination
  484 23232 REDIRECT   tcp  --  *      *       192.168.20.0/24
0.0.0.0/0          tcp dpt:80 redir ports 3129
    0     0 REDIRECT   tcp  --  *      *       192.168.3.0/24
0.0.0.0/0          tcp dpt:80 redir ports 3129
--------------

how do i see each ip bytes in and out , how can i insert command to each ip
traffic

iam using this for transparent proxy which is diverting to cache server
which is running on port 3129

thanks for the help in advance

hare


----- Original Message -----
From: "Stewart Thompson" <stewart.thompson@shaw.ca>
To: "HareRam" <hareram@sol.net.in>; <netfilter@lists.netfilter.org>
Sent: Tuesday, October 01, 2002 2:11 PM
Subject: RE: transfer Bytes Counting


> Hi Hare:
>
> Here is one way that was originally suggested by Antony Stone on the list.
>
> <Begin Quote>
>
> In my FORWARD chain, instead of ACCEPTing packets which are ESTABLISHED or
> RELATED, I send them to a user-defined chain called for example PKTCOUNT
>
> Then the PKTCOUNT chain contains rules like this:
>
> iptables -A PKTCOUNT -s 11.22.33.44 -j ACCEPT
> iptables -A PKTCOUNT -s 11.22.33.55 - j ACCEPT
> iptables -A PKTCOUNT -s 11.22.33.66 -j ACCEPT
>
> and so on, for each of the IP addresses you're interested in.   You could
of
> course use -d if you're more interested in destination addresses, or use
> both.
>
> Then the command iptables -L PKTCOUNT -n -v will show you the number of
> packets and the number of bytes which have matched on each rule in this
> chain
> - ie the number which matched each IP address.
>
> I actually have a cron job to do this once a minute and record all the
> numbers to an IP log file, which I can then parse with a Perl program to
> produce some pretty graphs. I'm sure mrtg could do this if you wanted to
> use that instead.
>
> I guess if you've already created a set of SNAT or DNAT rules to do the
> translations you want, then you probably don't even need to create the
> PKTCOUNT chain - just try doing iptables -L PREROUTING -n -v -t nat and it
> will tell you how many packets and bytes got translated by each rule.
>
> <End Quote>
>
> Hope that helps.
>
> Stu....
>
>
>
>
>
>
> -----Original Message-----
> From: netfilter-admin@lists.netfilter.org
> [mailto:netfilter-admin@lists.netfilter.org]On Behalf Of HareRam
> Sent: September 30, 2002 10:12 PM
> To: netfilter@lists.netfilter.org
> Subject: transfer Bytes Counting
>
> Hi all
>
> iam using IPtables since long
> now i have new requirement in my network
> i sould like to count tranfer bytes In/Out for individual IP or MAC
address
> could any give me small examples, how can i do this rather i achive this
>
> thanks in advance
>
> hare
>
>
>

Re: transfer Bytes Counting

[Antony Stone]

On Tuesday 01 October 2002 11:50 pm, Stewart Thompson wrote:

> Hi Hare:
>
> You seem to be loading a lot of modules
> for the simple rules you are using. Perhaps you have plans for them
> in the future. Hopefully Antony will jump in here and add to this advice.

Hi :-)

I can't really comment on the list of modules - it *does* seem long, yes, but 
I don't actually use modules on my firewalls - I compile everything in to the 
kernel and I don't even have module support turned on (so it's not possible 
to load a module I don't want running, or unload one I do want running...)

So long as the system is working I'd suggest looking at the ruleset to 
increase security and then maybe think about whether all the modules are 
needed once the rules are settled.

> 	Make a user defined chain for each on of your subnets.

I like this suggestion - it makes for much more efficient traversal of the 
rules, however I'm not sure how many IP address in total we're talking about 
here ?   How many machines do you have on your internal network ?

> Also, if your looking for security, which you should be if this accesses
> the Internet. Flush all your chains, and set your policies to DROP.

Even if your system does not access the Internet, you should still aim for 
security.   You can't trust local users much more than N.E. Hakkr out on the 
Internet...

*Definitely* set your INPUT and FORWARD policies to DROP, and then add rules 
to ACCEPT the traffic you want.   If you forget anything, add a rule to allow 
it.   Otherwise, if you forget to block something, you're allowing it through 
without knowing about it (and anyone who finds it is unlikely to tell you :-)

> 	If this is going to be involved, there are applications that might
> be better suited for keeping track of packets. Since it appears you are
> redirecting to a proxy, it may be a better place to do the packet counting.

Indeed.   The proxy logs will tell you some far more interesting information 
about which websites have been visited and which pages have been accessed - 
they should also give you byte counts for data transferred (although I'm not 
a squid expert so I can't be sure about the tedium of data which is 
available).

Depending on what you want to do with this data, you might want to look at 
iptraf, which is a console-based network monitor which will give you traffic 
summaries by IP address - it's not very good for automated archiving of stuff 
though.

The only other thing I would say about the method of recording byte / packet 
counts (aside from the comment I posted earlier today, which doesn't seem to 
have got out on the list yet, that you don't have to have a "-j TARGET" at 
the end of a rule if you don't want one, so you can have a list of 'empty' 
rules purely for counting purposes) is that you should be very careful about 
trying to use the nat tables for packet counting.   The nat mechanism in 
netfilter has been designed to be very efficient, and in fact only the first 
packet of a connection will traverse any explicit rules in your nat tables.   
All subsequent packets in a connection get automagically processed in the 
background, much more efficiently than if they went through all the rules in 
the nat tables.   Therefore the INPUT or FORWARD chains, in the filter table, 
are almost certainly the best place to do your counting - these will see all 
the packets.

Have fun :-)

Antony.

-- 

This email is intended for the use of the individual addressee(s) named above 
and may contain information that is confidential, privileged or unsuitable 
for overly sensitive persons with low self-esteem, no sense of humour, or 
irrational religious beliefs.

If you have received this email in error, you are required to shred it 
immediately, add some nutmeg, three egg whites and a dessertspoonful of 
caster sugar.   Whisk until soft peaks form, then place in a warm oven for 40 
minutes.   Remove promptly and let stand for 2 hours before adding some 
decorative kiwi fruit and cream.   Then notify me immediately by return email 
and eat the original message.

Firewall Question

[Bishop]

Hello all,

        I have  a question for you guys. I will installing a firewall on my
linux box soon. My question is would it be better for me to download a
firewall software off then net , or should I build my own firewall from
scratch. Let me know what you guys think, and which one would be better.


Thank you all

--Luis

RE: Firewall Question

[Stewart Thompson]

HI Luis:

	I prefer to write a script by hand using the CLI.
Most modern Linux distributions include Iptables/Netfilter.
You can also control when it starts up, and include other
things in it like Stunnel set up, conditional rules etc. I
have not used any of the firewall front ends, but a GUI is
probably easier if you are new. The trouble I have is the
rules are hidden behind the GUI interface. There are lots
of good sample scripts on the net that you can modify for
your purposes. Plus, you will get a much more interment
knowledge of Iptables. Just my two cents worth.

Stu.........




-----Original Message-----
From: netfilter-admin@lists.netfilter.org
[mailto:netfilter-admin@lists.netfilter.org]On Behalf Of Bishop
Sent: October 1, 2002 5:04 PM
To: stewart.thompson@shaw.ca; HareRam; netfilter@lists.netfilter.org
Subject: Firewall Question

Hello all,

        I have  a question for you guys. I will installing a firewall on my
linux box soon. My question is would it be better for me to download a
firewall software off then net , or should I build my own firewall from
scratch. Let me know what you guys think, and which one would be better.


Thank you all

--Luis

Re: transfer Bytes Counting

[HareRam]

Hi

thanks for the reply
i did the same, but iam not able to see the in and out bytes
is there any way i can send those packets to mysql
from there i can generate report

thanks
hare
----- Original Message -----
From: "Antony Stone" <Antony@Soft-Solutions.co.uk>
To: <netfilter@lists.netfilter.org>
Sent: Wednesday, October 02, 2002 5:04 AM
Subject: Re: transfer Bytes Counting


> On Tuesday 01 October 2002 11:50 pm, Stewart Thompson wrote:
>
> > Hi Hare:
> >
> > You seem to be loading a lot of modules
> > for the simple rules you are using. Perhaps you have plans for them
> > in the future. Hopefully Antony will jump in here and add to this
advice.
>
> Hi :-)
>
> I can't really comment on the list of modules - it *does* seem long, yes,
but
> I don't actually use modules on my firewalls - I compile everything in to
the
> kernel and I don't even have module support turned on (so it's not
possible
> to load a module I don't want running, or unload one I do want running...)
>
> So long as the system is working I'd suggest looking at the ruleset to
> increase security and then maybe think about whether all the modules are
> needed once the rules are settled.
>
> > Make a user defined chain for each on of your subnets.
>
> I like this suggestion - it makes for much more efficient traversal of the
> rules, however I'm not sure how many IP address in total we're talking
about
> here ?   How many machines do you have on your internal network ?
>
> > Also, if your looking for security, which you should be if this accesses
> > the Internet. Flush all your chains, and set your policies to DROP.
>
> Even if your system does not access the Internet, you should still aim for
> security.   You can't trust local users much more than N.E. Hakkr out on
the
> Internet...
>
> *Definitely* set your INPUT and FORWARD policies to DROP, and then add
rules
> to ACCEPT the traffic you want.   If you forget anything, add a rule to
allow
> it.   Otherwise, if you forget to block something, you're allowing it
through
> without knowing about it (and anyone who finds it is unlikely to tell you
:-)
>
> > If this is going to be involved, there are applications that might
> > be better suited for keeping track of packets. Since it appears you are
> > redirecting to a proxy, it may be a better place to do the packet
counting.
>
> Indeed.   The proxy logs will tell you some far more interesting
information
> about which websites have been visited and which pages have been
accessed -
> they should also give you byte counts for data transferred (although I'm
not
> a squid expert so I can't be sure about the tedium of data which is
> available).
>
> Depending on what you want to do with this data, you might want to look at
> iptraf, which is a console-based network monitor which will give you
traffic
> summaries by IP address - it's not very good for automated archiving of
stuff
> though.
>
> The only other thing I would say about the method of recording byte /
packet
> counts (aside from the comment I posted earlier today, which doesn't seem
to
> have got out on the list yet, that you don't have to have a "-j TARGET" at
> the end of a rule if you don't want one, so you can have a list of 'empty'
> rules purely for counting purposes) is that you should be very careful
about
> trying to use the nat tables for packet counting.   The nat mechanism in
> netfilter has been designed to be very efficient, and in fact only the
first
> packet of a connection will traverse any explicit rules in your nat
tables.
> All subsequent packets in a connection get automagically processed in the
> background, much more efficiently than if they went through all the rules
in
> the nat tables.   Therefore the INPUT or FORWARD chains, in the filter
table,
> are almost certainly the best place to do your counting - these will see
all
> the packets.
>
> Have fun :-)
>
> Antony.
>
> --
>
> This email is intended for the use of the individual addressee(s) named
above
> and may contain information that is confidential, privileged or unsuitable
> for overly sensitive persons with low self-esteem, no sense of humour, or
> irrational religious beliefs.
>
> If you have received this email in error, you are required to shred it
> immediately, add some nutmeg, three egg whites and a dessertspoonful of
> caster sugar. Whisk until soft peaks form, then place in a warm oven for
40
> minutes. Remove promptly and let stand for 2 hours before adding some
> decorative kiwi fruit and cream. Then notify me immediately by return
email
> and eat the original message.
>
>

Re: transfer Bytes Counting

[Antony Stone]

On Wednesday 02 October 2002 2:39 pm, HareRam wrote:

> Hi
>
> thanks for the reply
> i did the same, but iam not able to see the in and out bytes

If you mean a separate count of bytes in each direction on the connection, 
you would need to have two rules, one to count packets in and one to count 
packets out.

> is there any way i can send those packets to mysql
> from there i can generate report

I believe other people have posted to this list with mechanisms for capturing 
log entries to sql database - can anyone post a tool or URL to help with this?

Antony.

-- 

You can spend the whole of your life trying to be popular,
but at the end of the day the size of the crowd at your funeral
will be largely dictated by the weather.

 - Frank Skinner

RE: Firewall Question

[Rowan Reid]

> 
>         I have  a question for you guys. I will installing a 
> firewall on my linux box soon. My question is would it be 
> better for me to download a firewall software off then net , 
> or should I build my own firewall from scratch. Let me know 
> what you guys think, and which one would be better.

I'm fairly new to firewall, my last successful attepmed was using
Iptables, and the Strong Firewall script as a template. After learning
more about IPTables I definitely say start with a proven script then
follow It line for line making sure you understand it.


http://www.e-infomax.com/ipmasq/howto/c-html/stronger-firewall-examples.
html#RC.FIREWALL-2.4.X-STRONGER

RE: transfer Bytes Counting

[Clint Todish]

What might be exceeding useful is the addition of in/out packet counters in
/proc/net/ip_conntrack - kind of like Cisco's netflow. Not only would
historical info been gleaned, but a nice real-time "what's goin on with the
net" tool could be built off of that.

-C

-----Original Message-----
From: netfilter-admin@lists.netfilter.org
[mailto:netfilter-admin@lists.netfilter.org]On Behalf Of Antony Stone
Sent: Wednesday, October 02, 2002 9:33 AM
To: netfilter@lists.netfilter.org
Subject: Re: transfer Bytes Counting


On Wednesday 02 October 2002 2:39 pm, HareRam wrote:

> Hi
>
> thanks for the reply
> i did the same, but iam not able to see the in and out bytes

If you mean a separate count of bytes in each direction on the connection,
you would need to have two rules, one to count packets in and one to count
packets out.

> is there any way i can send those packets to mysql
> from there i can generate report

I believe other people have posted to this list with mechanisms for
capturing
log entries to sql database - can anyone post a tool or URL to help with
this?

Antony.

--

You can spend the whole of your life trying to be popular,
but at the end of the day the size of the crowd at your funeral
will be largely dictated by the weather.

 - Frank Skinner

Re: transfer Bytes Counting

[HareRam]

Hi Clint

thanks for the reply
i am not able to see any file called the same name in /proc/net/ directory
how to i export and get that file and put the in and out packets to mysql
can u give me some example

thanks
hare
----- Original Message -----
From: "Clint Todish" <ctodish@crayon.com>
To: "Antony Stone" <Antony@Soft-Solutions.co.uk>;
<netfilter@lists.netfilter.org>
Sent: Wednesday, October 02, 2002 9:43 PM
Subject: RE: transfer Bytes Counting


>
> What might be exceeding useful is the addition of in/out packet counters
in
> /proc/net/ip_conntrack - kind of like Cisco's netflow. Not only would
> historical info been gleaned, but a nice real-time "what's goin on with
the
> net" tool could be built off of that.
>
> -C
>
> -----Original Message-----
> From: netfilter-admin@lists.netfilter.org
> [mailto:netfilter-admin@lists.netfilter.org]On Behalf Of Antony Stone
> Sent: Wednesday, October 02, 2002 9:33 AM
> To: netfilter@lists.netfilter.org
> Subject: Re: transfer Bytes Counting
>
>
> On Wednesday 02 October 2002 2:39 pm, HareRam wrote:
>
> > Hi
> >
> > thanks for the reply
> > i did the same, but iam not able to see the in and out bytes
>
> If you mean a separate count of bytes in each direction on the connection,
> you would need to have two rules, one to count packets in and one to count
> packets out.
>
> > is there any way i can send those packets to mysql
> > from there i can generate report
>
> I believe other people have posted to this list with mechanisms for
> capturing
> log entries to sql database - can anyone post a tool or URL to help with
> this?
>
> Antony.
>
> --
>
> You can spend the whole of your life trying to be popular,
> but at the end of the day the size of the crowd at your funeral
> will be largely dictated by the weather.
>
>  - Frank Skinner
>
>
>