(Ab)using iptables to record byte count per IP?

(Ab)using iptables to record byte count per IP?

[Richard Hartmann]

Hi all,

I need to log the traffic each local IP in a NAT'ed network generates
per day.

Unfortunately, the environment is rather restricted and compiling new
software is not an option. Thus, I am wondering if I can abuse iptables,
which is in use anyway, to log for me, as well.

The end result should be a log which contains IP, date and traffic
generated. I don't really care about the output format as I will perl it
into the format I need, anyway.


Thanks for all input,
Richard

Re: (Ab)using iptables to record byte count per IP?

[Artūras Šlajus]

Richard Hartmann wrote:
 > Hi all,
 >
 > I need to log the traffic each local IP in a NAT'ed network generates
 > per day.
 >
 > Unfortunately, the environment is rather restricted and compiling new
 > software is not an option. Thus, I am wondering if I can abuse iptables,
 > which is in use anyway, to log for me, as well.
 >
 > The end result should be a log which contains IP, date and traffic
 > generated. I don't really care about the output format as I will perl it
 > into the format I need, anyway.

iptables -N ACCOUNTING
iptables -A FORWARD -j ACCOUNTING
iptables -A ACCOUNTING -s your_user_ip -j ACCEPT
iptables -A ACCOUNTING -d your_user_ip -j ACCEPT

bam, with iptables -L ACCOUNTING -n -v you can have packet/byte counters ;)

Re: (Ab)using iptables to record byte count per IP?

[Thomas Jacob]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

On Fri, 2009-01-09 at 13:50 +0200, Artūras Šlajus wrote:
> Richard Hartmann wrote:
>  > Hi all,
>  >
>  > I need to log the traffic each local IP in a NAT'ed network generates
>  > per day.
>  >
>  > Unfortunately, the environment is rather restricted and compiling new
>  > software is not an option. Thus, I am wondering if I can abuse iptables,
>  > which is in use anyway, to log for me, as well.
>  >
>  > The end result should be a log which contains IP, date and traffic
>  > generated. I don't really care about the output format as I will perl it
>  > into the format I need, anyway.
> 
> iptables -N ACCOUNTING
> iptables -A FORWARD -j ACCOUNTING
> iptables -A ACCOUNTING -s your_user_ip -j ACCEPT
> iptables -A ACCOUNTING -d your_user_ip -j ACCEPT
> 
> bam, with iptables -L ACCOUNTING -n -v you can have packet/byte counters ;)

Or you could use the external extension ipt_ACCOUNT

http://www.intra2net.com/de/produkte/opensource/ipt_account/ 

which does the same much faster and you don't have to
specify lots of accounting rules. Unfortunately you need
to patch your kernel for that, if you want to use it
as is.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: (Ab)using iptables to record byte count per IP?

[Richard Hartmann]

On Fri, Jan 9, 2009 at 12:55, Thomas Jacob <jacob@internet24.de> wrote:

> http://www.intra2net.com/de/produkte/opensource/ipt_account/

Neat. I can't use it in this case, but I will keep it in mind :)


Richard

Re: (Ab)using iptables to record byte count per IP?

[Amos Jeffries]

Thomas Jacob wrote:
> On Fri, 2009-01-09 at 13:50 +0200, Artūras Šlajus wrote:
>> Richard Hartmann wrote:
>>  > Hi all,
>>  >
>>  > I need to log the traffic each local IP in a NAT'ed network generates
>>  > per day.
>>  >
>>  > Unfortunately, the environment is rather restricted and compiling new
>>  > software is not an option. Thus, I am wondering if I can abuse iptables,
>>  > which is in use anyway, to log for me, as well.
>>  >
>>  > The end result should be a log which contains IP, date and traffic
>>  > generated. I don't really care about the output format as I will perl it
>>  > into the format I need, anyway.
>>
>> iptables -N ACCOUNTING
>> iptables -A FORWARD -j ACCOUNTING
>> iptables -A ACCOUNTING -s your_user_ip -j ACCEPT
>> iptables -A ACCOUNTING -d your_user_ip -j ACCEPT
>>
>> bam, with iptables -L ACCOUNTING -n -v you can have packet/byte counters ;)
> 
> Or you could use the external extension ipt_ACCOUNT
> 
> http://www.intra2net.com/de/produkte/opensource/ipt_account/ 
> 
> which does the same much faster and you don't have to
> specify lots of accounting rules. Unfortunately you need
> to patch your kernel for that, if you want to use it
> as is.

Whats needed and is anyone working towards getting that integrated properly?

AYJ

Re: (Ab)using iptables to record byte count per IP?

[Richard Hartmann]

On Fri, Jan 9, 2009 at 12:50, Artūras Šlajus <x11@arturaz.net> wrote:

> iptables -A ACCOUNTING -s your_user_ip -j ACCEPT
> iptables -A ACCOUNTING -d your_user_ip -j ACCEPT

Doesn't that mean that I am bypassing the rest of the
firewall rules?

Richard

Re: (Ab)using iptables to record byte count per IP?

[Robert Nichols]

Richard Hartmann wrote:
> On Fri, Jan 9, 2009 at 12:50, Artūras Šlajus <x11@arturaz.net> wrote:
> 
>> iptables -A ACCOUNTING -s your_user_ip -j ACCEPT
>> iptables -A ACCOUNTING -d your_user_ip -j ACCEPT
> 
> Doesn't that mean that I am bypassing the rest of the
> firewall rules?

Yes, it would.  Just leave off the "-j ACCEPT" or use "-j RETURN" if
you want to bypass the rest of the ACCOUNTING chain.  There is no
requirement that a rule have a target.  I have a couple of rules
like that in my "mangle" table PREROUTING and POSTROUTING chains,
and they work just fine.

You'll want to use iptables with the "-x" flag when reading the
counters so that you get exact counts and not numbers like "14G".

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

Re: (Ab)using iptables to record byte count per IP?

[Pablo Neira Ayuso]

Richard Hartmann wrote:
> Hi all,
> 
> I need to log the traffic each local IP in a NAT'ed network generates
> per day.
> 
> Unfortunately, the environment is rather restricted and compiling new
> software is not an option. Thus, I am wondering if I can abuse iptables,
> which is in use anyway, to log for me, as well.
> 
> The end result should be a log which contains IP, date and traffic
> generated. I don't really care about the output format as I will perl it
> into the format I need, anyway.

Polling iptables does not look to me a good option. The way to go should 
be ulogd2 or or the conntrackd in its simple statistics mode.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: (Ab)using iptables to record byte count per IP?

[Richard Hartmann]

On Fri, Jan 9, 2009 at 13:35, Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> Polling iptables does not look to me a good option. The way to go should be
> ulogd2 or or the conntrackd in its simple statistics mode.

Normally yes. But in this case, I need to use what's there and I know that
iptables is there :)


Richard

Re: (Ab)using iptables to record byte count per IP?

[Payam Chychi]

Hi,

Thats possible by using the pre-nat table. You will need to place an
accept all rule for each individual /32 ip address that you want to do
accounting for ( I use accept so that I can see all traffic destined
for the ip)

-Payam


On Fri, Jan 9, 2009 at 3:27 AM, Richard Hartmann
<richih.mailinglist@gmail.com> wrote:
> Hi all,
>
> I need to log the traffic each local IP in a NAT'ed network generates
> per day.
>
> Unfortunately, the environment is rather restricted and compiling new
> software is not an option. Thus, I am wondering if I can abuse iptables,
> which is in use anyway, to log for me, as well.
>
> The end result should be a log which contains IP, date and traffic
> generated. I don't really care about the output format as I will perl it
> into the format I need, anyway.
>
>
> Thanks for all input,
> Richard
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Payam Tarverdyan Chychi
Network Security Specialist / Network Engineer

Re: (Ab)using iptables to record byte count per IP?

[Vlado Drz(ík]

It depends on number of IPs that you want to account and what traffic
(pck/s) will be comming through your router.
The IP tables accouning based on adding IPs to chains (iptables -A
somechain -s <src_IP>) is simple but scales well just for small packet
counts and number of rules.
But for you it's probably best choice as it's really easy to implement.
For example create 2 tables:
iptables -N ACCT_IN
iptables -N ACCT_OUT

Add to forward somewhere before accepting rules (nearly begening of chain):
iptables -A FORWARD -i <inet_nic> -j ACCT_IN
iptables -A FORWARD -o <inet_nic> -j ACCT_OUT
Add IP addreses
iptables -A ACCT_IN -d <ipaddress 1> -j RETURN
iptables -A ACCT_IN -d <ipaddress 2> -j RETURN
....
Add oposite diraction:
iptables -A ACCT_OUT -s <ipaddress 1> -j RETURN
iptables -A ACCT_OUT -s <ipaddress 2> -j RETURN


If you are going to count many users >1000 it's better to consider
external modules like ipt_ACCCOUNT:
http://www.intra2net.com/de/produkte/opensource/ipt_account/
It scales very well for large numbers of IPs but bigest drawback is that
it's out of tree module and you need to compile kernel+iptables (so it's
really not a option for you).

If I understand the conntrack (CT) accounting correctly I see that there
is problem that it's flow based so if some flows are dismissed from CT
table quickly you can get unaccurate results if you read CT table in
wider intervals.

I'd much like to see also packet based accounting (not just flow based
one) in kernel. Do someone use conntrack based accounting in real life?

- vlado

Payam Chychi  wrote / napísal(a):
> Hi,
> 
> Thats possible by using the pre-nat table. You will need to place an
> accept all rule for each individual /32 ip address that you want to do
> accounting for ( I use accept so that I can see all traffic destined
> for the ip)
> 
> -Payam
> 
> 
> On Fri, Jan 9, 2009 at 3:27 AM, Richard Hartmann
> <richih.mailinglist@gmail.com> wrote:
>> Hi all,
>>
>> I need to log the traffic each local IP in a NAT'ed network generates
>> per day.
>>
>> Unfortunately, the environment is rather restricted and compiling new
>> software is not an option. Thus, I am wondering if I can abuse iptables,
>> which is in use anyway, to log for me, as well.
>>
>> The end result should be a log which contains IP, date and traffic
>> generated. I don't really care about the output format as I will perl it
>> into the format I need, anyway.
>>
>>
>> Thanks for all input,
>> Richard
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> 
> 

Re: (Ab)using iptables to record byte count per IP?

[Peter Renzland]

I do this to set up:

for SD in s d; do
iptables -N traffic_$SD
iptables -I FORWARD -j traffic_$SD
for IP in 100 103 105 108 120 122 124 126 130 132; do # adjust?
  iptables -A traffic_$SD -$SD 192.168.0.$IP # adjust?
done
done

And to collect:

# Traffic Accounting to be run Hourly at HH:59 -- Planiwa 2009.01.06
# Appends to file /var/traffic/CCYYMM a line: CCYYMMDD.HH [IP-suffix  
dst-MB src-MB] ...

[ -d /var/traffic ] || mkdir /var/traffic; cd /var/traffic
set -- $(date '+%Y%m %d.%H'); CCYYMM=$1; DDHH=$2
sleep 59

for SD in s d;do iptables -L -Z traffic_$SD -vnx;done | awk '
BEGIN {MB=2^20;KB=2^10; PRECISION=MB; SUBNET="192.168."} # adjust?
$8 ~ SUBNET {if ($1!=0) D[substr($8,11)]=$2; next}
$7 ~ SUBNET {if ($1!=0) S[substr($7,11)]=$2; next}
END {
printf "%s%s ", "'$CCYYMM'", "'$DDHH'"
for (IP in D)if((D[IP]>=PRECISION/2)||(S[IP]>=PRECISION/2))
  printf("%s %d %d ",IP,D[IP]/PRECISION+.5,S[IP]/PRECISION+.5);print  
"" # MB
}' >>$CCYYMM

Adjust as needed.

tail -20 /var/traffic/200901
20090108.10 100 49 5 122 50 3 124 18 1 126 36 1
20090108.11 100 3 0 122 156 46 124 247 6 108 20 3 126 6 0
20090108.12 122 372 155 124 282 6 108 5 1
20090108.13 100 0 6 122 318 176 124 56 2 126 1 0
20090108.14 122 324 210 124 34 2 126 28 1
20090108.15 122 171 252 124 35 2 108 11 3 126 26 1
20090108.16 100 0 1 122 8 249 124 74 3 108 19 4 126 13 1
20090108.17 122 6 224 124 114 4 108 33 3 126 72 2
20090108.18 122 5 151 124 36 1 126 1 0
20090108.19 100 1 11 122 18 72 126 28 0
20090108.20 100 0 1 122 5 1 108 8 2
20090108.21 100 1 1 122 98 3 108 16 2
20090108.22 120 133 3 108 10 2
20090108.23 100 1 8 122 144 5 108 60 2
20090109.00 100 0 3 122 21 2 108 49 3
20090109.01 100 0 3 122 1 1 124 11 1 108 23 3 126 14 1
20090109.02 124 21 1 108 26 4 126 5 1
20090109.03 100 5 73 108 4 1
20090109.04 100 0 5
20090109.05


I understand that current iptables has -m account, with separate  
counters by protocol, long and short stats.

But I only have v1.3.7 on 2.4.20 -- Tomato on a WRT54GL router.

Peter Renzland





On 09  Jan 9, at 06:27 , Richard Hartmann wrote:

> Hi all,
>
> I need to log the traffic each local IP in a NAT'ed network generates
> per day.
>
> Unfortunately, the environment is rather restricted and compiling new
> software is not an option. Thus, I am wondering if I can abuse  
> iptables,
> which is in use anyway, to log for me, as well.
>
> The end result should be a log which contains IP, date and traffic
> generated. I don't really care about the output format as I will  
> perl it
> into the format I need, anyway.
>
>
> Thanks for all input,
> Richard
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>