Accounting with iptables vs. snmp

Accounting with iptables vs. snmp

[Stefan-Michael. Guenther (in-put GbR)]

Hi,

using iptables I have setup a traffic accounting on one of our client's 
gateways:

iptables -A INPUT -i $WAN -j LOG --log-level debug
iptables -A OUTPUT -o $WAN -j LOG --log-level debug
iptables -A FORWARD -j LOG --log-level debug

syslogd collects the entries in a single file which is analysed daily.

The results corresponds to the amount of data I get with "iptables -L -v -n" .

The provider of our client uses snmp on his router to calculate the traffic.

Strange, but true: The numbers are never the same, sometimes iptables logs 
more traffic, sometimes snmp. The differences are between 1 and 25 %.

Obviously someone is doing something wrong. The provider is one of Germany's 
big player, so I guess I made the mistake. But where and why?

Thanks for any hint.

Stefan
-- 

*****************************************
in-put GbR - Das Linux-Systemhaus
Stefan-Michael Guenther
Moltkestrasse 49     D-76133 Karlsruhe
Tel./Fax : +49 (0)721 / 83044 - 98/93
http://www.in-put.de
*****************************************

Re: Accounting with iptables vs. snmp

[Richard Hauswald]

Stefan-Michael. Guenther (in-put GbR) wrote:
> Hi,
> 
> using iptables I have setup a traffic accounting on one of our client's 
> gateways:
> 
> iptables -A INPUT -i $WAN -j LOG --log-level debug
> iptables -A OUTPUT -o $WAN -j LOG --log-level debug
> iptables -A FORWARD -j LOG --log-level debug
> 
> syslogd collects the entries in a single file which is analysed daily.
> 
> The results corresponds to the amount of data I get with "iptables -L -v -n" .
> 
> The provider of our client uses snmp on his router to calculate the traffic.
> 
> Strange, but true: The numbers are never the same, sometimes iptables logs 
> more traffic, sometimes snmp. The differences are between 1 and 25 %.
> 
> Obviously someone is doing something wrong. The provider is one of Germany's 
> big player, so I guess I made the mistake. But where and why?
> 
> Thanks for any hint.
> 
> Stefan

Hello Stefan,
maybe (!)... your problem is simple so solve. You are appending this 
rules with the LOG target. So you will not count traffic which is 
blocked. Just write an -I instead of -A. But i don't know if thats the 
problem which took up to 25% of traffic difference. It sounds very 
strange, if you say that some times you count more than your provider 
and another day your provider counts more. Maybe you have an failure 
based on rounding the bytes to megabytes?

Regards
Richard

-- 
There are only 10 types of people in the world:
Those who understand binary, and those who don't

Re: Re: Accounting with iptables vs. snmp

[Stefan-Michael. Guenther (in-put GbR)]

Hello Richard,

> Hello Stefan,
> maybe (!)... your problem is simple so solve. You are appending this
> rules with the LOG target. So you will not count traffic which is
> blocked. Just write an -I instead of -A. But i don't know if thats the
> problem which took up to 25% of traffic difference. It sounds very
> strange, if you say that some times you count more than your provider
> and another day your provider counts more. Maybe you have an failure
> based on rounding the bytes to megabytes?
>
I don't block packets on this box, there is a cisco box between the net and 
the linux box. Last wednesday the difference was about 2.6 GB in only 24 
hours! The scripts doesn't do any rounding, I've switched this feature off to 
get exact results. Even with all those portscans and P2P-packets, I don't 
think that this could add up to 2.6 GB. And it wouldn't explain why the box 
sometimes reports more traffic that the provider.

Could it be that the box is to slow, to see and log all packets? Sometimes I 
find lines like "last message repeated 10 times" in the logfile but my 
scripts is able to analyse these lines, too. And again, this would mean equal 
or less traffic, but no more traffic than the providers reports.


Stefan


-- 

*****************************************
in-put GbR - Das Linux-Systemhaus
Stefan-Michael Guenther
Moltkestrasse 49     D-76133 Karlsruhe
Tel./Fax : +49 (0)721 / 83044 - 98/93
http://www.in-put.de
*****************************************

Re: Accounting with iptables vs. snmp

[Steven M Campbell]

Stefan-Michael. Guenther (in-put GbR) wrote:

>Hi,
>
>using iptables I have setup a traffic accounting on one of our client's 
>gateways:
>
>iptables -A INPUT -i $WAN -j LOG --log-level debug
>iptables -A OUTPUT -o $WAN -j LOG --log-level debug
>iptables -A FORWARD -j LOG --log-level debug
>
>  
>
SNMP will record arps and other traffic that hits the nic regardless of 
it being directed to that machine or not, iptables (above) will not.   
The numbers will likely always be different, I am surprised that 
iptables would ever report more data being processed than snmp would.

Re: Accounting with iptables vs. snmp

[Richard Hauswald]

Stefan-Michael. Guenther (in-put GbR) wrote:
> Hello Richard,
> 
> 
>>Hello Stefan,
>>maybe (!)... your problem is simple so solve. You are appending this
>>rules with the LOG target. So you will not count traffic which is
>>blocked. Just write an -I instead of -A. But i don't know if thats the
>>problem which took up to 25% of traffic difference. It sounds very
>>strange, if you say that some times you count more than your provider
>>and another day your provider counts more. Maybe you have an failure
>>based on rounding the bytes to megabytes?
>>
> 
> I don't block packets on this box, there is a cisco box between the net and 
> the linux box. Last wednesday the difference was about 2.6 GB in only 24 
> hours!
Thats much traffic...
> The scripts doesn't do any rounding, I've switched this feature off to 
> get exact results. Even with all those portscans and P2P-packets, I don't 
> think that this could add up to 2.6 GB.
Sure? :-)
> And it wouldn't explain why the box 
> sometimes reports more traffic that the provider.
Maybe there is another way to get access to the Internet. I mean another 
way then to use the Linux box as gateway. That would explain why 
sometimes more and some times less traffic is reported by your box than 
from your ISP. If you have a DMZ and your traffic counter is in the DMZ, 
than it will not count traffic for other DMZ servers. Is the box 
directly connected to the cisco gateway? I mean using a cross over cat5? 
If not, do so to exclude this failure possibility.
> Could it be that the box is to slow, to see and log all packets?
No. You are using iptables and not snort with a box connected to a 
monitoring port on a switch with 100MBit and much network traffic.
> Sometimes I find lines like "last message repeated 10 times" in the logfile but my 
> scripts is able to analyse these lines, too.
You could also change the LOG rule to a accept rule. If you do so, you 
have to tell cron.hourly (for example) to grep/awk out the values for 
the rule counters. It does not make sense with the log rule. If you 
want, you can grep it out every minute. that won't produce much system 
load. Try this way and test again.
> And again, this would mean equal 
> or less traffic, but no more traffic than the providers reports.
And again: This sounds very strange. :-(
> 
> Stefan

Richard

-- 
There are only 10 types of people in the world:
Those who understand binary, and those who don't