ethernet headers and iptables counters

ethernet headers and iptables counters

[Marc Lucke]

[-- Attachment #1: Type: text/plain, Size: 325 bytes --]

Hi,

Do the iptables counters include layer 2 ethernet packet headers?  If
so, how much traffic is this - is it a set amount?

Horst.Hansen@fhtw-berlin.de Sat Mar 08 12:29:13 2003 asked the question
that I was interested in - nobody seemed to answer him.

My apologies if this is an RTFA question.


good wishes to all,
Marc

[-- Attachment #2: Type: text/html, Size: 598 bytes --]

Re: ethernet headers and iptables counters

[Antony Stone]

On Monday 10 November 2003 9:14 am, Marc Lucke wrote:

> Hi,
>
> Do the iptables counters include layer 2 ethernet packet headers?  If
> so, how much traffic is this - is it a set amount?

No, because you may not be using ethernet.

Netfilter can be used for packets across ethernet, 802.11, PPP modems - all 
sorts of network transports.

Packet and byte counters in netfilter are IP (OSI layer 3) and upwards.

You can check this easily by creating a rule matching something specific (eg 
ping packets), send a known amount of traffic through the box (or measure it 
with a packet sniffer / protocol analyser such as ethereal, which will give 
you a detailed view of the contents of the packets), and then check what the 
counters say.

Antony

-- 

Anything that improbable is effectively impossible.

 - Murray Gell-Mann, Nobel Prizewinner in Physics
                                                     Please reply to the list;
                                                           please don't CC me.

Re: ethernet headers and iptables counters

[Antony Stone]

On Monday 10 November 2003 10:25 pm, Marc Lucke wrote:

> Antony - my thanks to you.  I rather suspected your answer would be
> true.  My problem is that I am getting very large differences in the
> usage I track using iptables and the usage that my ISP reports.
> iptables reports much more traffic - up to 12% in fact.  I have checked
> with the ISP who say they too only count layer 3 IP.

12% is a lot, and the fact that you think you have 12% more traffic than your 
ISP does is more strange than if it were the other way around.

It does occur to me to wonder whether the ISP counts IP headers, but I really 
would expect them to do so, and even so, an IP header is only 24 bytes, with 
up to 1452 bytes of data inside the packet (assuming an MTU of 1500).   For 
24 bytes to make 12% difference your data would be only 200 bytes on average, 
and this is far too low to be plausible.

> Just to be sure in terms of working out why I have such a large
> discrepancy with my ISP, is there a way that the MASQ rule could be
> having some sort of unwanted effect when examining the counters in the
> FORWARD rule?  My thought is not.

I agree with you.   The FORWARD chain counts bytes being forwarded (routed) 
through the netfilter box.   The nat tables in PREROUTING and/or POSTROUTING 
are quite separate and willshow very different byte counts (in fact these two 
will be wildly inaccurate and much lower than your true usage, because they 
do not show automagically de-NATted reply packets).

So, you have higher traffic counts in your netfilter rules than the ISP says 
is going across your external link.

What do your interface packet/byte counters say if you look at the output of 
ifconfig?   Do these agree with your netfilter measurements, with the ISPs 
numbers, or are they different again!?

The only other thing I can suggest is that you try putting in a very simple 
rule at the top of each of the INPUT, FORWARD and OUTPUT chains to count the 
traffic independently of your rules, and see if these counts agree with any 
other measurement being made:

iptables -I INPUT -i eth0
iptables -I FORWARD -i eth0
iptables -I FORWARD -o eth0
iptables -I OUTPUT -o eth0

(Yes, that's right, those rules are *very* simple - they don't even have a -j 
target at the end of them.   They're perfectly valid, and will do absolutely 
nothing to your packets except count them for when you come to look at 
"iptables -L -n -v -x")

Hope something amongst all this helps.

Antony.

-- 

The only problem with the Universe as a platform, though, is that it is 
currently running someone else's program.

 - Ken Karakotsios, author of SimLife
                                                     Please reply to the list;
                                                           please don't CC me.

Re: ethernet headers and iptables counters

[Simon Kitching]

On Tue, 2003-11-11 at 12:21, Antony Stone wrote:
> On Monday 10 November 2003 10:25 pm, Marc Lucke wrote:
> 
> > Antony - my thanks to you.  I rather suspected your answer would be
> > true.  My problem is that I am getting very large differences in the
> > usage I track using iptables and the usage that my ISP reports.
> > iptables reports much more traffic - up to 12% in fact.  I have checked
> > with the ISP who say they too only count layer 3 IP.
> 
> 12% is a lot, and the fact that you think you have 12% more traffic than your 
> ISP does is more strange than if it were the other way around.

What about traffic from your host back to itself?

For example, GNOME heavily uses the ORBIT corba server, which uses
TCP/IP to communicate between interacting applications.

Maybe you could add a separate rule for traffic from 127.0.0.1 and to
your local host address back to the local machine, to measure that?

NB: I'm no networking guru, so this may be a red herring...
And I didn't see the original posting for this, so don't know if the
server you are measuring traffic on is running anything else...


Regards,

Simon

Re: ethernet headers and iptables counters

[Antony Stone]

On Monday 10 November 2003 11:34 pm, Simon Kitching wrote:

> On Tue, 2003-11-11 at 12:21, Antony Stone wrote:
> > On Monday 10 November 2003 10:25 pm, Marc Lucke wrote:
> > > Antony - my thanks to you.  I rather suspected your answer would be
> > > true.  My problem is that I am getting very large differences in the
> > > usage I track using iptables and the usage that my ISP reports.
> > > iptables reports much more traffic - up to 12% in fact.  I have checked
> > > with the ISP who say they too only count layer 3 IP.
> >
> > 12% is a lot, and the fact that you think you have 12% more traffic than
> > your ISP does is more strange than if it were the other way around.
>
> What about traffic from your host back to itself?

That should go via interface lo, not via eth0.

> For example, GNOME heavily uses the ORBIT corba server, which uses
> TCP/IP to communicate between interacting applications.
>
> Maybe you could add a separate rule for traffic from 127.0.0.1 and to
> your local host address back to the local machine, to measure that?

The routing table entry for 127.0.0.0/8 should send all this traffic via the 
loopback interface - this should not be counted by anything looking at the 
ethernet devices.

Antony.

-- 

In science, one tries to tell people
in such a way as to be understood by everyone
something that no-one ever knew before.

In poetry, it is the exact opposite.

 - Paul Dirac
                                                     Please reply to the list;
                                                           please don't CC me.

Re: ethernet headers and iptables counters

[Leonardo Rodrigues Magalhães]

>
> It does occur to me to wonder whether the ISP counts IP headers, but I
really
> would expect them to do so, and even so, an IP header is only 24 bytes,
with
> up to 1452 bytes of data inside the packet (assuming an MTU of 1500).
For
> 24 bytes to make 12% difference your data would be only 200 bytes on
average,
> and this is far too low to be plausible.

    Your affirmation 'IP header is only 24 bytes' is wrong. IP header has,
at least, 20 bytes and can have up to 60 bytes if you use IP Options. But,
in almost all cases, you'll have IP with no options, which means 20 bytes
for the IP headers. It can have 24 bytes in some cases, no doubt, but it's
not right that is always will have 24 and only 24 bytes.

    There also Transport Layer (4) headers. Using TCP, for example, you'll
have at least 20 bytes on the TCP headers, being able to reach 64 bytes if
all TCP options are enabled.

    This small error cannot explain the 12% difference ....... but we should
notice it :)


    Sincerily,
    Leonardo Rodrigues