[LARTC] NAT-aware traffic analysis

[LARTC] NAT-aware traffic analysis

[Ming-Ching Tiew]

I have tried using iptraf for my NAT firewall to analyse the IP traffic.
Basically I am faced with this difficulty of related the source IP
to the outgoing interface to the internet, so I am wondering if
anyone has a suggestion for a different ways to do it, or a suggestion
for a better tool.

Details :-

Supposed : eth0 - LAN
                   eth1 - WAN1
                   eth2 - WAN2

And then all source IPs in the LAN are SNAT to the respective
WAN interface when leave for internet. There are also DNAT
traffic from internet to the LAN.

I want to breakdown the statistic of LAN users using the
internet. If I run iptraf on eth0, I will see the LAN stats, but I
don't know for sure which one really go out to which WAN
( some traffic does not even go out to the WAN at all ! ).

Then when I sniff at eth1 or eth2, I lost the information about the LAN IPs.

How could I do a stateful or NAT-aware traffic analysis ? Anyone has
a good suggestion ?


--------------------------------------------------------
Important Warning! 

*************************** 

This electronic communication (including any attached files) may contain confidential and/or legally privileged information and is only intended for the use of the person to whom it is addressed. If you are not the intended recipient, you do not have permission to read, use, disseminate, distribute, copy or retain any part of this communication or its attachments in any form. If this e-mail was sent to you by mistake, please take the time to notify the sender so that they can identify the problem and avoid any more mistakes in sending e-mail to you. The unauthorised use of information contained in this communication or its attachments may result in legal action against any person who uses it.

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] NAT-aware traffic analysis

[Salim S I]

A different approach is to use iptables counters in FORWARD chain (-s
$CLIENT_IP -i eth0 -o ! eth0). That would require a  rule for each user.


-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Ming-Ching Tiew
Sent: Wednesday, September 05, 2007 11:09 AM
To: lartc@mailman.ds9a.nl
Subject: [LARTC] NAT-aware traffic analysis


I have tried using iptraf for my NAT firewall to analyse the IP traffic.
Basically I am faced with this difficulty of related the source IP
to the outgoing interface to the internet, so I am wondering if
anyone has a suggestion for a different ways to do it, or a suggestion
for a better tool.

Details :-

Supposed : eth0 - LAN
                   eth1 - WAN1
                   eth2 - WAN2

And then all source IPs in the LAN are SNAT to the respective
WAN interface when leave for internet. There are also DNAT
traffic from internet to the LAN.

I want to breakdown the statistic of LAN users using the
internet. If I run iptraf on eth0, I will see the LAN stats, but I
don't know for sure which one really go out to which WAN
( some traffic does not even go out to the WAN at all ! ).

Then when I sniff at eth1 or eth2, I lost the information about the LAN
IPs.

How could I do a stateful or NAT-aware traffic analysis ? Anyone has
a good suggestion ?


--------------------------------------------------------
Important Warning! 

*************************** 

This electronic communication (including any attached files) may contain
confidential and/or legally privileged information and is only intended
for the use of the person to whom it is addressed. If you are not the
intended recipient, you do not have permission to read, use,
disseminate, distribute, copy or retain any part of this communication
or its attachments in any form. If this e-mail was sent to you by
mistake, please take the time to notify the sender so that they can
identify the problem and avoid any more mistakes in sending e-mail to
you. The unauthorised use of information contained in this communication
or its attachments may result in legal action against any person who
uses it.

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] NAT-aware traffic analysis

[Ming-Ching Tiew]

From: "Salim S I" <salim.si@cipherium.com.tw>


> A different approach is to use iptables counters in FORWARD chain (-s
> $CLIENT_IP -i eth0 -o ! eth0). That would require a  rule for each user.
> 
>

Well sort of theoretically possible but bad in pratice. If I have 300
internal users, I will have to create 300 iptable rules. Then if I 
want to analyse based on sport or dport, you can imagine the 
number of rules will be quite many.

Anyone has other suggestions ?

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] NAT-aware traffic analysis

[Martin A. Brown]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

 : I have tried using iptraf for my NAT firewall to analyse the IP 
 : traffic. Basically I am faced with this difficulty of related the 
 : source IP to the outgoing interface to the internet, so I am 
 : wondering if anyone has a suggestion for a different ways to do 
 : it, or a suggestion for a better tool.

I don't know of a flow analysis tool that records internal and 
external addresses at the NAT boundary.  Without knowing how you 
separate your traffic outbound, it'd be hard for us to guess what 
the shortcomings of any of these solutions might be, but here are a 
few ideas:

  * Record the state of /proc/net/ip_conntrack and your flow 
    information snapshots at exactly the same time.  Use the 
    ip_conntrack state information (programmatically) to yield
    the answers you want about usage information.

  * Use a flow analysis tool (e.g., argus) to record the flow 
    information on your internal interface.  Since you built the 
    rules for distributing traffic and selecting the path for 
    outbound flows, you should be able to map this same logic onto 
    your recorded flows.

In short, I think you may have better luck approaching the problem 
as a flow-analysis problem than a statistical summarization of 
traffic on any specific interface.

Good luck,

- -Martin

- -- 
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFG3i65HEoZD1iZ+YcRAkqiAJ4rp7p3Sg+b4i0PYvpXRlHZtrm/ogCfe52L
00fFE3OOeNHP8QIiTRuB9LM=Egrt
-----END PGP SIGNATURE-----
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] NAT-aware traffic analysis

[Marco Aurelio]

If you use IFB or IMQ you can shape the outgoing WAN traffic before NAT

On 9/5/07, Martin A. Brown <martin@linux-ip.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Greetings,
>
>  : I have tried using iptraf for my NAT firewall to analyse the IP
>  : traffic. Basically I am faced with this difficulty of related the
>  : source IP to the outgoing interface to the internet, so I am
>  : wondering if anyone has a suggestion for a different ways to do
>  : it, or a suggestion for a better tool.
>
> I don't know of a flow analysis tool that records internal and
> external addresses at the NAT boundary.  Without knowing how you
> separate your traffic outbound, it'd be hard for us to guess what
> the shortcomings of any of these solutions might be, but here are a
> few ideas:
>
>   * Record the state of /proc/net/ip_conntrack and your flow
>     information snapshots at exactly the same time.  Use the
>     ip_conntrack state information (programmatically) to yield
>     the answers you want about usage information.
>
>   * Use a flow analysis tool (e.g., argus) to record the flow
>     information on your internal interface.  Since you built the
>     rules for distributing traffic and selecting the path for
>     outbound flows, you should be able to map this same logic onto
>     your recorded flows.
>
> In short, I think you may have better luck approaching the problem
> as a flow-analysis problem than a statistical summarization of
> traffic on any specific interface.
>
> Good luck,
>
> - -Martin
>
> - --
> Martin A. Brown
> http://linux-ip.net/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)
>
> iD8DBQFG3i65HEoZD1iZ+YcRAkqiAJ4rp7p3Sg+b4i0PYvpXRlHZtrm/ogCfe52L
> 00fFE3OOeNHP8QIiTRuB9LM> =Egrt
> -----END PGP SIGNATURE-----
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>


-- 
Marco Casaroli
SapucaiNet Telecom
+55 35 34712377 ext. 5
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] NAT-aware traffic analysis

[Ming-Ching Tiew]

From: "Marco Aurelio" <marco.casaroli@gmail.com>

> If you use IFB or IMQ you can shape the outgoing WAN traffic before NAT
>

I am not sure if I understand this reply or the reply seems to me,
is not replying to my original question.

I am asking how to collect statistics about LAN users with respect
to their WAN usage, with LAN IP as the breakdown.

I am not asking how to do traffic shaping. And may I know how
does IMQ help that ?

Actually with more thought given to the problem, I think I am
quite inclined to using iptables ULOG. But ULOG solution
has a few things need mentioning :-

1. Might be very heavy on system loading. Hope people can
    clarify if it is a real concern. And anyone has experience using
    ULOG 2.x ? Will 2.x be more friendly to system loading
    compared to 1.x ?

2. Logging goes into either file or database. It's to be a offline
    monitoring mechanism. Is there a way to use ULOG for online
    monitoring ?

3. Next, each ULOG is only specifying one side of the traffic. eg :-

        iptables -A FORWARD -i eth0 -o eth1 -j ULOG .....

    I will need another iptables rule to specify the returning traffic, eg
:-

        iptables -A FORWARD -i eth1 -o eth0 -j ULOG .....

   Combining two independent logs as one connection will still be a
challenge.

Hope to see more suggestions and discussion.
Thank you.



_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] NAT-aware traffic analysis

[Marco Aurelio]

Sorry if didn't reply you as expected

Currently I use iptables to monitor how many bytes and packets each
client has transmitted:

Each client has an ACCEPT rule that matches their IP and MAC address

I can see the byte and packet counters with iptables -L -n -v

then, I use a script to parse this output and feed the apropriate RRD.

Previously, I used to parse the output of tc -s class ls dev ifb0
which gave me almost the same result

On 9/6/07, Ming-Ching Tiew <mingching.tiew@redtone.com> wrote:
>
> From: "Marco Aurelio" <marco.casaroli@gmail.com>
>
> > If you use IFB or IMQ you can shape the outgoing WAN traffic before NAT
> >
>
> I am not sure if I understand this reply or the reply seems to me,
> is not replying to my original question.
>
> I am asking how to collect statistics about LAN users with respect
> to their WAN usage, with LAN IP as the breakdown.
>
> I am not asking how to do traffic shaping. And may I know how
> does IMQ help that ?
>
> Actually with more thought given to the problem, I think I am
> quite inclined to using iptables ULOG. But ULOG solution
> has a few things need mentioning :-
>
> 1. Might be very heavy on system loading. Hope people can
>     clarify if it is a real concern. And anyone has experience using
>     ULOG 2.x ? Will 2.x be more friendly to system loading
>     compared to 1.x ?
>
> 2. Logging goes into either file or database. It's to be a offline
>     monitoring mechanism. Is there a way to use ULOG for online
>     monitoring ?
>
> 3. Next, each ULOG is only specifying one side of the traffic. eg :-
>
>         iptables -A FORWARD -i eth0 -o eth1 -j ULOG .....
>
>     I will need another iptables rule to specify the returning traffic, eg
> :-
>
>         iptables -A FORWARD -i eth1 -o eth0 -j ULOG .....
>
>    Combining two independent logs as one connection will still be a
> challenge.
>
> Hope to see more suggestions and discussion.
> Thank you.
>
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>


-- 
Marco Casaroli
SapucaiNet Telecom
+55 35 34712377 ext. 5
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc