iptables analyzer

iptables analyzer

[Earl A.Killian]

Has anyone built a iptables analyzer?  I am thinking writing one, but
I don't want to reinvent what might already exist.

I have in mind something that tries all "interesting" (as gleaned from
the rules themselves) combinations of IP addresses, protocols, ports,
etc., simulates the ruleset and says what happens.  The result would
be displayed in table form.

One could test the equivalence of two iptables specifications by
analyzing them and comparing the output (useful for seeing if an
"optmization" is correct).

Re: iptables analyzer

[SBlaze]

--- "Earl A.Killian" <netfilter@lists.killian.com> wrote:
> Has anyone built a iptables analyzer?  I am thinking writing one, but
> I don't want to reinvent what might already exist.
> 
> I have in mind something that tries all "interesting" (as gleaned from
> the rules themselves) combinations of IP addresses, protocols, ports,
> etc., simulates the ruleset and says what happens.  The result would
> be displayed in table form.
> 
> One could test the equivalence of two iptables specifications by
> analyzing them and comparing the output (useful for seeing if an
> "optmization" is correct).
> 
I too am interested in this... and thus set out to find some projects to do
this as I am a novice shell scripter and know zilch on programming...

This is what I have found although nothing really gets to the grit like I would
like...

Iam
http://intevation.de/iam/

IPAC-NG - IP accounting next generation
http://ipac-ng.sourceforge.net/

IAG - IP-Accounting Grapher
http://dev.lashout.net/iag/

BW ACCT v0.9.4
http://www.bwacct.org/

ADMLogger
http://aaron.marasco.com/linux.html

Firewall Log Daemon - firelogd
http://rouxdoo.freeshell.org/dmn/

Firewall Monitor (fwmon)
http://www.scaramanga.co.uk/fwmon/

fwlogwatch 0.9.3
http://www.kyb.uni-stuttgart.de/boris/software.shtml

PStat 
http://members.fortunecity.com/mrsi/

iptacct - Accounting for Linux 2.4
http://tretmine.org/iptacct/

SASacct
http://rousse.pm.org/sasacct/

Simple Little Firewall Monitor
http://slfm.sourceforge.net/

tvc4
http://fsi-server.physik.uni-erlangen.de/~sndapara/shaping/index_en.shtml

While i have not tried any of these projects... I don't think they are what I
am looking for. I don't really need graphics or things displayed in MySQL
databases.. I would simple like to be able to saya... hey how many packets have
I gottin in the last 5 or so mins...and see the difference from the 5 mins
before. Graphing this daily weekly monthy..etc would be nice...but hey I'm
willing to settle.

Good luck in your hunt. Hope this helps.

=====
In the absence of order there will be chaos.

__________________________________
Do you Yahoo!?
Exclusive Video Premiere - Britney Spears
http://launch.yahoo.com/promos/britneyspears/

Re: iptables analyzer

[Earl A.Killian]

Actually I'm planning something static, independent of the traffic
received by the host.

Basically, it would take iptables-save output and make a list of all the
* protocols used (the -p's)
* the src addresses used (the -s's)
* the dst addresses used (the -d's)
* the src ports used (the --sport's)
* the dst ports used (the --dport's)
* the input interfaces used (the -i's)
* the output interfaces used (the -o's)
* the --icmp-types used
then augment each of the above lists with an "other" entry, and then
and then simulate every combination of the above and
* the 4 conntrack states
* the possible ip flags (-f)
* the possible tcp flags
against the rules, and print a table of the final target (ACCEPT or
DROP).

The output would be big, but some intelligent pruning (use of '*' to
merge multiple rows when possible) should make it more manageable.

One use is to simply have paranoid humans review the output.
Another use is to run it on two different iptable rule sets and
compare what's different (e.g. if you reorder rules for efficiency,
this would check that you haven't inadvertently changed the firewall
semantics).

Re: iptables analyzer

[SBlaze]

Heh what about people who run iptables from a script... like me?


--- "Earl A.Killian" <netfilter@lists.killian.com> wrote:
> Actually I'm planning something static, independent of the traffic
> received by the host.
> 
> Basically, it would take iptables-save output and make a list of all the
> * protocols used (the -p's)
> * the src addresses used (the -s's)
> * the dst addresses used (the -d's)
> * the src ports used (the --sport's)
> * the dst ports used (the --dport's)
> * the input interfaces used (the -i's)
> * the output interfaces used (the -o's)
> * the --icmp-types used
> then augment each of the above lists with an "other" entry, and then
> and then simulate every combination of the above and
> * the 4 conntrack states
> * the possible ip flags (-f)
> * the possible tcp flags
> against the rules, and print a table of the final target (ACCEPT or
> DROP).
> 
> The output would be big, but some intelligent pruning (use of '*' to
> merge multiple rows when possible) should make it more manageable.
> 
> One use is to simply have paranoid humans review the output.
> Another use is to run it on two different iptable rule sets and
> compare what's different (e.g. if you reorder rules for efficiency,
> this would check that you haven't inadvertently changed the firewall
> semantics).


=====
In the absence of order there will be chaos.

__________________________________
Do you Yahoo!?
Exclusive Video Premiere - Britney Spears
http://launch.yahoo.com/promos/britneyspears/

RE: iptables analyzer

[Daniel Chemko]

In all honesty, the tool may have limited small scale benefits, but the
only way to scale into bigger projects would be to write a firewall
structural design before even implementing the rule structure.

1. The analyzer needs to be able to predict the load of each protocol /
rule match in order to 'order' the rules in the most effective manner.

EG:

Rule #1: 5% of the traffic
Rule #2: 10% of the traffic
Rule #3: 85% of traffic
This is a non-optimized ruleset, but unless the analyzer can use
historical data, the effectiveness of this optimizer is diminished.
Ideally, a history based ruleset analyzer would reorganize the rules in
the following order:

Rule #3
Rule #2
Rule #1
But! Now, this may break the actual logic of the program as well, so you
have to allow for making historical rule adjustments while taking into
account the correctness of the ruleset flow. Making this work for larger
set of rules and for dynamically adjusting based on short term forecasts
would start with head starting to hurt then my going to bed all dizzy
and such.


2. Complex rulesets with large numbers of branches and conditional rules
would cause killer complexity, and the changes to that structure would
blow away any non-trivial iptables ruleset creation tools.

EG:

A->B
A->C (!A->B)
A-R1 (!A->B & !A->C)
A-R2 (!A->B & !A->C & !A-R1)
A-R3 (!A->B & !A->C & !A-R1 & !A-R2)
B-R1 (A->B)
B-R2 (A->B & !B-R1)
B-R3 (A->B & !B-R1 & !B-R2)
B->D (A->B & !B-R1 & !B-R2 & !BR3)
C-R1 (!A->B & A->C)
C->D (!A->B & A->C & !C-R1)
D-R1 (((A->B & !B-R1 & !B-R2 & !BR3)) | (!A->B & A->C & !C-R1))

If you can parse the args, return the correct structural dependencies of
each rule throughout the path to the end of the ruleset then actually
optimize the ruleset for the best efficiency (which can vary depending
on the needs of the firewall), I will surely come visit whatever corner
of the world you are from and offer you a b33r, because you would surely
deserve it!

Remember, the dependencies of each rule are compounded with the existing
rules that did not pass the filter, which is assuming that the jumps
relinquished control over the packet, but as log, some rules pass
control back to the branch changing the packet's state and continues!
All this must be accounted for!


3. Programming in functionality that intelligently uses the iptables
extensions would bloat the program to a dramatic extent. 

Well, but now, I imagine you get the difficulty of this point as well.

Re: iptables analyzer

[Jeffrey Laramie]

[-- Attachment #1: Type: text/html, Size: 5511 bytes --]

RE: iptables analyzer

[Gaby Schilders]

[-- Attachment #1: Type: text/plain, Size: 5138 bytes --]

It gets more interesting if you consider this: I would like to create rules solely based on my needs, including making rules per machine. Then have this 'tool' use the mathematical concept of domains to distill the minimal rule-set. (This is about rules overlapping needlesly).
 
Gaby

-----Original Message-----
From: Jeffrey Laramie [mailto:JALaramie@Loudoun-Fairfax.com]
Sent: woensdag 29 oktober 2003 6:34
To: netfilter@lists.netfilter.org
Subject: Re: iptables analyzer


This is interesting stuff. I'm not a programmer or an expert, but as a longtime user I have a couple thoughts:

Daniel Chemko wrote:


In all honesty, the tool may have limited small scale benefits, but the

only way to scale into bigger projects would be to write a firewall

structural design before even implementing the rule structure.



1. The analyzer needs to be able to predict the load of each protocol /

rule match in order to 'order' the rules in the most effective manner.



EG:



Rule #1: 5% of the traffic

Rule #2: 10% of the traffic

Rule #3: 85% of traffic

This is a non-optimized ruleset, but unless the analyzer can use

historical data, the effectiveness of this optimizer is diminished.

Ideally, a history based ruleset analyzer would reorganize the rules in

the following order:



Rule #3

Rule #2

Rule #1

But! Now, this may break the actual logic of the program as well, so you

have to allow for making historical rule adjustments while taking into

account the correctness of the ruleset flow. Making this work for larger

set of rules and for dynamically adjusting based on short term forecasts

would start with head starting to hurt then my going to bed all dizzy

and such.



  


You would have to have a complete dataset from every packet that passes netfilter to run any kind of rule analysis. Knowing which rule dropped a packet isn't the same as knowing exactly which condition of that rule caused the drop. Could be iface, dport, sport, etc. You would almost have to duplicate the logic of netfilter itself to do any meaningful analysis at this level. However I think a simpler but still useful tool could be created by using the packet and byte count kept by netfilter to sort rules within chains


2. Complex rulesets with large numbers of branches and conditional rules

would cause killer complexity, and the changes to that structure would

blow away any non-trivial iptables ruleset creation tools.



EG:



A->B

A->C (!A->B)

A-R1 (!A->B & !A->C)

A-R2 (!A->B & !A->C & !A-R1)

A-R3 (!A->B & !A->C & !A-R1 & !A-R2)

B-R1 (A->B)

B-R2 (A->B & !B-R1)

B-R3 (A->B & !B-R1 & !B-R2)

B->D (A->B & !B-R1 & !B-R2 & !BR3)

C-R1 (!A->B & A->C)

C->D (!A->B & A->C & !C-R1)

D-R1 (((A->B & !B-R1 & !B-R2 & !BR3)) | (!A->B & A->C & !C-R1))

  


Yeah, this is going to be tough but keep in mind that most people don't want to change the logic of their rules. Creating a routine that models potential changes in program flow is a whole nother ballgame and specifying the flow logic you're trying to achieve would be far more effort than doing it yourself. IMHO an analyzer should assume that no rule change should be allowed to change the logical flow of the rules. It should only be allowed to test for efficiency within chains or perhaps recommend new chains that improve efficiency without changing the final rule destination of the packet. Otherwise the analyzer would surely break your logic and then you'd be really screwed. Given this constraint the task becomes much more manageable.


If you can parse the args, return the correct structural dependencies of

each rule throughout the path to the end of the ruleset then actually

optimize the ruleset for the best efficiency (which can vary depending

on the needs of the firewall), I will surely come visit whatever corner

of the world you are from and offer you a b33r, because you would surely

deserve it!

  


Only one b22r?



Remember, the dependencies of each rule are compounded with the existing

rules that did not pass the filter, which is assuming that the jumps

relinquished control over the packet, but as log, some rules pass

control back to the branch changing the packet's state and continues!

All this must be accounted for!



  


Like I said, I don't think this level of analysis is realistic or even desirable. On the other hand a tool that parses your rules for obviously poor design (established, related rule at end of chain or not there at all, rules that can never be reached, etc.), and then optimizes the efficiency of the rules within each chain based on historical data would be useful for unsophisticated users (like me!)


3. Programming in functionality that intelligently uses the iptables

extensions would bloat the program to a dramatic extent. 



Well, but now, I imagine you get the difficulty of this point as well.





  


The phrase walk before you run comes to mind, and good luck to anyone who takes on this project.  :-)

Jeff



[-- Attachment #2: Type: text/html, Size: 6663 bytes --]

Re: iptables analyzer

[Vicnent Haverlant]

-- 
   .~.     Vincent Haverlant  -- Galadril -- #ICQ: 35695155   
   /V\      http://www.haverlant.org/
  /( )\      Parinux (http://www.parinux.org/)
  ^^-^^       MUD -- FranDUMII (http://perso.enst.fr/~frandum/)

Re: iptables analyzer

[Herve Eychenne]

On Tue, Oct 28, 2003 at 02:53:01AM -0800, SBlaze wrote:

> Iam
> http://intevation.de/iam/

> IPAC-NG - IP accounting next generation
> http://ipac-ng.sourceforge.net/

> IAG - IP-Accounting Grapher
> http://dev.lashout.net/iag/

> BW ACCT v0.9.4
> http://www.bwacct.org/

> ADMLogger
> http://aaron.marasco.com/linux.html

> Firewall Log Daemon - firelogd
> http://rouxdoo.freeshell.org/dmn/

> Firewall Monitor (fwmon)
> http://www.scaramanga.co.uk/fwmon/

> fwlogwatch 0.9.3
> http://www.kyb.uni-stuttgart.de/boris/software.shtml

> PStat 
> http://members.fortunecity.com/mrsi/

> iptacct - Accounting for Linux 2.4
> http://tretmine.org/iptacct/

> SASacct
> http://rousse.pm.org/sasacct/

> Simple Little Firewall Monitor
> http://slfm.sourceforge.net/

> tvc4
> http://fsi-server.physik.uni-erlangen.de/~sndapara/shaping/index_en.shtml

> While i have not tried any of these projects... I don't think they are what I
> am looking for. I don't really need graphics or things displayed in MySQL
> databases.. I would simple like to be able to saya... hey how many packets have
> I gottin in the last 5 or so mins...and see the difference from the 5 mins
> before. Graphing this daily weekly monthy..etc would be nice...but hey I'm
> willing to settle.

Maybe it's because you didn't try wflogs yet. ;-)

I think the command you're looking for may be something like this:

# wflogs --strict-parsing=nowarning --sort=none --filter '$start_time >= [5
  minutes ago]' -o text --summary=no --header=no --resolve=0
  --sresolve=no --whois=0

Please don't get fooled by the command line length... wflogs is just a
quite powerful tool...

http://www.wallfire.org/wflogs/

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/