Advanced Logging

Advanced Logging

[ratheesh k]

Hi ,

           I have implemented  firewall  in my linux machine using
iptables . It is able to prevent attacks and LOG just before dropping
packets . Since i know a little about iptables , i could go thru
/var/log/messages and find out information about attacks . Is there
any application which will analyze logs and  give a brief information
to user about the attacks  ?

For example , suppose there was a syn flood attack ,the application
should analyse the /var/log/messages or by some means should know
about the attack and let the user know about that .If there is no
application ,  could you give some hints on how to develop an
application .Any comment is  appreciated .

Thanks,
Ratheesh

Re: Advanced Logging

[Curby]

On Sun, May 30, 2010 at 7:28 AM, ratheesh k <ratheesh.ksz@gmail.com> wrote:
> any application which will analyze logs and  give a brief information
> to user about the attacks  ?

I've also been wondering about the existence of such tools, but I
haven't done any research yet.  In particular I'm hoping to explore
parallel coordinate plots, which can be used to map out source and
dest IPs and ports and show traffic patterns.  You could see one IP
port-scanning different dest ports, or many different IPs SYN flooding
a particular dest host.

http://en.wikipedia.org/wiki/Parallel_coordinates

As a generalization of Ratheesh's question, does anyone have
recommendations or personal favorites for iptables log visualizations
for an at-a-glance overview of traffic patterns?

Thanks!

--Mike

Re: Advanced Logging

[Tomáš Vlček]

Maybe psad (Port Scan Attack Detector) is that what are you looking
for. Check http://cipherdyne.org/psad/index.html.

There is also a great book about this program (and 3 additional
programs as well) written by the author of these programs. This book
covers psad (analyser of iptables logs), fwsnort (translator of Snort
rules into iptables rules), fwknop (single packet authorization) and
ways how to visualize iptables logs. Details can be found at
http://www.nostarch.com/firewalls_mr.htm.

I hope this helps...

Tomáš Vlček

On Sun, May 30, 2010 at 4:28 PM, ratheesh k <ratheesh.ksz@gmail.com> wrote:
>
> Hi ,
>
>           I have implemented  firewall  in my linux machine using
> iptables . It is able to prevent attacks and LOG just before dropping
> packets . Since i know a little about iptables , i could go thru
> /var/log/messages and find out information about attacks . Is there
> any application which will analyze logs and  give a brief information
> to user about the attacks  ?
>
> For example , suppose there was a syn flood attack ,the application
> should analyse the /var/log/messages or by some means should know
> about the attack and let the user know about that .If there is no
> application ,  could you give some hints on how to develop an
> application .Any comment is  appreciated .
>
> Thanks,
> Ratheesh
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Advanced Logging

[ratheesh k]

2010/5/30 Tomáš Vlček <tomasvlcek@gmail.com>:
> Maybe psad (Port Scan Attack Detector) is that what are you looking
> for. Check http://cipherdyne.org/psad/index.html.

I gone through the link . It seems to be heavy for my embedded application .

My embedded box is a router with two inerfaces - wan0 and lan0 . I
should get information regarding various attacks tried on lan clients
.I have some implementation in mind .(see below )

1  Is there any tool fit my requirement or  there any tool , i can do
a little  modification in code   and use .
2 . Is my idea feasible to implement ? . Is it worth implementing ,
because it is run as part of softirq_rx kernel thread . Will it dampen
performance ?
3 . Could i do this as part of connection tracking module . If , could
you guide  a little ?

*********************************************************************************************************************************************************************
"install two  hook functions on NF_IP_FORWARD  hook with
NF_IP_PRI_FIRST( first_hook_func )  and  NF_IP_PRI_LAST(last_hook_func
) priority .   iptables drop all  DOS attack ,flood   pkts in FORWARD
chain with NF_IP_PRI_FILTER priority ,

A tuple ( destip , srcip , protocol , timestamp , in_count , out_count
 ) will be created for each NEW packet in first_hook_func for all
packets from wan to lan . there should be different linked list tuple
for differnt type of packets ( syn attck , icmp  flood  etc ) . And
in_count is incremented   . And in last_hook_func , i  do hashing to
find the correct tuple . if found , out_count is incremented . In
last_hook_func itself , deletion of old entries ( if timestamp <
jiffies -delta ) needs to be done . And in last_hook_func , proc entry
is updated ,if in_count - out_count  is greater than some particular
threshold .That means , a lot of same kind of packet visited
first_hook_func but dropped in Filter table .

********************************************************************************************************************************************************************





>
> There is also a great book about this program (and 3 additional
> programs as well) written by the author of these programs. This book
> covers psad (analyser of iptables logs), fwsnort (translator of Snort
> rules into iptables rules), fwknop (single packet authorization) and
> ways how to visualize iptables logs. Details can be found at
> http://www.nostarch.com/firewalls_mr.htm.
>
> I hope this helps...
>
> Tomáš Vlček
>
> On Sun, May 30, 2010 at 4:28 PM, ratheesh k <ratheesh.ksz@gmail.com> wrote:
>>
>> Hi ,
>>
>>           I have implemented  firewall  in my linux machine using
>> iptables . It is able to prevent attacks and LOG just before dropping
>> packets . Since i know a little about iptables , i could go thru
>> /var/log/messages and find out information about attacks . Is there
>> any application which will analyze logs and  give a brief information
>> to user about the attacks  ?
>>
>> For example , suppose there was a syn flood attack ,the application
>> should analyse the /var/log/messages or by some means should know
>> about the attack and let the user know about that .If there is no
>> application ,  could you give some hints on how to develop an
>> application .Any comment is  appreciated .
>>
>> Thanks,
>> Ratheesh
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Advanced Logging

[Jan Engelhardt]

On Thursday 2010-06-03 20:15, ratheesh k wrote:
>2010/5/30 Tomáš Vlček <tomasvlcek@gmail.com>:
>> Maybe psad (Port Scan Attack Detector) is that what are you looking
>> for. Check http://cipherdyne.org/psad/index.html.
>
>I gone through the link . It seems to be heavy for my embedded application .

Yes it looks complicated from a developer POV. I myself think
why would it have to put up with analyzing log messages
(which are known to be not overly precise) when direct delivery
with libnetfilter_queue/_log seems like a more ideal goal - 
eliminating the extra trip through syslog and the fs.

Re: Advanced Logging

[ratheesh k]

On Fri, Jun 4, 2010 at 12:32 AM, Jan Engelhardt <jengelh@medozas.de> wrote:
>
> On Thursday 2010-06-03 20:15, ratheesh k wrote:
>>2010/5/30 Tomáš Vlček <tomasvlcek@gmail.com>:
>>> Maybe psad (Port Scan Attack Detector) is that what are you looking
>>> for. Check http://cipherdyne.org/psad/index.html.
>>
>>I gone through the link . It seems to be heavy for my embedded application .
>
> Yes it looks complicated from a developer POV. I myself think
> why would it have to put up with analyzing log messages
> (which are known to be not overly precise) when direct delivery
> with libnetfilter_queue/_log seems like a more ideal goal -
> eliminating the extra trip through syslog and the fs.
\
Thanks Jan .
ulogd demon could be modified to do analysis of packet  to find out
what of kind of attack has taken place ?


-Ratheesh

Re: Advanced Logging

[Mart Frauenlob]

On 03.06.2010 20:15, ratheesh k wrote:
> 2010/5/30 Tomáš Vlček <tomasvlcek@gmail.com>:


>>>           I have implemented  firewall  in my linux machine using
>>> iptables . It is able to prevent attacks and LOG just before dropping
>>> packets . Since i know a little about iptables , i could go thru
>>> /var/log/messages and find out information about attacks . Is there
>>> any application which will analyze logs and  give a brief information
>>> to user about the attacks  ?
>>>
>>> For example , suppose there was a syn flood attack ,the application
>>> should analyse the /var/log/messages or by some means should know
>>> about the attack and let the user know about that .If there is no
>>> application ,  could you give some hints on how to develop an
>>> application .Any comment is  appreciated .


>> Maybe psad (Port Scan Attack Detector) is that what are you looking
>> for. Check http://cipherdyne.org/psad/index.html.
>
> I gone through the link . It seems to be heavy for my embedded
application .
>
> My embedded box is a router with two inerfaces - wan0 and lan0 . I
> should get information regarding various attacks tried on lan clients
> .I have some implementation in mind .(see below )
>
> 1  Is there any tool fit my requirement or  there any tool , i can do
> a little  modification in code   and use .
> 2 . Is my idea feasible to implement ? . Is it worth implementing ,
> because it is run as part of softirq_rx kernel thread . Will it dampen
> performance ?
> 3 . Could i do this as part of connection tracking module . If , could
> you guide  a little ?
>
snort (snort.org) comes into my mind here.
afaik it has the ability to create inline iptables rules.
maybe worth a look?


best regards

mart

Re: Advanced Logging

[Mart Frauenlob]

On 03.06.2010 22:17, netfilter-owner@vger.kernel.org wrote:
> On 03.06.2010 20:15, ratheesh k wrote:
>> 2010/5/30 Tomáš Vlček <tomasvlcek@gmail.com>:
> 
> 
>>>>           I have implemented  firewall  in my linux machine using
>>>> iptables . It is able to prevent attacks and LOG just before dropping
>>>> packets . Since i know a little about iptables , i could go thru
>>>> /var/log/messages and find out information about attacks . Is there
>>>> any application which will analyze logs and  give a brief information
>>>> to user about the attacks  ?
>>>>
>>>> For example , suppose there was a syn flood attack ,the application
>>>> should analyse the /var/log/messages or by some means should know
>>>> about the attack and let the user know about that .If there is no
>>>> application ,  could you give some hints on how to develop an
>>>> application .Any comment is  appreciated .
> 
> 
>>> Maybe psad (Port Scan Attack Detector) is that what are you looking
>>> for. Check http://cipherdyne.org/psad/index.html.
>>
>> I gone through the link . It seems to be heavy for my embedded
> application .
>>
>> My embedded box is a router with two inerfaces - wan0 and lan0 . I
>> should get information regarding various attacks tried on lan clients
>> .I have some implementation in mind .(see below )
>>
>> 1  Is there any tool fit my requirement or  there any tool , i can do
>> a little  modification in code   and use .
>> 2 . Is my idea feasible to implement ? . Is it worth implementing ,
>> because it is run as part of softirq_rx kernel thread . Will it dampen
>> performance ?
>> 3 . Could i do this as part of connection tracking module . If , could
>> you guide  a little ?
>>
> snort (snort.org) comes into my mind here.
> afaik it has the ability to create inline iptables rules.
> maybe worth a look?
> 

Reading again, I think the answer was too short.

Doing it all on one embedded device might itself be not that safe.
Besides the effect that the resources maybe limited.
Saved logs on a compromised host could be modified.

Now if you simply analyze logs some time after the attack has happened
it may be a bit late, even if an application has sent you an email or
such, you might read it ~12 hours later.
In most cases you only catch the most obvious 'noisy' attack flood/scan.
Well you could send an abuse mail, worth the hassle?
You couldn't really do much interactively.

If you are after a pure iptables log message parser for a single host,
things might be limited to some awk/grep/shell/etc... script for pretty
printing.
Most things I've seen would at least require some webserver and/or
database in the background. Many focus on a larger scope/network.

You might just try a search on freshmeat.net or sf.net for i.e.
'iptables log analyzer' or similar.
I.e. I know arnos-iptables-firewall (not that I use that as my nf
generator) has a pretty printing script shipping with it.

So the next step would be some sort of IDS.
But this may also be overkill for your device. I can't tell.
Running a snort instance with inline functionality would give you not
just an opportunity to react to a wider range of attacks (L7) much more
gracefully, also there is a wide range of logging options (and backend
analyze tools available, which of course require more resources and
should be placed on separate hosts - i.e. BASE, or Prelude (with snort
as sensor)).
Doing only minimal text logging for important events might give enough
information without overloading your device.

Just some thoughts.
Hope it helps.

Mart