Logging suggestions

Logging suggestions

[hareram]

Hi all

I have large  capacity link like STM with my provider
Iam planning to log the every packet, source IP , Dest IP, Src port, Dest 
port , Type of Service
what kind of system configuration recommended

what kind of  logging system is better ?
iam trying to use Ulogd and try to log every packet, but the kernel crashes 
after 5min

i have Xeon 1GB ram with 40GB Scsi HDD running FC1

How can logging whole history of the packet with the better performance
with out any payload to the user access, and transparently

any suggestions will be appreciated

hare

Re: Logging suggestions

[Taylor, Grant]

Oh, my, that will be tough.  Keep in mind that you will quite likely be generating more data doing the log than you will be having pass through your box, especially if the traffic has a small payload.  I would really be tempted to look at using TCPDump to dump everything to a file and then parse the file.  I don't think the LOG or ULOG target are designed for such high traffic volumes.  Even if they were, LOG logs to SYSLOG, which it in and of it's self is not meant for that high of a volume.  I've never messed with ULOG so I can't say.  Are you needing payload as well or just header information?  I still think TCPDump or some form of an IDS would be a better bet.  Try taking a look at Snort and see if that will come close to what you need.



Grant. . . .

hareram wrote:
> Hi all
> 
> I have large  capacity link like STM with my provider
> Iam planning to log the every packet, source IP , Dest IP, Src port, 
> Dest port , Type of Service
> what kind of system configuration recommended
> 
> what kind of  logging system is better ?
> iam trying to use Ulogd and try to log every packet, but the kernel 
> crashes after 5min
> 
> i have Xeon 1GB ram with 40GB Scsi HDD running FC1
> 
> How can logging whole history of the packet with the better performance
> with out any payload to the user access, and transparently
> 
> any suggestions will be appreciated
> 
> hare
> 
> 
> 

Re: Logging suggestions

[Moritz Gartenmeister]

http://www.qosient.com/argus/

is maybe what you need. it runs stable and i always found the data i needed.

cheers
moritz

Taylor, Grant wrote:
> Oh, my, that will be tough.  Keep in mind that you will quite likely be 
> generating more data doing the log than you will be having pass through 
> your box, especially if the traffic has a small payload.  I would really 
> be tempted to look at using TCPDump to dump everything to a file and 
> then parse the file.  I don't think the LOG or ULOG target are designed 
> for such high traffic volumes.  Even if they were, LOG logs to SYSLOG, 
> which it in and of it's self is not meant for that high of a volume.  
> I've never messed with ULOG so I can't say.  Are you needing payload as 
> well or just header information?  I still think TCPDump or some form of 
> an IDS would be a better bet.  Try taking a look at Snort and see if 
> that will come close to what you need.
> 
> 
> 
> Grant. . . .
> 
> hareram wrote:
> 
>> Hi all
>>
>> I have large  capacity link like STM with my provider
>> Iam planning to log the every packet, source IP , Dest IP, Src port, 
>> Dest port , Type of Service
>> what kind of system configuration recommended
>>
>> what kind of  logging system is better ?
>> iam trying to use Ulogd and try to log every packet, but the kernel 
>> crashes after 5min
>>
>> i have Xeon 1GB ram with 40GB Scsi HDD running FC1
>>
>> How can logging whole history of the packet with the better performance
>> with out any payload to the user access, and transparently
>>
>> any suggestions will be appreciated
>>
>> hare
>>
>>
>>
> 
> 

-- 
Uplink student association
Moritz Gartenmeister
Bülachstrasse 1 F
8057 Zürich
Switzerland

Re: Logging suggestions

[Mariusz Kruk]

On Tue, Apr 12, 2005 at 08:51:54PM +0530, hareram wrote:
> I have large  capacity link like STM with my provider
> Iam planning to log the every packet, source IP , Dest IP, Src port, Dest 
> port , Type of Service
> what kind of system configuration recommended

First of all, I would advise to think if you really really need the
logging of every packet that badly. Perhaps only logging new
connections would be sufficient. Otherwise you'll generate huge amounts
of data. Especially, if your link is quite full.

-- 
/\-\/\-\/\-\/\-\/\-\/\-\/\ Irrelevant  proper  order  of words is, yes.
\  Kruk@epsilon.eu.org   / Yoda of Borg
/ http://epsilon.eu.org/ \ 
\/-/\/-/\/-/\/-/\/-/\/-/\/ 

Re: Logging suggestions

[hareram]

Hi

thanks for the
Suggestions

can you point me to how can loggin only new connections
any docs or examples will be helpfull

hare
----- Original Message ----- 
From: "Mariusz Kruk" <Kruk@epsilon.eu.org>
To: <netfilter@lists.netfilter.org>
Sent: Tuesday, April 12, 2005 9:41 PM
Subject: Re: Logging suggestions


> On Tue, Apr 12, 2005 at 08:51:54PM +0530, hareram wrote:
>> I have large  capacity link like STM with my provider
>> Iam planning to log the every packet, source IP , Dest IP, Src port, Dest
>> port , Type of Service
>> what kind of system configuration recommended
>
> First of all, I would advise to think if you really really need the
> logging of every packet that badly. Perhaps only logging new
> connections would be sufficient. Otherwise you'll generate huge amounts
> of data. Especially, if your link is quite full.
>
> -- 
> /\-\/\-\/\-\/\-\/\-\/\-\/\ Irrelevant  proper  order  of words is, yes.
> \  Kruk@epsilon.eu.org   / Yoda of Borg
> / http://epsilon.eu.org/ \
> \/-/\/-/\/-/\/-/\/-/\/-/\/
>
>
> 

Re: Logging suggestions

[Taylor Grant]

> can you point me to how can loggin only new connections
> any docs or examples will be helpfull

iptables -t filter -A FORWARD -i $INet -o $LAN -m state --state NEW -j LOG



Grant. . . .

Re: Logging suggestions

[Marc Haber]

On Tue, Apr 12, 2005 at 08:51:54PM +0530, hareram wrote:
> I have large  capacity link like STM with my provider
> Iam planning to log the every packet, source IP , Dest IP, Src port, Dest 
> port , Type of Service
> what kind of system configuration recommended

If you want to do this for accounting purposes, then it is feasible to
sum up packet count and packet size. ulog-acctd can be used for that
purpose.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany  |  lose things."    Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature |  How to make an American Quilt | Fax: *49 621 72739835