Consolidate Audit's msgs

Consolidate Audit's msgs

[dump]

Hi,

I was wondering if there had already been an effort or solution to
consolidate msgs from auditd into a single line.
I'm talking about buffering the messages until EOE (or timing out/empty
buffer if EOE doesn't come on errors), and concatenating messages with
the same ID into a single message. Potentially also transforming the
message syntax while at it.

I'm asking because some loggers will only accept specific message formats.

I looked at the plugins, but, from what I gather, the kernel sends the
messages as raw strings and I'm not sure of the performance/memory
impact when auditd cranks out a lot of messages.

An alternative could be to send all the msgs as text to a remote auditd
host using audispd-remote, and processing the log file on that host.
It means even more messages to process however and I'm not sure the text
file interface will be fast enough/might have too much disk activity and
break often, etc. if auditd again, cranks out a lot of messages from
many hosts (like several thousand per second).

Any insight?

Re: Consolidate Audit's msgs

[Steve Grubb]

On Tuesday, January 10, 2012 09:05:19 PM dump@tzib.net wrote:
> I was wondering if there had already been an effort or solution to
> consolidate msgs from auditd into a single line.

No. There was something in the FAQ page about how to do it
http://people.redhat.com/sgrubb/audit/audit-faq.txt

Item #1

> I'm talking about buffering the messages until EOE (or timing out/empty
> buffer if EOE doesn't come on errors), and concatenating messages with
> the same ID into a single message. Potentially also transforming the
> message syntax while at it.
> 
> I'm asking because some loggers will only accept specific message formats.
> 
> I looked at the plugins, but, from what I gather, the kernel sends the
> messages as raw strings and I'm not sure of the performance/memory
> impact when auditd cranks out a lot of messages.

This is a tricky problem. The basic gist is that any manipulation, formating, 
correlation, or sequencing of events is a user space problem from the kernel's 
point of view. There is nothing in the kernel to serialize events. Its not 
uncommon for events to get interlaced with other events. So, ausearch has a 
linked list of linked lists to store events until its certain it has it all. 
Anything doing reformatting would need to do someting like this, too.

> An alternative could be to send all the msgs as text to a remote auditd
> host using audispd-remote, and processing the log file on that host.
> It means even more messages to process however and I'm not sure the text
> file interface will be fast enough/might have too much disk activity and
> break often, etc. if auditd again, cranks out a lot of messages from
> many hosts (like several thousand per second).

Well, I guess you can try things like cgroups and priority boosting to make sure 
the audit daemon and plugins stays ahead of anything generating events.

-Steve