Re: Advice on enriching logs with user and group names before moving them to a central log repository

[Steve Grubb <sgrubb@redhat.com>]

On Friday, August 10, 2012 07:51:29 PM Burn Alting wrote:
> Steve,
> 
> I will go ahead with my audispd child program that enriches logs and use
> rsyslog to get them to a central repository.
> I also plan to concatenate all messages belonging to the same event (ie
> time:event_id) and send this as one syslog message to the central
> repository.
> I'd rather do this on the client systems rather than at my central
> repository, in order to gain benefits from effectively, distributed
> processing.
> 
> I have some concerns though:
>     - Does the concatenation of messages belonging to one event, outside
> of bad code on my part, have some non-obvious risks (from those of you
> who have done this?)

The only problem might be that you will no longer be able to use any of the 
native reporting tools. If you don't use them anyways, then no problem.


>     - I intend that my code will have as small an overhead as I can, but
> do I risk issues such as overruns of the audispd queue?

Yes. You need to make it multi threaded if you do experience overflows with one 
thread dequeueing and another processing.


>     - Do messages from different events ever get intermixed in the
> output via audispd? And hence I need to cater for multiple simultaneous
> events streaming in?

Yes. This is a big problem. About 2 years ago I fixed this in ausearch/report. 
I started to fix this in libauparse but then I remembered it has this state 
machine in it to deal with the feed interface. I didn't write that code so it 
will take some time for me to figure out what it doing before fixing this 
problem. But basically you need a list of lists where each list is a 
collection of records that form one event.
 
 
> I will contribute my code to this list for what's it worth once I've
> completed it ... perhaps it can be added to the contrib/plugin tree
> given it passes this list's peer review.

I do plan to solve this problem at some point. Fixing the libauparse issue 
mentioned above is higher on my priority list.

-Steve