EOE events in auparse output

EOE events in auparse output

[Nikolai Kondrashov]

Hi Steve, everyone,

I was playing with auditd and aushape on Fedora 24 and found some strange
entries in my log. There was a separate *event* produced by auparse containing
a single EOE record. These events had the same serial number as the directly
preceding events, which were exclusively containing SYSCALL records.

Those EOE records didn't appear in the audit.log file.

Is this a bug? Is this normal?

Thank you.

Nick

Re: EOE events in auparse output

[Steve Grubb]

On Monday, December 5, 2016 3:00:39 PM EST Nikolai Kondrashov wrote:
> I was playing with auditd and aushape on Fedora 24 and found some strange
> entries in my log. There was a separate *event* produced by auparse
> containing a single EOE record. These events had the same serial number as
> the directly preceding events, which were exclusively containing SYSCALL
> records.
> 
> Those EOE records didn't appear in the audit.log file.
> 
> Is this a bug? Is this normal?

The record is not created by auparse. The kernel sends this whenever there is 
a multipart event. This record is stripped before putting the event to disk to 
save disk space. We can get along with this because it can be deduced later 
and running reports from disk is not realtime. On the realtime interface it is 
passed along so that recognizing that an event is complete can occur 
immediately upon receipt. Realtime event processing kind of needs this 
guarantee.

-Steve

Re: EOE events in auparse output

[Nikolai Kondrashov]

Hi Steve,

Thank you for a quick answer, please see my answer below.

On 12/05/2016 05:27 PM, Steve Grubb wrote:
> On Monday, December 5, 2016 3:00:39 PM EST Nikolai Kondrashov wrote:
>> I was playing with auditd and aushape on Fedora 24 and found some strange
>> entries in my log. There was a separate *event* produced by auparse
>> containing a single EOE record. These events had the same serial number as
>> the directly preceding events, which were exclusively containing SYSCALL
>> records.
>>
>> Those EOE records didn't appear in the audit.log file.
>>
>> Is this a bug? Is this normal?
>
> The record is not created by auparse. The kernel sends this whenever there is
> a multipart event. This record is stripped before putting the event to disk to
> save disk space. We can get along with this because it can be deduced later
> and running reports from disk is not realtime. On the realtime interface it is
> passed along so that recognizing that an event is complete can occur
> immediately upon receipt. Realtime event processing kind of needs this
> guarantee.

I was able to find the original patch adding the EOE record output and
discussion around it, so I understand its purpose.

However, since libauparse is supposed to provide the service of communicating
event boundaries to its users, does it make sense for it to return the EOE
record? Especially as a separate, empty event, which doesn't add any
information?

Nick

Re: EOE events in auparse output

[Steve Grubb]

On Monday, December 5, 2016 5:34:12 PM EST Nikolai Kondrashov wrote:
> However, since libauparse is supposed to provide the service of
> communicating event boundaries to its users, does it make sense for it to
> return the EOE record? Especially as a separate, empty event, which doesn't
> add any information?

I suppose it could be stripped from the event as its real purpose is locating 
the event boundary. Since I don't know if the event will be relayed on to 
another analytic processor I've just kept it there. For example, you could 
have a realtime plugin that passes its information to another process for 
correlation and escalation. In that case keeping the record makes sense. But 
for xml/json it can be dropped because it has its own way of defining an event 
boundary.

-Steve

Re: EOE events in auparse output

[Nikolai Kondrashov]

On 12/05/2016 05:54 PM, Steve Grubb wrote:
> On Monday, December 5, 2016 5:34:12 PM EST Nikolai Kondrashov wrote:
>> However, since libauparse is supposed to provide the service of
>> communicating event boundaries to its users, does it make sense for it to
>> return the EOE record? Especially as a separate, empty event, which doesn't
>> add any information?
>
> I suppose it could be stripped from the event as its real purpose is locating
> the event boundary. Since I don't know if the event will be relayed on to
> another analytic processor I've just kept it there. For example, you could
> have a realtime plugin that passes its information to another process for
> correlation and escalation. In that case keeping the record makes sense.

Hmm, perhaps, but I think it's not a useful feature, and a rather confusing
one. I think there aren't many actual use cases between simply passing the raw
log, and passing what was parsed with auparse, in whatever shape, and had
event boundaries explicitly defined already. However, I'd leave this up to
you.

> But for xml/json it can be dropped because it has its own way of defining an
> event boundary.

Yes, I can implement dropping it.

Thanks!

Nick