Grouping audit events in an auditd log parser

Grouping audit events in an auditd log parser

[Alan Evangelista]

[-- Attachment #1.1: Type: text/plain, Size: 1323 bytes --]

Hi! I was using auditbeat to connect to the audit kernel module and get
filesystem operations events from it. However, as I discussed in another
thread, it seems that the audit events kernel queue is buggy in kernel 3.1,
the kernel version available on CentOS 7. This means that if
auditbeat crashes for any reason, I'll start losing FS events. In order to
make my event detection more resilient, I decided to move the critical
point of failure from auditbeat to auditd: I'll let auditd write a log file
and write a fluentd parser to read from that log file.

Writing that auditd log parser, I reached a question: how to group audit
records in a FS event (for instance, there are SYSCALL, CWD and PATHs audit
events for a single file creation, deletion or renaming). Can I assume that
*all* records of an event will always appear sequentially in the log file
with the same timestamp/event ID or records from different filesystem
operations can be interleaved? Ex: simultaneous fsop 1 and fsop 2 could
show up like:

TYPE:SYSCALL msg=audit(167111.123:1) (...)
TYPE:CWD msg=audit(167111.123:1) (...)
TYPE:SYSCALL msg=audit(167112.123:2) (...)
TYPE:PATH msg=audit(167111.123:1) (...)
TYPE:PATH msg=audit(167111.123:1) (...)
TYPE:CWD msg=audit(167112.123:1) (...)
TYPE:PATH msg=audit(167112.123:1) (...)

?

Thanks in advance!

[-- Attachment #1.2: Type: text/html, Size: 1557 bytes --]

[-- Attachment #2: Type: text/plain, Size: 106 bytes --]

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Grouping audit events in an auditd log parser

[Steve Grubb]

On Tuesday, April 6, 2021 9:03:50 AM EDT Alan Evangelista wrote:
> Hi! I was using auditbeat to connect to the audit kernel module and get
> filesystem operations events from it. However, as I discussed in another
> thread, it seems that the audit events kernel queue is buggy in kernel 3.1,
> the kernel version available on CentOS 7. This means that if
> auditbeat crashes for any reason, I'll start losing FS events. In order to
> make my event detection more resilient, I decided to move the critical
> point of failure from auditbeat to auditd: I'll let auditd write a log file
> and write a fluentd parser to read from that log file.
> 
> Writing that auditd log parser, I reached a question: how to group audit
> records in a FS event (for instance, there are SYSCALL, CWD and PATHs audit
> events for a single file creation, deletion or renaming). Can I assume
> that *all* records of an event will always appear sequentially in the log
> file with the same timestamp/event ID or records from different filesystem
> operations can be interleaved? Ex: simultaneous fsop 1 and fsop 2 could
> show up like:
> 
> TYPE:SYSCALL msg=audit(167111.123:1) (...)
> TYPE:CWD msg=audit(167111.123:1) (...)
> TYPE:SYSCALL msg=audit(167112.123:2) (...)
> TYPE:PATH msg=audit(167111.123:1) (...)
> TYPE:PATH msg=audit(167111.123:1) (...)
> TYPE:CWD msg=audit(167112.123:1) (...)
> TYPE:PATH msg=audit(167112.123:1) (...)
> 
> ?

Nope. You cannot assume that. Serialization is left as an exercise  for user 
space. Also, there is an audit parsing library that knows how to group events 
and hides all additional complexity. You can either use it to parse the log 
directly or to get the events from the realtime interface.

-Steve


--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit

Re: Grouping audit events in an auditd log parser

[Lenny Bruzenak]

On 4/6/21 8:55 AM, Steve Grubb wrote:

> On Tuesday, April 6, 2021 9:03:50 AM EDT Alan Evangelista wrote:
>> Hi! I was using auditbeat to connect to the audit kernel module and get
>> filesystem operations events from it. However, as I discussed in another
>> thread, it seems that the audit events kernel queue is buggy in kernel 3.1,
>> the kernel version available on CentOS 7. This means that if
>> auditbeat crashes for any reason, I'll start losing FS events. In order to
>> make my event detection more resilient, I decided to move the critical
>> point of failure from auditbeat to auditd: I'll let auditd write a log file
>> and write a fluentd parser to read from that log file.
>>
>> Writing that auditd log parser, I reached a question: how to group audit
>> records in a FS event (for instance, there are SYSCALL, CWD and PATHs audit
>> events for a single file creation, deletion or renaming). Can I assume
>> that *all* records of an event will always appear sequentially in the log
>> file with the same timestamp/event ID or records from different filesystem
>> operations can be interleaved? Ex: simultaneous fsop 1 and fsop 2 could
>> show up like:
>>
>> TYPE:SYSCALL msg=audit(167111.123:1) (...)
>> TYPE:CWD msg=audit(167111.123:1) (...)
>> TYPE:SYSCALL msg=audit(167112.123:2) (...)
>> TYPE:PATH msg=audit(167111.123:1) (...)
>> TYPE:PATH msg=audit(167111.123:1) (...)
>> TYPE:CWD msg=audit(167112.123:1) (...)
>> TYPE:PATH msg=audit(167112.123:1) (...)
>>
>> ?
> Nope. You cannot assume that. Serialization is left as an exercise  for user 
> space. Also, there is an audit parsing library that knows how to group events 
> and hides all additional complexity. You can either use it to parse the log 
> directly or to get the events from the realtime interface.
>
> -Steve
>

See :
http://security-plus-data-science.blogspot.com/2017/06/using-auparse-in-python.html

Sounds like you could create a custom audisp plugin to do what you want.

LCB

-- 
Lenny Bruzenak
MagitekLTD

--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit