Questions about the standard (Google Summer of Code Project)

Questions about the standard (Google Summer of Code Project)

[Mateusz Piotrowski]

[-- Attachment #1.1: Type: text/plain, Size: 2549 bytes --]

Hi,

I participate in Google Summer of Code and my project involves converting Linux Audit logs to BSM logs.

As I was writing a parser and converter I stumbled upon a couple of things I do not understand and I cannot find in the documentation:

1. Where are all the elements like auditd start, user, etc. listed? I cannot find any document which specifies what can occurs between the colon (separating the type and the msg=audit(…) from the fields) and the record’s fields. 

2. Why are there two spaces between the colon and the first field in records of type=CWD and a field cwd=“/root”? Here’s an example: 

        type=CWD msg=audit(1464013682.961:409):  cwd="/root”

3. According to Red Hat’s documentation[1]:

 > Each record consists of several name=value pairs separated by a white space or a comma.
 
 a) Is a white space always a space? Can be any white space like the tab character?
 b) Why do some records are separated by a comma and a whitespace? Example:

            type=DAEMON_START msg=audit(1363713609.192:5426): auditd start, ver=2.2 format=raw kernel=2.6.32-358.2.1.el6.x86_64 auid=500 pid=4979 subj=unconfined_u:system_r:auditd_t:s0 res=success

 I’ve posted the question on Unix & Linux SE: [3].

4. Is it possible that there are duplicate fields in a record? Something like (which doesn’t make much sense obviously):

        type=CWD msg=audit(1464013682.961:409):  cwd="/root” cwd=“/usr”

 I’ve already asked a similar question on Unix & Linux SE: [4].

5. Is there a document which answers my questions? That would be cool!


Thanks a lot for help!

Cheers!

Matuesz Piotrowski

[GSoC project’s wiki]: https://wiki.freebsd.org/SummerOfCode2016/NonBSMtoBSMConversionTools <https://wiki.freebsd.org/SummerOfCode2016/NonBSMtoBSMConversionTools>
[1]: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Security_Guide/sec-Understanding_Audit_Log_Files.html <https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Security_Guide/sec-Understanding_Audit_Log_Files.html>
[3]: http://unix.stackexchange.com/questions/293975/undocumented-format-of-linux-audit-log-records <http://unix.stackexchange.com/questions/293975/undocumented-format-of-linux-audit-log-records>
[4]: http://unix.stackexchange.com/questions/293809/can-i-be-sure-that-the-name-of-a-linux-audit-records-field-is-unique <http://unix.stackexchange.com/questions/293809/can-i-be-sure-that-the-name-of-a-linux-audit-records-field-is-unique>


[-- Attachment #1.2: Type: text/html, Size: 4074 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Questions about the standard (Google Summer of Code Project)

[Steve Grubb]

Hello,

On Wednesday, July 13, 2016 1:23:29 PM EDT Mateusz Piotrowski wrote:
> I participate in Google Summer of Code and my project involves converting
> Linux Audit logs to BSM logs.
> 
> As I was writing a parser and converter I stumbled upon a couple of things I
> do not understand and I cannot find in the documentation:

The linux audit system has a library, libauparse, that encapsulates all the 
quirks of the audit system so that writing applications like a translator is 
easy. I would recommend using that as a starting point so that you don't have 
to recreate it from scratch.


> 1. Where are all the elements like auditd start, user, etc. listed? I cannot
> find any document which specifies what can occurs between the colon
> (separating the type and the msg=audit(…) from the fields) and the record’s
> fields.

There really is none, Libauparse takes care of all of this so that you don't 
have to. If you are wanting to do translation, you can feed the logs into 
auparse and then just format the event the way you want.

That said, there is a big change coming soon which might make your project 
easier. I'm planning to create a field classification extension to auparse that 
will allow you to say,  "give me the subject of this event", "give me the 
action being performed", "give me the object", "give me the results". This 
would probably make tranlators of all kinds easier to write.


> 2. Why are there two spaces between the colon and the first field in records
> of type=CWD and a field cwd=“/root”? Here’s an example:
> 
>         type=CWD msg=audit(1464013682.961:409):  cwd="/root”

Human error? We use strtok_r to parse and it doesn't care.


> 3. According to Red Hat’s documentation[1]:
>  > Each record consists of several name=value pairs separated by a white
>  > space or a comma.
>  a) Is a white space always a space?

Yes.

>  Can be any white space like the tab character?

No.

>  b) Why do some records are separated by a comma and a
> whitespace? Example:
> 
>             type=DAEMON_START msg=audit(1363713609.192:5426): auditd start,
> ver=2.2 format=raw kernel=2.6.32-358.2.1.el6.x86_64 auid=500 pid=4979
> subj=unconfined_u:system_r:auditd_t:s0 res=success

A long time ago the records were meant to be both human readable (don't laugh) 
and machine consumable. Over time these have been converted name=value pairs. 
Even the one you mention above has been fixed.

>  I’ve posted the question on Unix & Linux SE: [3].
> 
> 4. Is it possible that there are duplicate fields in a record?

Sometimes. I've tried to fix those when it happens. The problem is that not 
everyone runs their audit code by this mail list so that we can check it to 
see that its well formed. What I am planning to do is write an audit event 
validation suite that checks that events are well formed and that expected 
events are being written when they are supposed to and in the order that they 
are supoosed to. Cleaning up these events is high on my TODO list.

> Something
> like (which doesn’t make much sense obviously):
> 
>         type=CWD msg=audit(1464013682.961:409):  cwd="/root” cwd=“/usr”

Something like this will not happen, its more likely around auid and uid. The 
reason being that the kernel adds somethings automatically because its a 
trusted source of information. User space can write contradictory information. 
For example if a daemon is working on behalf of a user but its auid has not 
been set for the user, then you might see this.

>  I’ve already asked a similar question on Unix & Linux SE: [4].

This mail list is where you will get the best answers.

> 5. Is there a document which answers my questions? That would be cool!

https://github.com/linux-audit/audit-documentation/wiki

-Steve

--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit

Re: Questions about the standard (Google Summer of Code Project)

[Mateusz Piotrowski]

Hello,

Thank you for your reply! It is absolutely amazing. It clarified a lot.

>> b) Why do some records are separated by a comma and a
>> whitespace? Example:
>> 
>>            type=DAEMON_START msg=audit(1363713609.192:5426): auditd start,
>> ver=2.2 format=raw kernel=2.6.32-358.2.1.el6.x86_64 auid=500 pid=4979
>> subj=unconfined_u:system_r:auditd_t:s0 res=success
> 
> A long time ago the records were meant to be both human readable (don't laugh) 
> and machine consumable. Over time these have been converted name=value pairs. 
> Even the one you mention above has been fixed.

I am not sure if I understood; does it mean that: `auditd start, ver=2.2` is outdated and deprecated? I’m confused because y Debian did produced a log file with this element. 


Cheers,

-m

--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit

Re: Questions about the standard (Google Summer of Code Project)

[Steve Grubb]

On Thursday, July 14, 2016 6:10:00 PM EDT Mateusz Piotrowski wrote:
> Hello,
> 
> Thank you for your reply! It is absolutely amazing. It clarified a lot.
> 
> >> b) Why do some records are separated by a comma and a
> >> 
> >> whitespace? Example:
> >>            type=DAEMON_START msg=audit(1363713609.192:5426): auditd
> >>            start, ver=2.2 format=raw kernel=2.6.32-358.2.1.el6.x86_64
> >>            auid=500 pid=4979 subj=unconfined_u:system_r:auditd_t:s0
> >>            res=success
> > 
> > A long time ago the records were meant to be both human readable (don't
> > laugh) and machine consumable. Over time these have been converted
> > name=value pairs. Even the one you mention above has been fixed.
> 
> I am not sure if I understood; does it mean that: `auditd start, ver=2.2` is
> outdated and deprecated? I’m confused because Debian did produced a log
> file with this element.

Every time that I find something that doe not conform to name=value, I change 
it or submit a patch. The current output is:

type=DAEMON_START msg=audit(1468495169.149:9305): op=start ver=2.6.5 
format=enriched kernel=4.6.3-300.fc24.x86_64 auid=4294967295 pid=987 
subj=system_u:system_r:auditd_t:s0 res=success\x1dAUID="unset"

Audit 2.2 is from March 2012. That is pretty old now. 

Also notice that this event is in the enriched format. Auparse also hides that 
fact so that you don't need to worry about it. If you write your own parser, 
you'll need to handle both formats.

-Steve

--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit