should I loose audit data if I only care about the record's fields?

should I loose audit data if I only care about the record's fields?

[Klaus Heinrich Kiwi]

Hi,

 when I started building my dispatcher plug-in, I assumed that I'd only
need the fields values in each record to have all the data I needed. My
plug-in for remote logging aimed at consolidating the audit data in
another server, so I probably need all the audit data I can get from the
Audit subsystem, possibly in a format that is compatible with the target
system (thus using the record fields for mapping)

Giving another look the some audit records, I saw that this approach was
probably not sufficient to describe the audited operation as a whole.

Example record:
type=USER_CHAUTHTOK msg=audit(1194995431.057:58485): user pid=30759
uid=0 auid=0 subj=root:system_r:unconfined_t:s0-s0:c0.c1023
msg='op=adding user to shadow group acct=klausk
exe="/usr/sbin/usermod" (hostname=?, addr=?, terminal=pts/1
res=success)'

using walk_test() from the test routine (python):
---
event 1 has 1 records
    record 1 of type 1108(USER_CHAUTHTOK) has 12 fields
    line=1 file=None
    event time: 1194995431.57:58485, host=None
        type=USER_CHAUTHTOK (USER_CHAUTHTOK)
        pid=30759 (30759)
        uid=0 (root)
        auid=0 (root)
        subj=root:system_r:unconfined_t:s0-s0:c0.c1023
(root:system_r:unconfined_t:s0-s0:c0.c1023)
        op=adding (adding)
        acct=klausk (klausk)
        exe="/usr/sbin/usermod" (/usr/sbin/usermod)
        hostname=? (?)
        addr=? (?)
        terminal=pts/1 (pts/1)
        res=success (success)
---
'op=adding' - adding what? no information about what's going on here.
_side note_: just noticed that the original record is telling 'adding
user to shadow group' when in fact I was adding the user to the 'nobody'
group, plus others, with 'usermod -G' - I'll check that again later.

Another example is the LOGIN record:
original record:
type=LOGIN msg=audit(1193547601.367:36782): login pid=11698 uid=0 old
auid=4294967295 new auid=0

---walk_test()----
event 1 has 1 records
    record 1 of type 1006(LOGIN) has 5 fields
    line=1 file=None
    event time: 1193547601.367:36782, host=None
        type=LOGIN (LOGIN)
        pid=11698 (11698)
        uid=0 (root)
        auid=4294967295 (unset)
        auid=0 (root)
---
two auid fields? which is old and which is new? ok maybe not the
brightest example but IMO still valid.

There are probably more examples besides those two. 

Maybe auparse is aimed to just help us when we need to extract data, but
it is well-settled that someone will need the whole record to actually
know what's going on - please tell me if that is the case.

Thoughts?

 Klaus
 

Re: should I loose audit data if I only care about the record's fields?

[John Dennis]

Klaus Heinrich Kiwi wrote:
> Hi,
> 
>  when I started building my dispatcher plug-in, I assumed that I'd only
> need the fields values in each record to have all the data I needed. My
> plug-in for remote logging aimed at consolidating the audit data in
> another server, so I probably need all the audit data I can get from the
> Audit subsystem, possibly in a format that is compatible with the target
> system (thus using the record fields for mapping)
> 
> Giving another look the some audit records, I saw that this approach was
> probably not sufficient to describe the audited operation as a whole.
> 
> Example record:
> type=USER_CHAUTHTOK msg=audit(1194995431.057:58485): user pid=30759
> uid=0 auid=0 subj=root:system_r:unconfined_t:s0-s0:c0.c1023
> msg='op=adding user to shadow group acct=klausk
> exe="/usr/sbin/usermod" (hostname=?, addr=?, terminal=pts/1
> res=success)'
> 
> using walk_test() from the test routine (python):
> ---
> event 1 has 1 records
>     record 1 of type 1108(USER_CHAUTHTOK) has 12 fields
>     line=1 file=None
>     event time: 1194995431.57:58485, host=None
>         type=USER_CHAUTHTOK (USER_CHAUTHTOK)
>         pid=30759 (30759)
>         uid=0 (root)
>         auid=0 (root)
>         subj=root:system_r:unconfined_t:s0-s0:c0.c1023
> (root:system_r:unconfined_t:s0-s0:c0.c1023)
>         op=adding (adding)
>         acct=klausk (klausk)
>         exe="/usr/sbin/usermod" (/usr/sbin/usermod)
>         hostname=? (?)
>         addr=? (?)
>         terminal=pts/1 (pts/1)
>         res=success (success)
> ---
> 'op=adding' - adding what? no information about what's going on here.
> _side note_: just noticed that the original record is telling 'adding
> user to shadow group' when in fact I was adding the user to the 'nobody'
> group, plus others, with 'usermod -G' - I'll check that again later.
> 
> Another example is the LOGIN record:
> original record:
> type=LOGIN msg=audit(1193547601.367:36782): login pid=11698 uid=0 old
> auid=4294967295 new auid=0
> 
> ---walk_test()----
> event 1 has 1 records
>     record 1 of type 1006(LOGIN) has 5 fields
>     line=1 file=None
>     event time: 1193547601.367:36782, host=None
>         type=LOGIN (LOGIN)
>         pid=11698 (11698)
>         uid=0 (root)
>         auid=4294967295 (unset)
>         auid=0 (root)
> ---
> two auid fields? which is old and which is new? ok maybe not the
> brightest example but IMO still valid.
> 
> There are probably more examples besides those two. 
> 
> Maybe auparse is aimed to just help us when we need to extract data, but
> it is well-settled that someone will need the whole record to actually
> know what's going on - please tell me if that is the case.

I'm not sure follow what you're asking but let me through a few thoughts 
out.

auparse only purpose is to parse and extract audit data, it doesn't 
interpret the data (other than simple things like mapping numeric values 
to strings). The job of interpreting the audit event is up to the caller 
of auparse. During interpretation only you can know what data you need 
to look at. There may or may not be a relationship between individual 
events, you're going to have to perform some analysis across events to 
determine if that is the case.

When you say "only need the field values" are you saying you're throwing 
away the field names and if so what about multiple fields with the same 
name?

Also, be aware records in an event cannot be merged because the same 
fields may appear in more than one record, if you do that you'll have 
collisions and lose all but the most recent field value you read out of 
the event.

In the above example, if the op field was really a multiword string but 
its value only appeared as the first word in the string, then that looks 
like a bug. I'm not personally familiar with that field in that record.



> Thoughts?
> 
>  Klaus
>  
> 
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit


-- 
John Dennis <jdennis@redhat.com>

Re: should I loose audit data if I only care about the record's fields?

[klausk]

[-- Attachment #1.1: Type: text/plain, Size: 3077 bytes --]

(sorry about the html posting)

John Dennis <jdennis@redhat.com> wrote on 11/14/2007 09:30:16 AM:

> > original record:
> > type=LOGIN msg=audit(1193547601.367:36782): login pid=11698 uid=0 old
> > auid=4294967295 new auid=0
> > 
> > ---walk_test()----
> > event 1 has 1 records
> >     record 1 of type 1006(LOGIN) has 5 fields
> >     line=1 file=None
> >     event time: 1193547601.367:36782, host=None
> >         type=LOGIN (LOGIN)
> >         pid=11698 (11698)
> >         uid=0 (root)
> >         auid=4294967295 (unset)
> >         auid=0 (root)
> > ---

> I'm not sure follow what you're asking but let me through a few thoughts 

> out.
> 
> auparse only purpose is to parse and extract audit data, it doesn't 
> interpret the data (other than simple things like mapping numeric values 

> to strings). The job of interpreting the audit event is up to the caller 

> of auparse. During interpretation only you can know what data you need 
> to look at. There may or may not be a relationship between individual 
> events, you're going to have to perform some analysis across events to 
> determine if that is the case.
> 
> When you say "only need the field values" are you saying you're throwing 

> away the field names and if so what about multiple fields with the same 
> name?

That got out bad. I wanted to know if by having only the event type, 
event timestamp+serial, and all the fieldname=value pairs would be
sufficient to describe an audit record completely. Look at the above
example (taken out of auparse testsuite). Can I describe the record
as a whole if I just have the data that 'walk_test()' gives me? I'd
expect so, but there are still some audit data not placed in any
field.

> Also, be aware records in an event cannot be merged because the same 
> fields may appear in more than one record, if you do that you'll have 
> collisions and lose all but the most recent field value you read out of 
> the event.

I'm not merging records. In the plug-in, I use the same concept applied
in the Linux Audit subsystem, that is having a unique identifier shared
among all the records of a same event.

> In the above example, if the op field was really a multiword string but 
> its value only appeared as the first word in the string, then that looks 

> like a bug. I'm not personally familiar with that field in that record.

It seems like a field value cannot contain any spaces, so the 'operation'
specified by usermod should be something like 
'op=adding-supplemental-group-to-user new_group=sys acct=klausk'.

But then again we have another issue: it's up to the application to choose
the operation name, the field name and what it means - or if there is any
field at all! This many degrees of freedom may mean hell to people who 
actually are trying to extract information from these records.

Standard formats, standard fields names with well-known meanings would 
certainly help.

 Thanks,

 Klaus
-- 
Klaus Heinrich Kiwi/Brazil/IBM <klausk@br.ibm.com>
Software Engineer
IBM STG, Linux Technology Center
Phone:(+55-19) 2132-1909 [T/L 839-1909]

[-- Attachment #1.2: Type: text/html, Size: 4435 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: should I loose audit data if I only care about the record's fields?

[Steve Grubb]

On Wednesday 14 November 2007 10:24:46 klausk@br.ibm.com wrote:
> It seems like a field value cannot contain any spaces,

Correct. I've mentioned that many times in different code reviews. You can add 
data to the field for human use, but it won't be associated with the field's 
value directly.

> so the 'operation' specified by usermod should be something like
> 'op=adding-supplemental-group-to-user new_group=sys acct=klausk'.

Yes, if all that text is needed. Sometimes, though, a new record type should 
be created to carry the meaning.

> But then again we have another issue: it's up to the application to choose
> the operation name, the field name and what it means - or if there is any
> field at all! This many degrees of freedom may mean hell to people who
> actually are trying to extract information from these records.

Exactly. The audit fields are a gentlemen's agreement. This is why I've added 
specific audit logging functions is to try to enforce some kind of 
reasonableness to it. But, there are still fields that could get filled in 
with spaces and make the parser not get it all.


> Standard formats, standard fields names with well-known meanings would
> certainly help.

That is what I've been trying to do with the audit logging functions. Some 
people have complained about it. I guess they wanted freedom. But the logging 
functions are the only way to guarantee that certain necessary fields get 
recorded. I can't really do much about the content that gets recorded within 
a field...

-Steve

Re: should I loose audit data if I only care about the record's fields?

[Steve Grubb]

On Tuesday 13 November 2007 18:30:45 Klaus Heinrich Kiwi wrote:
> Example record:
> type=USER_CHAUTHTOK msg=audit(1194995431.057:58485): user pid=30759
> uid=0 auid=0 subj=root:system_r:unconfined_t:s0-s0:c0.c1023
> msg='op=adding user to shadow group acct=klausk
> exe="/usr/sbin/usermod" (hostname=?, addr=?, terminal=pts/1
> res=success)'
>
> using walk_test() from the test routine (python):
> ---
>         op=adding (adding)
> ---
> 'op=adding' - adding what? no information about what's going on here.

This is an audit record that should probably be fixed in the application's 
source code.

> _side note_: just noticed that the original record is telling 'adding
> user to shadow group' when in fact I was adding the user to the 'nobody'
> group, plus others, with 'usermod -G' - I'll check that again later.

Yeah, might be a bug. shadow-utils is horrible for auditing since it has so 
many exit points that need to be audited. In my opinion, all the apps in it 
need restructuring for the logging/auditing.

> Another example is the LOGIN record:
> original record:
> type=LOGIN msg=audit(1193547601.367:36782): login pid=11698 uid=0 old
> auid=4294967295 new auid=0
>
> ---walk_test()----
> event 1 has 1 records
>     record 1 of type 1006(LOGIN) has 5 fields
>     line=1 file=None
>     event time: 1193547601.367:36782, host=None
>         type=LOGIN (LOGIN)
>         pid=11698 (11698)
>         uid=0 (root)
>         auid=4294967295 (unset)
>         auid=0 (root)
> ---
> two auid fields? which is old and which is new? ok maybe not the
> brightest example but IMO still valid.

Yep, that is implicit in the ordering. 

> Maybe auparse is aimed to just help us when we need to extract data, but
> it is well-settled that someone will need the whole record to actually
> know what's going on - please tell me if that is the case.

You can access the whole record with auparse_get_record_text().


> Thoughts?

There is also a section of code that is not written. There are plans to access 
the "in-between" data as an ancillary field. I believe there are FIXME's in 
the code where this should be. Unfortunately, I can't get to it for a little 
while.

-Steve