Filtering out non-interactive users

Filtering out non-interactive users

[PJB]

Hello,

I've recently been working on a number of systems that need to fulfill
auditing requirements for things such as "failed program executions,"
"failed file/directory deletions" and such, and we have been attempting to
use auditd to fulfill these requirements. However we've been having
difficulty filtering out the 'noise' from non-interactive processes since
our requirements only need us to capture these events for real human
users.

In older versions of the audit code, we used the following type of system
call auditing rule which seemed to work pretty well:

-a exit,always -S creat -S open -S openat -S truncate -S ftruncate -F
success=0 -F auid!=-1


Filtering on an 'auid!=-1' seemed to do a very good job of stripping out
system calls from daemon processes and such. However at some point I guess
this was changed because we no longer seem to be able to capture any
system calls at all when we have this filter defined on a rule.

Can someone point me to documentation/examples or help me out with the
proper syntax for setting up rules that will exclude the background
processes? We are using auditd 1.7.4 now and the 'auid' filter above no
longer does the job.

Any help would be very much appreciated! Thanks.
Patrick

Re: Filtering out non-interactive users

[Steve Grubb]

On Friday, January 14, 2011 11:37:01 am PJB wrote:
> I've recently been working on a number of systems that need to fulfill
> auditing requirements for things such as "failed program executions,"
> "failed file/directory deletions" and such, and we have been attempting to
> use auditd to fulfill these requirements. However we've been having
> difficulty filtering out the 'noise' from non-interactive processes since
> our requirements only need us to capture these events for real human
> users.
> 
> In older versions of the audit code, we used the following type of system
> call auditing rule which seemed to work pretty well:
> 
> -a exit,always -S creat -S open -S openat -S truncate -S ftruncate -F
> success=0 -F auid!=-1

This rule looks correct except that if you have a 64 bit system,  I would suggest a -F 
arch=b32 between the '-a' and '-S' and then another copy of the rule for the 64 bit 
arch.
 
> Filtering on an 'auid!=-1' seemed to do a very good job of stripping out
> system calls from daemon processes and such. However at some point I guess
> this was changed because we no longer seem to be able to capture any
> system calls at all when we have this filter defined on a rule.

That should work. I'd try listing the rules back out to see if something is getting 
mis-translated going into the kernel.
 
> Can someone point me to documentation/examples or help me out with the
> proper syntax for setting up rules that will exclude the background
> processes? We are using auditd 1.7.4 now and the 'auid' filter above no
> longer does the job.

There's been a lot of bugs fixed since then. You might try building a newer auditctl 
and trying it out to see if that makes a difference. Also note that the event capturing 
is done by the kernel and the kernel version would matter more than the auditd 
version.

Are you getting other events like logins? Just making sure your disk isn't full or 
something else. And when you do auditctl -s, it shows the audit system is enabled?

-Steve

Re: Filtering out non-interactive users

[PJB]

On Fri, Jan 14, 2011 at 05:21:49PM -0500, Steve Grubb [sgrubb@redhat.com] wrote:
> > In older versions of the audit code, we used the following type of system
> > call auditing rule which seemed to work pretty well:
> > 
> > -a exit,always -S creat -S open -S openat -S truncate -S ftruncate -F
> > success=0 -F auid!=-1
> 
> This rule looks correct except that if you have a 64 bit system,  I would suggest a -F 
> arch=b32 between the '-a' and '-S' and then another copy of the rule for the 64 bit 
> arch.

We are running purely 32-bit systems so I left out the architecture
filter. However while trying to debug I did add it in and it seemed to
make no difference. 
  
> > Can someone point me to documentation/examples or help me out with the
> > proper syntax for setting up rules that will exclude the background
> > processes? We are using auditd 1.7.4 now and the 'auid' filter above no
> > longer does the job.
> 
> There's been a lot of bugs fixed since then. You might try building a newer auditctl 
> and trying it out to see if that makes a difference. Also note that the event capturing 
> is done by the kernel and the kernel version would matter more than the auditd 
> version.

Unfortunately I'm in one of those situations where changing software
versions will cause severe heartburn with management and customer types
due to concerns about baseline stability, so I have to stick with what we
have right now. The kernel is 2.6.33.1 with no extra patches, as far as I
know. 

> Are you getting other events like logins? Just making sure your disk isn't full or 
> something else. And when you do auditctl -s, it shows the audit system is enabled?

We are getting CWD, PATH, and SYSCALL audit events in the log, but only
from files/directories that have an explicit watch set on them. I haven't
seen any other type of audit event other than those three come through,
and again only on things that we set explicit watches on.

Thanks,
Patrick

Re: Filtering out non-interactive users

[Steve Grubb]

On Saturday, January 15, 2011 08:39:30 pm PJB wrote:
> On Fri, Jan 14, 2011 at 05:21:49PM -0500, Steve Grubb [sgrubb@redhat.com] wrote:
> > > In older versions of the audit code, we used the following type of
> > > system call auditing rule which seemed to work pretty well:
> > > 
> > > -a exit,always -S creat -S open -S openat -S truncate -S ftruncate -F
> > > success=0 -F auid!=-1
> > 
> > This rule looks correct except that if you have a 64 bit system,  I would
> > suggest a -F arch=b32 between the '-a' and '-S' and then another copy of
> > the rule for the 64 bit arch.
> 
> We are running purely 32-bit systems so I left out the architecture
> filter. However while trying to debug I did add it in and it seemed to
> make no difference.

Right, it would make no difference on a 32 bit system. One possibility, though, I don't 
know how many people actually use 32 bit systems these days and there could be a bug 
that everyone has missed. Another possibility is that some other rule is preventing 
the auditing to work. You might try deleting all rules and then adding just the one 
rule like this:

# auditctl -a exit,always -F arch=b64 -S creat -S open -S openat -S truncate -S 
ftruncate -F success=0 -F auid!=-1 -k unsuccessful

Then after a few minutes:

# ausearch --start recent -k unsuccessful --raw | aureport --summary --file

I get a bunch of files listed for the -ENOENT return code.


> > > Can someone point me to documentation/examples or help me out with the
> > > proper syntax for setting up rules that will exclude the background
> > > processes? We are using auditd 1.7.4 now and the 'auid' filter above no
> > > longer does the job.
> > 
> > There's been a lot of bugs fixed since then. You might try building a
> > newer auditctl and trying it out to see if that makes a difference. Also
> > note that the event capturing is done by the kernel and the kernel
> > version would matter more than the auditd version.
> 
> Unfortunately I'm in one of those situations where changing software
> versions will cause severe heartburn with management and customer types
> due to concerns about baseline stability, so I have to stick with what we
> have right now. The kernel is 2.6.33.1 with no extra patches, as far as I
> know.

That should work unless the is a 32 bit bug everyone has missed or you have another 
rule preventing the logging. If you do cat /proc/self/loginuid, do you get a number > 
0? Also, if you use auid!=4294967295, does that work?

-Steve
 

> > Are you getting other events like logins? Just making sure your disk
> > isn't full or something else. And when you do auditctl -s, it shows the
> > audit system is enabled?
> 
> We are getting CWD, PATH, and SYSCALL audit events in the log, but only
> from files/directories that have an explicit watch set on them. I haven't
> seen any other type of audit event other than those three come through,
> and again only on things that we set explicit watches on.
> 
> Thanks,
> Patrick
> 
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit

Re: Filtering out non-interactive users

[PJB]

On Sun, Jan 16, 2011 at 10:00:11AM -0500, Steve Grubb [sgrubb@redhat.com] wrote:
> > > > Can someone point me to documentation/examples or help me out with the
> > > > proper syntax for setting up rules that will exclude the background
> > > > processes? We are using auditd 1.7.4 now and the 'auid' filter above no
> > > > longer does the job.
> > > 
> > > There's been a lot of bugs fixed since then. You might try building a
> > > newer auditctl and trying it out to see if that makes a difference. Also
> > > note that the event capturing is done by the kernel and the kernel
> > > version would matter more than the auditd version.
> > 
> > Unfortunately I'm in one of those situations where changing software
> > versions will cause severe heartburn with management and customer types
> > due to concerns about baseline stability, so I have to stick with what we
> > have right now. The kernel is 2.6.33.1 with no extra patches, as far as I
> > know.
> 
> That should work unless the is a 32 bit bug everyone has missed or you have another 
> rule preventing the logging. If you do cat /proc/self/loginuid, do you get a number > 
> 0? Also, if you use auid!=4294967295, does that work?

The loginuid is 4294967295. If I pass '-F auid!=4294967295' into the
filters, when I run 'auditctl -l' the rules are listed, but each one has
'auid=2147483647 (0x7fffffff)'. I get log entries then, but they are all
tagged with auid 4294967295. Is this proper or did I stumble upon a bug
after all?

I've managed a workaround for most of my systems; since we do not permit
direct root login to anything, using a filter of '-F uid!=0' manages to
filter out most of the background activity. However I do have a couple of
systems that only have a root user so this method does not work. 

Thanks again!
Patrick

Re: Filtering out non-interactive users

[Steve Grubb]

On Wednesday, January 19, 2011 09:01:55 am PJB wrote:
> > That should work unless the is a 32 bit bug everyone has missed or you
> > have another  rule preventing the logging. If you do cat
> > /proc/self/loginuid, do you get a number > 0? Also, if you use
> > auid!=4294967295, does that work?
> 
> The loginuid is 4294967295. If I pass '-F auid!=4294967295' into the
> filters, when I run 'auditctl -l' the rules are listed, but each one has
> 'auid=2147483647 (0x7fffffff)'. I get log entries then, but they are all
> tagged with auid 4294967295. Is this proper or did I stumble upon a bug
> after all?

That is a 32 bit bug. I'm looking at how best to solve this. Probably all variants of 
uid and gid are affected by this.

-Steve

Re: Filtering out non-interactive users

[PJB]

On Wed, Jan 19, 2011 at 09:33:30AM -0500, Steve Grubb [sgrubb@redhat.com] wrote:
> On Wednesday, January 19, 2011 09:01:55 am PJB wrote:
> > > That should work unless the is a 32 bit bug everyone has missed or you
> > > have another  rule preventing the logging. If you do cat
> > > /proc/self/loginuid, do you get a number > 0? Also, if you use
> > > auid!=4294967295, does that work?
> > 
> > The loginuid is 4294967295. If I pass '-F auid!=4294967295' into the
> > filters, when I run 'auditctl -l' the rules are listed, but each one has
> > 'auid=2147483647 (0x7fffffff)'. I get log entries then, but they are all
> > tagged with auid 4294967295. Is this proper or did I stumble upon a bug
> > after all?
> 
> That is a 32 bit bug. I'm looking at how best to solve this. Probably all variants of 
> uid and gid are affected by this.

I was afraid you would say that! Would it be a bug in the auditd userspace
programs or in the kernel code? I assume it's the former?

Patrick

Re: Filtering out non-interactive users

[Steve Grubb]

On Wednesday, January 19, 2011 09:48:25 am PJB wrote:
> On Wed, Jan 19, 2011 at 09:33:30AM -0500, Steve Grubb [sgrubb@redhat.com] wrote:
> > On Wednesday, January 19, 2011 09:01:55 am PJB wrote:
> > > > That should work unless the is a 32 bit bug everyone has missed or
> > > > you have another  rule preventing the logging. If you do cat
> > > > /proc/self/loginuid, do you get a number > 0? Also, if you use
> > > > auid!=4294967295, does that work?
> > > 
> > > The loginuid is 4294967295. If I pass '-F auid!=4294967295' into the
> > > filters, when I run 'auditctl -l' the rules are listed, but each one
> > > has 'auid=2147483647 (0x7fffffff)'. I get log entries then, but they
> > > are all tagged with auid 4294967295. Is this proper or did I stumble
> > > upon a bug after all?
> > 
> > That is a 32 bit bug. I'm looking at how best to solve this. Probably all
> > variants of uid and gid are affected by this.
> 
> I was afraid you would say that! Would it be a bug in the auditd userspace
> programs or in the kernel code? I assume it's the former?

Specifically, the bug is right here:
https://fedorahosted.org/audit/browser/trunk/lib/libaudit.c#L1134

AUID on 32 bit should be treated like the AUDIT_INODE field. Still looking at it...

-Steve

Re: Filtering out non-interactive users

[Steve Grubb]

On Wednesday, January 19, 2011 09:01:55 am PJB wrote:
> On Sun, Jan 16, 2011 at 10:00:11AM -0500, Steve Grubb [sgrubb@redhat.com] wrote:
> > > > > Can someone point me to documentation/examples or help me out with
> > > > > the proper syntax for setting up rules that will exclude the
> > > > > background processes? We are using auditd 1.7.4 now and the 'auid'
> > > > > filter above no longer does the job.

I note that you say you are using 1.7.4. I tried to replicate the problem on a 686 VM. 
I got different results from you.

#auditctl -a always,exit -S open -F success=0 -F auid!=4294967295
# auditctl -l
LIST_RULES: exit,always success=0 auid!=-1 (0xffffffff) syscall=open

So, then I started bisecting the code until found this commit:

https://fedorahosted.org/audit/changeset/268

So, you would need audit version 1.7.13 or later. Please try again with a newer audit 
package. Sorry for speaking too soon.

-Steve