Linux-audit Archive on lore.kernel.org
 help / color / mirror / Atom feed
* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Guenter Roeck @ 2015-01-22 16:58 UTC (permalink / raw)
  To: Paul Moore; +Cc: linux-fsdevel, linux-audit, rgb, sd, linux-kernel, viro
In-Reply-To: <3252765.lrITUkIS9l@sifl>

On 01/22/2015 08:22 AM, Paul Moore wrote:
> On Wednesday, January 21, 2015 09:36:34 PM Guenter Roeck wrote:
>> On 01/21/2015 08:59 PM, Paul Moore wrote:
>>> This patchset has some important changes from the previous revision,
>>> namely a fix from Al Viro (included in 2/5) that resolves a boot panic
>>> on some systems as well as some smaller, less noteworthy fixes found
>>> in the linux-next announcement thread from January 20th (refcount bump
>>> in __audit_reusename() and a inode type in __audit_inode()).
>>>
>>> This patchset still needs some additional testing to verify that the
>>> audit code still functions properly (the minor fixes mentioned above)
>>> and there is an additional patch from Al that should be included as
>>> well, but I wanted to post this and push the series to the audit next
>>> branch quickly since a number of folks were affected by the boot panic.
>>>
>>> ---
>>>
>>> Paul Moore (5):
>>>         fs: rework getname_kernel to handle up to PATH_MAX sized filenames
>>>         fs: create proper filename objects using getname_kernel()
>>>         audit: enable filename recording via getname_kernel()
>>>         audit: fix filename matching in __audit_inode() and
>>>         __audit_inode_child()
>>>         audit: replace getname()/putname() hacks with reference counters
>>
>> Hi Paul,
>>
>> What is the baseline for this patch set ? Obviously -next won't work,
>> and it does not apply to mainline either.
>
> This patchset currently lives, along with one other unrelated patch, in the
> audit next branch:
>
>   * git://git.infradead.org/users/pcmoore/audit
>
> I'm currently testing these in combination with the patch Al posted last
> night.  Assuming all goes well I'll drop them from the audit next branch and
> toss all six patches (these plus Al's) into another branch in case Al wants to
> pull them for the VFS tree.
>
The version in the audit next tree works with my qemu microblaze test,
so feel free to add

Tested-by: Guenter Roeck <linux@roeck-us.net>

Guenter

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Guenter Roeck @ 2015-01-22 17:13 UTC (permalink / raw)
  To: Al Viro; +Cc: Paul Moore, linux-fsdevel, linux-audit, rgb, sd, linux-kernel
In-Reply-To: <20150122075429.GV29656@ZenIV.linux.org.uk>

On 01/21/2015 11:54 PM, Al Viro wrote:
> On Wed, Jan 21, 2015 at 09:36:34PM -0800, Guenter Roeck wrote:
>> On 01/21/2015 08:59 PM, Paul Moore wrote:
>>> This patchset has some important changes from the previous revision,
>>> namely a fix from Al Viro (included in 2/5) that resolves a boot panic
>>> on some systems as well as some smaller, less noteworthy fixes found
>>> in the linux-next announcement thread from January 20th (refcount bump
>>> in __audit_reusename() and a inode type in __audit_inode()).
>>>
>>> This patchset still needs some additional testing to verify that the
>>> audit code still functions properly (the minor fixes mentioned above)
>>> and there is an additional patch from Al that should be included as
>>> well, but I wanted to post this and push the series to the audit next
>>> branch quickly since a number of folks were affected by the boot panic.
>>>
>>> ---
>>>
>>> Paul Moore (5):
>>>        fs: rework getname_kernel to handle up to PATH_MAX sized filenames
>>>        fs: create proper filename objects using getname_kernel()
>>>        audit: enable filename recording via getname_kernel()
>>>        audit: fix filename matching in __audit_inode() and __audit_inode_child()
>>>        audit: replace getname()/putname() hacks with reference counters
>>>
>> Hi Paul,
>>
>> What is the baseline for this patch set ? Obviously -next won't work,
>> and it does not apply to mainline either.
>
> FWIW, I've ported that on top of vfs.git#for-next; result is in
> vfs.git#experimental.  Paul, are you OK with that one?
>
I also tested that one. Works as expected.

Thanks,
Guenter

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Paul Moore @ 2015-01-22 21:25 UTC (permalink / raw)
  To: Al Viro; +Cc: Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd, linux-kernel
In-Reply-To: <4011794.crgx99Gu8a@sifl>

On Thursday, January 22, 2015 11:23:44 AM Paul Moore wrote:
> On Thursday, January 22, 2015 07:54:29 AM Al Viro wrote:
> > On Wed, Jan 21, 2015 at 09:36:34PM -0800, Guenter Roeck wrote:
> > > On 01/21/2015 08:59 PM, Paul Moore wrote:
> > > >This patchset has some important changes from the previous revision,
> > > >namely a fix from Al Viro (included in 2/5) that resolves a boot panic
> > > >on some systems as well as some smaller, less noteworthy fixes found
> > > >in the linux-next announcement thread from January 20th (refcount bump
> > > >in __audit_reusename() and a inode type in __audit_inode()).
> > > >
> > > >This patchset still needs some additional testing to verify that the
> > > >audit code still functions properly (the minor fixes mentioned above)
> > > >and there is an additional patch from Al that should be included as
> > > >well, but I wanted to post this and push the series to the audit next
> > > >branch quickly since a number of folks were affected by the boot panic.
> > > >
> > > >---
> > > >
> > > >Paul Moore (5):
> > > >       fs: rework getname_kernel to handle up to PATH_MAX sized
> > > >       filenames
> > > >       fs: create proper filename objects using getname_kernel()
> > > >       audit: enable filename recording via getname_kernel()
> > > >       audit: fix filename matching in __audit_inode() and
> > > >       __audit_inode_child()
> > > >       audit: replace getname()/putname() hacks with reference counters
> > > 
> > > Hi Paul,
> > > 
> > > What is the baseline for this patch set ? Obviously -next won't work,
> > > and it does not apply to mainline either.
> > 
> > FWIW, I've ported that on top of vfs.git#for-next; result is in
> > vfs.git#experimental.  Paul, are you OK with that one?
> 
> Okay, hang on let me test that ...

Your experimental branch looks good to me, thanks.

-- 
paul moore
security @ redhat


^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Al Viro @ 2015-01-22 21:29 UTC (permalink / raw)
  To: Paul Moore
  Cc: Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd, linux-kernel
In-Reply-To: <25860171.h6aocaXFeh@sifl>

On Thu, Jan 22, 2015 at 04:25:13PM -0500, Paul Moore wrote:

> Your experimental branch looks good to me, thanks.

Pushed into for-next; I'm probably going to move that stuff into a never-rebased
branch, merged into for-next and safe to pull into your tree if you want to do
something on top of that set.

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Al Viro @ 2015-01-22 21:40 UTC (permalink / raw)
  To: Paul Moore
  Cc: Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd, linux-kernel
In-Reply-To: <20150122212902.GW29656@ZenIV.linux.org.uk>

On Thu, Jan 22, 2015 at 09:29:03PM +0000, Al Viro wrote:
> On Thu, Jan 22, 2015 at 04:25:13PM -0500, Paul Moore wrote:
> 
> > Your experimental branch looks good to me, thanks.
> 
> Pushed into for-next; I'm probably going to move that stuff into a never-rebased
> branch, merged into for-next and safe to pull into your tree if you want to do
> something on top of that set.

OK, vfs.git#getname is it; it's in never-to-be-rebased mode and it's merged
into vfs.git#for-next (as of right now; HEAD is 9ee4c4).  If you need to do
something on top of that stuff, pulling vfs.git#getname is safe.

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Paul Moore @ 2015-01-22 22:05 UTC (permalink / raw)
  To: Al Viro; +Cc: Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd, linux-kernel
In-Reply-To: <20150122214000.GX29656@ZenIV.linux.org.uk>

On Thursday, January 22, 2015 09:40:01 PM Al Viro wrote:
> On Thu, Jan 22, 2015 at 09:29:03PM +0000, Al Viro wrote:
> > On Thu, Jan 22, 2015 at 04:25:13PM -0500, Paul Moore wrote:
> > > Your experimental branch looks good to me, thanks.
> > 
> > Pushed into for-next; I'm probably going to move that stuff into a
> > never-rebased branch, merged into for-next and safe to pull into your
> > tree if you want to do something on top of that set.
> 
> OK, vfs.git#getname is it; it's in never-to-be-rebased mode and it's merged
> into vfs.git#for-next (as of right now; HEAD is 9ee4c4).  If you need to do
> something on top of that stuff, pulling vfs.git#getname is safe.

Great, thanks.

-- 
paul moore
security @ redhat

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Al Viro @ 2015-01-23  5:30 UTC (permalink / raw)
  To: Paul Moore
  Cc: Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd, linux-kernel
In-Reply-To: <20150122214000.GX29656@ZenIV.linux.org.uk>

On Thu, Jan 22, 2015 at 09:40:01PM +0000, Al Viro wrote:
> On Thu, Jan 22, 2015 at 09:29:03PM +0000, Al Viro wrote:
> > On Thu, Jan 22, 2015 at 04:25:13PM -0500, Paul Moore wrote:
> > 
> > > Your experimental branch looks good to me, thanks.
> > 
> > Pushed into for-next; I'm probably going to move that stuff into a never-rebased
> > branch, merged into for-next and safe to pull into your tree if you want to do
> > something on top of that set.
> 
> OK, vfs.git#getname is it; it's in never-to-be-rebased mode and it's merged
> into vfs.git#for-next (as of right now; HEAD is 9ee4c4).  If you need to do
> something on top of that stuff, pulling vfs.git#getname is safe.

Unfortunately, that thing was -rc2-based, leading to conflict with mainline
in kernel/auditsc.c.  My fault, I hadn't realized that "audit: create private
file name copies when auditing inodes" in audit tree was, in fact, present in
mainline.  vfs.git#getname2 is -rc3-based, same resulting kernel/auditsc.c as
in #getname.  Please, use that.  vfs.git#for-next merges from that one now,
so tomorrow -next should have no problems from vfs.git...

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Paul Moore @ 2015-01-23 16:15 UTC (permalink / raw)
  To: Al Viro; +Cc: Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd, linux-kernel
In-Reply-To: <20150123053056.GC29656@ZenIV.linux.org.uk>

On Friday, January 23, 2015 05:30:56 AM Al Viro wrote:
> On Thu, Jan 22, 2015 at 09:40:01PM +0000, Al Viro wrote:
> > On Thu, Jan 22, 2015 at 09:29:03PM +0000, Al Viro wrote:
> > > On Thu, Jan 22, 2015 at 04:25:13PM -0500, Paul Moore wrote:
> > > > Your experimental branch looks good to me, thanks.
> > > 
> > > Pushed into for-next; I'm probably going to move that stuff into a
> > > never-rebased branch, merged into for-next and safe to pull into your
> > > tree if you want to do something on top of that set.
> > 
> > OK, vfs.git#getname is it; it's in never-to-be-rebased mode and it's
> > merged
> > into vfs.git#for-next (as of right now; HEAD is 9ee4c4).  If you need to
> > do
> > something on top of that stuff, pulling vfs.git#getname is safe.
> 
> Unfortunately, that thing was -rc2-based, leading to conflict with mainline
> in kernel/auditsc.c.  My fault, I hadn't realized that "audit: create
> private file name copies when auditing inodes" in audit tree was, in fact,
> present in mainline.  vfs.git#getname2 is -rc3-based, same resulting
> kernel/auditsc.c as in #getname.  Please, use that.  vfs.git#for-next
> merges from that one now, so tomorrow -next should have no problems from
> vfs.git...

No worries, thanks for the update.

-- 
paul moore
security @ redhat

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Sedat Dilek @ 2015-01-24  9:03 UTC (permalink / raw)
  To: Al Viro
  Cc: Paul Moore, Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd,
	LKML
In-Reply-To: <20150123053056.GC29656@ZenIV.linux.org.uk>

On Fri, Jan 23, 2015 at 6:30 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Thu, Jan 22, 2015 at 09:40:01PM +0000, Al Viro wrote:
>> On Thu, Jan 22, 2015 at 09:29:03PM +0000, Al Viro wrote:
>> > On Thu, Jan 22, 2015 at 04:25:13PM -0500, Paul Moore wrote:
>> >
>> > > Your experimental branch looks good to me, thanks.
>> >
>> > Pushed into for-next; I'm probably going to move that stuff into a never-rebased
>> > branch, merged into for-next and safe to pull into your tree if you want to do
>> > something on top of that set.
>>
>> OK, vfs.git#getname is it; it's in never-to-be-rebased mode and it's merged
>> into vfs.git#for-next (as of right now; HEAD is 9ee4c4).  If you need to do
>> something on top of that stuff, pulling vfs.git#getname is safe.
>
> Unfortunately, that thing was -rc2-based, leading to conflict with mainline
> in kernel/auditsc.c.  My fault, I hadn't realized that "audit: create private
> file name copies when auditing inodes" in audit tree was, in fact, present in
> mainline.  vfs.git#getname2 is -rc3-based, same resulting kernel/auditsc.c as
> in #getname.  Please, use that.  vfs.git#for-next merges from that one now,
> so tomorrow -next should have no problems from vfs.git...
>

I have tested vfs.git#getname2 on top of Linux v3.19-rc5-184-gc4e00f1
(plus block-loopmq patchset) and it boots fine on Ubuntu/precise
amd64.

Just curious, where will this audit-filename-handling overhaul go through?
Through Paul's audit-next or Al's vfs-next tree?

AFAICS, a new linux-next will be available on Monday (2015-01-26).
I try to retest with this.

- Sedat -

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Stephen Rothwell @ 2015-01-24 22:54 UTC (permalink / raw)
  To: Sedat Dilek
  Cc: Al Viro, Paul Moore, Guenter Roeck, linux-fsdevel, linux-audit,
	rgb, sd, LKML
In-Reply-To: <CA+icZUWOkW-ujARrnT4T3wsZh0EbkHR+p2q2c1-zJYKtP46hfA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 318 bytes --]

Hi Sedat,

On Sat, 24 Jan 2015 10:03:06 +0100 Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> AFAICS, a new linux-next will be available on Monday (2015-01-26).
> I try to retest with this.

Actually, Tuesday - Monday is a holiday here.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply

* Re: Does the order / position of audit rule's arguments matter?
From: Steve Grubb @ 2015-01-25 18:05 UTC (permalink / raw)
  To: Jan Lieskovsky; +Cc: Richard Guy Briggs, Shawn Wells, linux-audit
In-Reply-To: <2143821220.13855761.1421691556035.JavaMail.zimbra@redhat.com>

On Monday, January 19, 2015 01:19:16 PM Jan Lieskovsky wrote:
> Thank you both for quick replies.
> 
> ----- Original Message -----
> 
> > From: "Steve Grubb" <sgrubb@redhat.com>
> > To: "Richard Guy Briggs" <rgb@redhat.com>
> > Cc: linux-audit@redhat.com, "Jan Lieskovsky" <jlieskov@redhat.com>, "Shawn
> > Wells" <shawn@redhat.com> Sent: Monday, January 19, 2015 7:11:10 PM
> > Subject: Re: Does the order / position of audit rule's arguments matter?
> > 
> > On Monday, January 19, 2015 01:06:42 PM Richard Guy Briggs wrote:
> > > On 15/01/19, Steve Grubb wrote:
> > > > On Monday, January 19, 2015 12:57:11 PM Jan Lieskovsky wrote:
> > > > > Hello folks,
> > > > > 
> > > > >   wasn't able to find answer to the following question in the
> > > > >   auditctl
> > > > > 
> > > > > manual page, thus checking here - does the order / position in which
> > > > > the
> > > > > auditctl's | /etc/audit/audit.rules' audit rule arguments are listed
> > > > > in
> > > > > the rule matter or all permutations of the arguments are allowed?
> > > > 
> > > > Yes, its a first match wins system. I tell people to order from
> > > > specific
> > > > to
> > > > general. IOW, put a watch on /etc/shadow before a watch on /etc.
> > > 
> > > I don't think that answers Jan's question.  I understood the question to
> > > be the ordering of arguments *within* a rule.
> 
> Yes, was about this case. But good to know also order of rules matters
> (to list them that way).
> 
> >  I believe the answer is
> >  
> > > "no".
> > > 
> > > so:
> > > 	-a always,exit -F path=/bin/ping -F perm=x -F auid>=500 -F
> > 
> > auid!=4294967295
> > 
> > > -k privileged would be equivalent to:
> > > 	-a always,exit -F path=/bin/ping -F perm=x -F auid!=4294967295 -F
> > 
> > auid>=500
> > 
> > > -k privileged
> > 
> > If that is the case, then you want to have the fields in the order in
> > which
> > the
> > system can decide "no" as fast as possible.
> 
> Meaning the audit rule's arguments to be sorted? Or just follow the form
> as it's listed for example in /usr/share/doc/audit-2.3.7/stig.rules file?
> (IOW action first, then path to binary, then other -F arguments - for these
> to be listed in ascending alphabetical order?)

Yes, the audit system is like
for each rule
  if (syscall & mask)
     for each field
        if fval != rval
           next rule
        endif
     done
     record event
  endif
done

So, by deciding "no" quickly, it can move on to the next rule to iterate over 
the fields. 

The stig.rules is ordered correctly and is intended that anyone that needs to 
STIG a box simply copies the file "as is" to the rules.d directory. Any 
problems between what the file has and what's needed should be reported here so 
the file can be corrected.

-Steve

^ permalink raw reply

* Re: [PATCH v2 0/5] Overhaul the audit filename handling
From: Paul Moore @ 2015-01-26 15:52 UTC (permalink / raw)
  To: sedat.dilek
  Cc: Al Viro, Guenter Roeck, linux-fsdevel, linux-audit, rgb, sd, LKML
In-Reply-To: <CA+icZUWOkW-ujARrnT4T3wsZh0EbkHR+p2q2c1-zJYKtP46hfA@mail.gmail.com>

On Saturday, January 24, 2015 10:03:06 AM Sedat Dilek wrote:
> I have tested vfs.git#getname2 on top of Linux v3.19-rc5-184-gc4e00f1
> (plus block-loopmq patchset) and it boots fine on Ubuntu/precise
> amd64.

Great, thank you for the additional testing.
 
> Just curious, where will this audit-filename-handling overhaul go through?
> Through Paul's audit-next or Al's vfs-next tree?

Al wanted to carry the patchset so that is where it lives now, you should see 
it arrive in Linus' tree via the VFS tree.

-- 
paul moore
security @ redhat

^ permalink raw reply

* Catching process termination on SIGKILL
From: hsultan @ 2015-01-26 23:14 UTC (permalink / raw)
  To: linux-audit

Hi,

So I'm curious, auditd catches abnormal process termination (SIGSEGV, 
...) with a 1701 audit message, can catch 'clean' termination by 
monitoring syscall (exit, exitgroup), however I don't see anything to 
catch process termination by a SIGKILL.
if I audit the kill() system call then I see the call to send the 
signal, but I would have expected the system to offer auditing of an 
actual SIGKILL *reception* (because you can pass -1 as target PID to 
sigkill, which kills all processes reachable by the caller and will make 
auditing by syscall very hard to do), am I missing something ? Is there 
a parameter to set somehow that I'm missing ?

Thanks,

Hassan

^ permalink raw reply

* Re: Catching process termination on SIGKILL
From: Steve Grubb @ 2015-01-27  0:41 UTC (permalink / raw)
  To: linux-audit
In-Reply-To: <5601880178188ab58cf241b359ebf97d@thefroid.net>

On Monday, January 26, 2015 03:14:20 PM hsultan@thefroid.net wrote:
> So I'm curious, auditd catches abnormal process termination (SIGSEGV,
> ...) with a 1701 audit message, can catch 'clean' termination by
> monitoring syscall (exit, exitgroup), however I don't see anything to
> catch process termination by a SIGKILL.
> if I audit the kill() system call then I see the call to send the
> signal, but I would have expected the system to offer auditing of an
> actual SIGKILL *reception* (because you can pass -1 as target PID to
> sigkill, which kills all processes reachable by the caller and will make
> auditing by syscall very hard to do), am I missing something ?

I don't think so.

> Is there a parameter to set somehow that I'm missing ?

No. This would probably need some kind of kernel patch to enable. Its never 
really come up that anyone would want to monitor for this. Typically the 
monitoring is on the sending side rather than the receiving side.

We collect anything that leads to a core dump because that is an anomally. No 
one should have segfaulting code on a production system. However, the kernel 
does not allow a SIGKILL to be delivered to processes the user has no rights 
to send it to, so its not really an abnormal event. I could see someone maybe 
wanting to monitor this, but its never been a priority to solve this problem.

-Steve

^ permalink raw reply

* Re: Detecting loading of libraries
From: Steve Grubb @ 2015-01-27  0:48 UTC (permalink / raw)
  To: linux-audit
In-Reply-To: <5dc8468401e6007eaad18a0b7d782927@thefroid.net>

On Wednesday, January 21, 2015 04:01:59 PM hsultan@thefroid.net wrote:
> I'm wondering if there's a good way of detecting the loading of
> libraries by processes (I am specifically NOT talking about the uselib
> syscall).

This has never been a problem people needed a solution for. Its always been 
assumed that the runtime linker does the right thing.

> strace shows me apps do open(...)/mmap/mprotect
> I'm currently intercepting mmap calls, however no additional context
> records are given to provide the name of the library, and the file
> descriptor is the 5th parameter, so I can't get that either to match it
> to an open(...)
> 
> Is there a way to do this that I'm missing ?

I'd almost thing you'd want to patch ld.so to provide this...but then its not 
running as a privileged process. So, it can't do it. Ld is the thing that 
knows the _intent_ behind the open and mmap and mprot. Nothing else does.

-Steve

^ permalink raw reply

* Re: Catching process termination on SIGKILL
From: hsultan @ 2015-01-27  1:56 UTC (permalink / raw)
  To: linux-audit
In-Reply-To: <4299392.Ypj558huPe@x2>

On 2015-01-26 16:41, Steve Grubb wrote:
> On Monday, January 26, 2015 03:14:20 PM hsultan@thefroid.net wrote:
>> So I'm curious, auditd catches abnormal process termination 
>> (SIGSEGV,
>> ...) with a 1701 audit message, can catch 'clean' termination by
>> monitoring syscall (exit, exitgroup), however I don't see anything 
>> to
>> catch process termination by a SIGKILL.
>> if I audit the kill() system call then I see the call to send the
>> signal, but I would have expected the system to offer auditing of an
>> actual SIGKILL *reception* (because you can pass -1 as target PID to
>> sigkill, which kills all processes reachable by the caller and will 
>> make
>> auditing by syscall very hard to do), am I missing something ?
>
> I don't think so.
>
>> Is there a parameter to set somehow that I'm missing ?
>
> No. This would probably need some kind of kernel patch to enable. Its 
> never
> really come up that anyone would want to monitor for this. Typically 
> the
> monitoring is on the sending side rather than the receiving side.
>
> We collect anything that leads to a core dump because that is an 
> anomally. No
> one should have segfaulting code on a production system. However, the 
> kernel
> does not allow a SIGKILL to be delivered to processes the user has no 
> rights
> to send it to, so its not really an abnormal event. I could see 
> someone maybe
> wanting to monitor this, but its never been a priority to solve this 
> problem.

I see. Auditing SIGKILL reception would allow for easy tracking of 
process activity by following clone/fork/vfork/exit/exit group/abnormal 
termination and then SIGKILL. Without it, it becomes a kludge requiring 
to track kill/tkill/tgkill and trying to find which process will accept 
the SIGKILL sent and which won't, which then requires keeping track of 
process privileges and such.

I'll try to figure out what a patch to audit the KILL reception would 
look like, intent would be to provide the sender's PID + the target PID 
in the audit msg. Should that be a new AUDIT msg type or do you see it 
fit within an existing msg type ?

Thanks,

Hassan

^ permalink raw reply

* Re: Catching process termination on SIGKILL
From: Tetsuo Handa @ 2015-01-27 12:11 UTC (permalink / raw)
  To: hsultan; +Cc: linux-audit
In-Reply-To: <45a5a4d7425943aa52df4117448cf2ce@thefroid.net>

Hassan wrote:
> On 2015-01-26 16:41, Steve Grubb wrote:
> > We collect anything that leads to a core dump because that is an 
> > anomally. No
> > one should have segfaulting code on a production system. However, the 
> > kernel
> > does not allow a SIGKILL to be delivered to processes the user has no 
> > rights
> > to send it to, so its not really an abnormal event. I could see 
> > someone maybe
> > wanting to monitor this, but its never been a priority to solve this 
> > problem.

Well, the OOM killer can deliver SIGKILL to processes the user has no rights
to send it to. ;-)

> I see. Auditing SIGKILL reception would allow for easy tracking of 
> process activity by following clone/fork/vfork/exit/exit group/abnormal 
> termination and then SIGKILL. Without it, it becomes a kludge requiring 
> to track kill/tkill/tgkill and trying to find which process will accept 
> the SIGKILL sent and which won't, which then requires keeping track of 
> process privileges and such.

Do you have to implement it using audit subsystem? If you want to track
process activity for temporary (or debug) purpose, SystemTap would do it.

---------- program start ----------
# stap -e '
probe kernel.function("do_exit") {
  if ($code & 0x7F)
    printf("%s %s(%u) exiting with signal %u\n",
           ctime(gettimeofday_s()), execname(), pid(), $code & 0x7F);
}'
---------- program end ----------

---------- output example start ----------
Sat May 3 06:00:39 2014 a.out(2101) exiting with signal 11
Sat May 3 06:00:48 2014 sleep(2102) exiting with signal 2
Sat May 3 06:01:17 2014 sleep(2105) exiting with signal 9
Sat May 3 06:01:21 2014 a.out(2131) exiting with signal 11
---------- output example end ----------

> 
> I'll try to figure out what a patch to audit the KILL reception would 
> look like, intent would be to provide the sender's PID + the target PID 
> in the audit msg. Should that be a new AUDIT msg type or do you see it 
> fit within an existing msg type ?

SystemTap would do it, if you can accept SystemTap.

^ permalink raw reply

* Re: Catching process termination on SIGKILL
From: hsultan @ 2015-01-27 19:03 UTC (permalink / raw)
  To: Tetsuo Handa; +Cc: linux-audit
In-Reply-To: <201501272111.HED17151.FOQOLFOFMSHJVt@I-love.SAKURA.ne.jp>

On 2015-01-27 04:11, Tetsuo Handa wrote:
...
> Do you have to implement it using audit subsystem? If you want to 
> track
> process activity for temporary (or debug) purpose, SystemTap would do 
> it.
>
> ---------- program start ----------
> # stap -e '
> probe kernel.function("do_exit") {
>   if ($code & 0x7F)
>     printf("%s %s(%u) exiting with signal %u\n",
>            ctime(gettimeofday_s()), execname(), pid(), $code & 0x7F);
> }'
> ---------- program end ----------
>
> ---------- output example start ----------
> Sat May 3 06:00:39 2014 a.out(2101) exiting with signal 11
> Sat May 3 06:00:48 2014 sleep(2102) exiting with signal 2
> Sat May 3 06:01:17 2014 sleep(2105) exiting with signal 9
> Sat May 3 06:01:21 2014 a.out(2131) exiting with signal 11
> ---------- output example end ----------
>
>>
>> I'll try to figure out what a patch to audit the KILL reception 
>> would
>> look like, intent would be to provide the sender's PID + the target 
>> PID
>> in the audit msg. Should that be a new AUDIT msg type or do you see 
>> it
>> fit within an existing msg type ?
>
> SystemTap would do it, if you can accept SystemTap.

Sadly I can't use SystemTap as I do not control the systems where my 
code will be running so can't be sure that debug information will be 
available :/

Thanks,

Hassan

^ permalink raw reply

* [PATCH 1/2] audit: don't lose set wait time on first successful call to audit_log_start()
From: Richard Guy Briggs @ 2015-01-28  0:34 UTC (permalink / raw)
  To: linux-audit; +Cc: Richard Guy Briggs, eparis

Copy the set wait time to a working value to avoid losing the set value if the
queue overflows.

Signed-off-by: Richard Guy Briggs <rgb@redhat.com>
---
 kernel/audit.c |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/kernel/audit.c b/kernel/audit.c
index 53bb39b..b333f03 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -107,6 +107,7 @@ static u32	audit_rate_limit;
  * When set to zero, this means unlimited. */
 static u32	audit_backlog_limit = 64;
 #define AUDIT_BACKLOG_WAIT_TIME (60 * HZ)
+static u32	audit_backlog_wait_time_master = AUDIT_BACKLOG_WAIT_TIME;
 static u32	audit_backlog_wait_time = AUDIT_BACKLOG_WAIT_TIME;
 static u32	audit_backlog_wait_overflow = 0;
 
@@ -338,7 +339,7 @@ static int audit_set_backlog_limit(u32 limit)
 static int audit_set_backlog_wait_time(u32 timeout)
 {
 	return audit_do_config_change("audit_backlog_wait_time",
-				      &audit_backlog_wait_time, timeout);
+				      &audit_backlog_wait_time_master, timeout);
 }
 
 static int audit_set_enabled(u32 state)
@@ -843,7 +844,7 @@ static int audit_receive_msg(struct sk_buff *skb, struct nlmsghdr *nlh)
 		s.lost			= atomic_read(&audit_lost);
 		s.backlog		= skb_queue_len(&audit_skb_queue);
 		s.version		= AUDIT_VERSION_LATEST;
-		s.backlog_wait_time	= audit_backlog_wait_time;
+		s.backlog_wait_time	= audit_backlog_wait_time_master;
 		audit_send_reply(skb, seq, AUDIT_GET, 0, 0, &s, sizeof(s));
 		break;
 	}
@@ -1394,7 +1395,7 @@ struct audit_buffer *audit_log_start(struct audit_context *ctx, gfp_t gfp_mask,
 		return NULL;
 	}
 
-	audit_backlog_wait_time = AUDIT_BACKLOG_WAIT_TIME;
+	audit_backlog_wait_time = audit_backlog_wait_time_master;
 
 	ab = audit_buffer_alloc(ctx, gfp_mask, type);
 	if (!ab) {
-- 
1.7.1

^ permalink raw reply related

* [PATCH 2/2] audit: don't reset working wait time accidentally with auditd
From: Richard Guy Briggs @ 2015-01-28  0:34 UTC (permalink / raw)
  To: linux-audit; +Cc: Richard Guy Briggs, eparis
In-Reply-To: <2192ffc51189b5caa7d7172d59fea6fcc8bf07a5.1422392773.git.rgb@redhat.com>

During a queue overflow condition while we are waiting for auditd to drain the
queue to make room for regular messages, we don't want a successful auditd that
has bypassed the queue check to reset the backlog wait time.

Signed-off-by: Richard Guy Briggs <rgb@redhat.com>
---
 kernel/audit.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/kernel/audit.c b/kernel/audit.c
index b333f03..73293ea 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -1395,7 +1395,8 @@ struct audit_buffer *audit_log_start(struct audit_context *ctx, gfp_t gfp_mask,
 		return NULL;
 	}
 
-	audit_backlog_wait_time = audit_backlog_wait_time_master;
+	if (!reserve)
+		audit_backlog_wait_time = audit_backlog_wait_time_master;
 
 	ab = audit_buffer_alloc(ctx, gfp_mask, type);
 	if (!ab) {
-- 
1.7.1

^ permalink raw reply related

* Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-01-28 14:57 UTC (permalink / raw)
  To: linux-audit@redhat.com


[-- Attachment #1.1: Type: text/plain, Size: 1625 bytes --]

Hi Steve,

I am Logeswari working for HP.

We want to know audit performance impact on RHEL and Suse linux to help us evaluate linux audit as data source for our host based IDS.
When we ran our own performance test with a test audispd plugin, we found if a system can perform 200000 open/close system calls per second without auditing, system can perform only 3000 open/close system calls auditing is enabled for open/close system call which is a HUGE impact on the system performance. It would be great if anyone can help us answering the following questions.


1)      Is this performance impact expected? If yes, what is the reason behind it and can we fix it?

2)      Have anyone done any benchmarking for performance impact? If yes, can you please share the numbers and also the steps/programs used the run the same.

3)      Help us validating the performance test we have done in our test setup using the steps mentioned along with the results attached.

Attached test program (loader.c) to invoke open and close system calls.
Attached idskerndsp is the audispd plugin program.
We used time command to determine how much time the system took to complete 50000 open/close system calls without (results attached Without-auditing) and with auditing enabled on the system (With-auditing-NOLOG-audispd-plugin and With-auditing-RAW)

System details:

1 CPU machine

OS Version
RHEL 6.5

Kernel Version
uname -r
2.6.32-431.el6.x86_64

Note: auditd was occupying 35% of CPU and was sleeping for most of the time whereas kauditd was occupying 20% of the CPU.

Thanks & Regards,
Logeswari.



[-- Attachment #1.2: Type: text/html, Size: 6956 bytes --]

[-- Attachment #2: loader.c --]
[-- Type: text/plain, Size: 4359 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>

void create_load(int iters);

int   high_rate = 0;
int   num_iters = 50000;
int   fd1;
char  file1[50];

/* Purpose: To create system load by invoking system calls used by templates.
 *
 * Note: The unlink(2) of a file can be an expensive operation (i.e., event 
 *       rate goes way down).
 * Note: Needs to be run as a non-ids user since IDDS is typically configured
 *       to not audit ids. Some system calls below require you to run as root.
 */

main(int argc, char **argv) {

  int              num_children;
  int              iters;
  int              i;
  char             c;
  struct passwd   *passwd_entry;

  while ((c = getopt(argc, argv, "hi:")) != -1) {
    switch (c) {
    case 'h':
      /*
       * Desire "high" event rate
       */
      high_rate = 1;
      argc--;
      break;
    case 'i':
      /*
       * Desire a specified number of iterations
       */
      num_iters = atoi(optarg);
      argc--;
      break;
    default:
      fprintf(stderr,"Unknown option: %c\n",optarg);
      exit(1);
    }
  }
 
  if(argv[optind] != NULL) {
    num_children = atoi(argv[optind]);
  } else {
    num_children = 4;
  }
  num_children = 1;

  /* fork child processes, if any requested */
  for(i=1; i < num_children; i++) {
    if(fork() == 0) {

      printf("child pid: %d\n",getpid());

      /* Setup file names based on child's pid */
      //sprintf(file1,"./file1_%d",getpid());

      /* each child creates load */	
      iters=0;
      if (num_iters == -1) {
	while(1) {
	  create_load(iters);
	  iters++;
	  if( (iters % 1000) == 0) {
	    printf("pid %d iteration %d\n",getpid(),iters);
	  }
	}
      } else {
	while(iters < num_iters) {
	  create_load(iters);
	  iters++;
	  if( (iters % 1000) == 0) {
	    printf("pid %d iteration %d\n",getpid(),iters);
	  }
	}
      }
    }
  }

  /* Parent creates load also */
  printf("parent pid: %d\n",getpid());

  /* Setup file names based on parent's pid */
  //sprintf(file1,"./file1_%d",getpid());

  iters=0;
  if (num_iters == -1) {
    while(1) {
      create_load(iters);
      iters++;
      if( (iters % 1000) == 0) {
	printf("pid %d iteration %d\n",getpid(),iters);
      }
    }
  } else {
    while(iters < num_iters) {
      create_load(iters);
      iters++;
      if( (iters % 1000) == 0) {
	printf("pid %d iteration %d\n",getpid(),iters);
      }
    }
  }

} /* main */


void create_load(int iters) {

  int pid;
  char *args[2];
  struct stat stat_buf;

  fd1 = open("file1", O_RDWR, 0777);
  if (fd1 == -1) {
    fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
	    getpid(),errno,strerror(errno));
    exit(1);
  }

  if (close(fd1) == -1) {
      fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
	      getpid(),errno,strerror(errno));
      exit(1);
  }

  /*if (chown("file1",0,0) == -1) {
   fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
           getpid(),0,0,errno,strerror(errno));
    exit(1);
  }

  pid = fork();
  if(pid == 0) {
      fprintf(stderr,"child pid %d: fork!\n",getpid());
      args[0] = "/bin/ls";
      args[1] = NULL;
      close(1);
      close(2);
      execve(args[0], args, NULL);
      fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
              getpid(),args[0],errno,strerror(errno));
      _exit(1);
    } else if (pid < 0) {
      fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
              getpid(),errno,strerror(errno));
      exit(1);
    } else {
      fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
    }

    pid = vfork();
    if(pid == 0) {
      args[0] = "/bin/pwd";
      args[1] = NULL;
      close(1);
      close(2);
      execv(args[0], args);
      fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
              getpid(),args[0],errno,strerror(errno));
      _exit(1);
    } else if (pid < 0) {
      fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
              getpid(),errno,strerror(errno));
      exit(1);
  }*/

  return;
} /* create_load() */

[-- Attachment #3: idskerndsp.c --]
[-- Type: text/plain, Size: 1513 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <syslog.h>
#include <string.h>
#include <ctype.h>
#include <pwd.h>
#include <sys/stat.h>
#include <sys/select.h>
#include <errno.h>
#include "libaudit.h"
#include "auparse.h"

/* Global Data */
static auparse_state_t *au = NULL;


/* Local declarations */
static void handle_event(auparse_state_t *au,
		auparse_cb_event_t cb_event_type, void *user_data);

int main(int argc, char *argv[])
{
	char tmp[MAX_AUDIT_MESSAGE_LENGTH+1];

	/* Initialize the auparse library */
	au = auparse_init(AUSOURCE_FEED, 0);
	if (au == NULL) {
		return -1;
	}
	auparse_add_callback(au, handle_event, NULL, NULL);
	
	do {
		fd_set read_mask;
		struct timeval tv;
		int retval;

		do {
			tv.tv_sec = 5;
			tv.tv_usec = 0;
			FD_ZERO(&read_mask);
			FD_SET(0, &read_mask);
			retval= select(1, &read_mask, NULL, NULL, &tv);
                } while (retval == -1 && errno == EINTR);

		/* Now the event loop */
		if (retval > 0) {
			if (fgets_unlocked(tmp, MAX_AUDIT_MESSAGE_LENGTH,
				stdin)){
				auparse_feed(au, tmp, strnlen(tmp,
						MAX_AUDIT_MESSAGE_LENGTH));
			}
		} else if (retval == 0)
			auparse_flush_feed(au);
		if (feof(stdin))
			break;
	} while (1);

	/* Flush any accumulated events from queue */
	auparse_flush_feed(au);

	auparse_destroy(au);
	return 0;
}

static void handle_event(auparse_state_t *au,
		auparse_cb_event_t cb_event_type, void *user_data)
{
	return;
}


[-- Attachment #4: Without-auditing.txt --]
[-- Type: text/plain, Size: 219 bytes --]

Audit Status
# auditctl -s
AUDIT_STATUS: enabled=0 flag=1 pid=20358 rate_limit=0 backlog_limit=320 lost=0 backlog=0

Without auditing enabled, time taken is 

real    0m0.252s
user    0m0.018s
sys     0m0.215s

[-- Attachment #5: With-auditing-NOLOG-audispd-plugin.txt --]
[-- Type: text/plain, Size: 520 bytes --]

audispd-plugin configuration

# cat /etc/audisp/plugins.d/idskerndsp.conf

active = yes
direction = out
path = /ux/ids/idskerndsp
type = always
args = --test
format = string

Rules Configured

# auditctl -l
LIST_RULES: exit,always syscall=open,close

Audit Status

# auditctl -s
AUDIT_STATUS: enabled=1 flag=1 pid=20358 rate_limit=0 backlog_limit=320 lost=0 backlog=0

With log_format = NOLOG, above rule enabled for auditing, time taken is

real    0m16.849s
user    0m0.045s
sys     0m3.838s

[-- Attachment #6: With-auditing-RAW.txt --]
[-- Type: text/plain, Size: 629 bytes --]

We tried to disable the plugin i.e. idskerndsp and restarted auditd process to log the audit events to disk.

audispd-plugin configuration

# cat /etc/audisp/plugins.d/idskerndsp.conf

active = no
direction = out
path = /ux/ids/idskerndsp
type = always
args = --test
format = string

Rules Configured

# auditctl -l
LIST_RULES: exit,always syscall=open,close

Audit Status

# auditctl -s
AUDIT_STATUS: enabled=1 flag=1 pid=20819 rate_limit=0 backlog_limit=320 lost=0 backlog=0

With log_format = RAW, above rule enabled for auditing, time taken is

real    2m41.484s
user    0m0.028s
sys     0m8.789s

[-- Attachment #7: Type: text/plain, Size: 0 bytes --]



^ permalink raw reply

* Re: Linux audit performance impact
From: Steve Grubb @ 2015-01-28 15:16 UTC (permalink / raw)
  To: linux-audit; +Cc: Viswanath, Logeswari P (MCOU OSTL)
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB9141659CA2@G4W3222.americas.hpqcorp.net>

Hello,

On Wednesday, January 28, 2015 02:57:58 PM Viswanath, Logeswari P wrote:
> We want to know audit performance impact on RHEL and Suse linux to help us
> evaluate linux audit as data source for our host based IDS. When we ran our
> own performance test with a test audispd plugin, we found if a system can
> perform 200000 open/close system calls per second without auditing, system
> can perform only 3000 open/close system calls auditing is enabled for
> open/close system call which is a HUGE impact on the system performance. It
> would be great if anyone can help us answering the following questions.
> 
> 
> 1)      Is this performance impact expected? If yes, what is the reason
> behind it and can we fix it?

I'll leave this for the kernel guys to answer. That said, I think more 
detailed information might be helpful.

If auditd is not started and events go to syslog, does the performance change? 
To do this audit=1 on boot line and auditctl -R /etc/rules.d/your.rules

what rules do you have loaded?

What do you get when audit is enabled and no rules loaded?

If you have other syscall rules loaded that are not open and openat or close, 
does the performance change? I suspect that if you trigger a rule, you are 
thrown onto the slow path. Open is perhaps the most lengthy because of 
multiple auxiliary records and path resolution. But we need data to tell.

That said, I know that the kernel audit path changed a couple years ago so it 
might be worthwhile to test against an old kernel to see if the change has 
affected performance.

-Steve

> 2)      Have anyone done any benchmarking for performance impact? If yes,
> can you please share the numbers and also the steps/programs used the run
> the same.
> 
> 3)      Help us validating the performance test we have done in our test
> setup using the steps mentioned along with the results attached.
> 
> Attached test program (loader.c) to invoke open and close system calls.
> Attached idskerndsp is the audispd plugin program.
> We used time command to determine how much time the system took to complete
> 50000 open/close system calls without (results attached Without-auditing)
> and with auditing enabled on the system (With-auditing-NOLOG-audispd-plugin
> and With-auditing-RAW)
> 
> System details:
> 
> 1 CPU machine
> 
> OS Version
> RHEL 6.5
> 
> Kernel Version
> uname -r
> 2.6.32-431.el6.x86_64
> 
> Note: auditd was occupying 35% of CPU and was sleeping for most of the time
> whereas kauditd was occupying 20% of the CPU.
> 
> Thanks & Regards,
> Logeswari.

^ permalink raw reply

* Re: Linux audit performance impact
From: Satish Chandra Kilaru @ 2015-01-28 15:18 UTC (permalink / raw)
  To: Viswanath, Logeswari P (MCOU OSTL); +Cc: linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB9141659CA2@G4W3222.americas.hpqcorp.net>


[-- Attachment #1.1: Type: text/plain, Size: 2142 bytes --]

Write your own program to receive audit events directly without using
auditd...
That should be faster ....
Auditd will log the events to disk causing more I/o than u need...

On Wednesday, January 28, 2015, Viswanath, Logeswari P (MCOU OSTL) <
logeswari.pv@hp.com> wrote:

>  Hi Steve,
>
>
>
> I am Logeswari working for HP.
>
>
>
> We want to know audit performance impact on RHEL and Suse linux to help us
> evaluate linux audit as data source for our host based IDS.
>
> When we ran our own performance test with a test audispd plugin, we found
> if a system can perform 200000 open/close system calls per second without
> auditing, system can perform only 3000 open/close system calls auditing is
> enabled for open/close system call which is a HUGE impact on the system
> performance. It would be great if anyone can help us answering the
> following questions.
>
>
>
> 1)      Is this performance impact expected? If yes, what is the reason
> behind it and can we fix it?
>
> 2)      Have anyone done any benchmarking for performance impact? If yes,
> can you please share the numbers and also the steps/programs used the run
> the same.
>
> 3)      Help us validating the performance test we have done in our test
> setup using the steps mentioned along with the results attached.
>
>
>
> Attached test program (loader.c) to invoke open and close system calls.
>
> Attached idskerndsp is the audispd plugin program.
>
> We used time command to determine how much time the system took to
> complete 50000 open/close system calls without (results attached
> Without-auditing) and with auditing enabled on the system
> (With-auditing-NOLOG-audispd-plugin and With-auditing-RAW)
>
>
>
> System details:
>
>
>
> 1 CPU machine
>
>
>
> *OS Version*
>
> RHEL 6.5
>
>
>
> *Kernel Version*
>
> uname –r
>
> 2.6.32-431.el6.x86_64
>
>
>
> Note: auditd was occupying 35% of CPU and was sleeping for most of the
> time whereas kauditd was occupying 20% of the CPU.
>
>
>
> Thanks & Regards,
>
> Logeswari.
>
>
>
>
>


-- 
Please Donate to www.wikipedia.org

[-- Attachment #1.2: Type: text/html, Size: 3839 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]



^ permalink raw reply

* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-01-28 15:52 UTC (permalink / raw)
  To: Steve Grubb, linux-audit@redhat.com
In-Reply-To: <5473650.MEjjDQdHXo@x2>

Hi Steve,

Thanks for the quick reply.

Please look in-line for my replies.

Regards,
Logeswari.

-----Original Message-----
From: Steve Grubb [mailto:sgrubb@redhat.com] 
Sent: Wednesday, January 28, 2015 8:46 PM
To: linux-audit@redhat.com
Cc: Viswanath, Logeswari P (MCOU OSTL)
Subject: Re: Linux audit performance impact

Hello,

On Wednesday, January 28, 2015 02:57:58 PM Viswanath, Logeswari P wrote:
> We want to know audit performance impact on RHEL and Suse linux to 
> help us evaluate linux audit as data source for our host based IDS. 
> When we ran our own performance test with a test audispd plugin, we 
> found if a system can perform 200000 open/close system calls per 
> second without auditing, system can perform only 3000 open/close 
> system calls auditing is enabled for open/close system call which is a 
> HUGE impact on the system performance. It would be great if anyone can help us answering the following questions.
> 
> 
> 1)      Is this performance impact expected? If yes, what is the reason
> behind it and can we fix it?

I'll leave this for the kernel guys to answer. That said, I think more detailed information might be helpful.

If auditd is not started and events go to syslog, does the performance change? 
To do this audit=1 on boot line and auditctl -R /etc/rules.d/your.rules

Logeswari=>System can perform 15000 open/close system calls per second which is better than earlier results.

what rules do you have loaded?

Logeswari=> # auditctl -l
LIST_RULES: exit,always syscall=open,close
 
What do you get when audit is enabled and no rules loaded?

Logeswari=> Impact is there but not major.

If you have other syscall rules loaded that are not open and openat or close, does the performance change? I suspect that if you trigger a rule, you are thrown onto the slow path. Open is perhaps the most lengthy because of multiple auxiliary records and path resolution. But we need data to tell.

Logeswari=> Yes, there is an major impact. I enabled write system call and this rule is first in the set of rules along with open/close.

That said, I know that the kernel audit path changed a couple years ago so it might be worthwhile to test against an old kernel to see if the change has affected performance.

Logeswari=> We tested with kernel 2.6.32. Should we test with old/new kernel?

-Steve

> 2)      Have anyone done any benchmarking for performance impact? If yes,
> can you please share the numbers and also the steps/programs used the 
> run the same.
> 
> 3)      Help us validating the performance test we have done in our test
> setup using the steps mentioned along with the results attached.
> 
> Attached test program (loader.c) to invoke open and close system calls.
> Attached idskerndsp is the audispd plugin program.
> We used time command to determine how much time the system took to 
> complete
> 50000 open/close system calls without (results attached 
> Without-auditing) and with auditing enabled on the system 
> (With-auditing-NOLOG-audispd-plugin
> and With-auditing-RAW)
> 
> System details:
> 
> 1 CPU machine
> 
> OS Version
> RHEL 6.5
> 
> Kernel Version
> uname -r
> 2.6.32-431.el6.x86_64
> 
> Note: auditd was occupying 35% of CPU and was sleeping for most of the 
> time whereas kauditd was occupying 20% of the CPU.
> 
> Thanks & Regards,
> Logeswari.

^ permalink raw reply

* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-01-28 15:53 UTC (permalink / raw)
  To: Satish Chandra Kilaru; +Cc: linux-audit@redhat.com
In-Reply-To: <CAAnai+W5WWJ1n=s1ojODH92AeU81jR44fqNHLz+d2zyyHkjG=A@mail.gmail.com>


[-- Attachment #1.1: Type: text/plain, Size: 2287 bytes --]

Thanks for the quick reply Satish.

From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com]
Sent: Wednesday, January 28, 2015 8:49 PM
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: linux-audit@redhat.com<mailto:linux-audit@redhat.com>
Subject: Re: Linux audit performance impact

Write your own program to receive audit events directly without using auditd...
That should be faster ....
Auditd will log the events to disk causing more I/o than u need...

On Wednesday, January 28, 2015, Viswanath, Logeswari P (MCOU OSTL) <logeswari.pv@hp.com<mailto:logeswari.pv@hp.com>> wrote:
Hi Steve,

I am Logeswari working for HP.

We want to know audit performance impact on RHEL and Suse linux to help us evaluate linux audit as data source for our host based IDS.
When we ran our own performance test with a test audispd plugin, we found if a system can perform 200000 open/close system calls per second without auditing, system can perform only 3000 open/close system calls auditing is enabled for open/close system call which is a HUGE impact on the system performance. It would be great if anyone can help us answering the following questions.


1)      Is this performance impact expected? If yes, what is the reason behind it and can we fix it?

2)      Have anyone done any benchmarking for performance impact? If yes, can you please share the numbers and also the steps/programs used the run the same.

3)      Help us validating the performance test we have done in our test setup using the steps mentioned along with the results attached.

Attached test program (loader.c) to invoke open and close system calls.
Attached idskerndsp is the audispd plugin program.
We used time command to determine how much time the system took to complete 50000 open/close system calls without (results attached Without-auditing) and with auditing enabled on the system (With-auditing-NOLOG-audispd-plugin and With-auditing-RAW)

System details:

1 CPU machine

OS Version
RHEL 6.5

Kernel Version
uname –r
2.6.32-431.el6.x86_64

Note: auditd was occupying 35% of CPU and was sleeping for most of the time whereas kauditd was occupying 20% of the CPU.

Thanks & Regards,
Logeswari.




--
Please Donate to www.wikipedia.org<http://www.wikipedia.org>

[-- Attachment #1.2: Type: text/html, Size: 8416 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]



^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox