Near Term Audit Road Map

Near Term Audit Road Map

[Steve Grubb]

Hi,

With the proposals sent to the list, I wanted to talk about how this might 
play out code-wise. With regard to the current code base, I am working on a 
1.8 release. This would represent finishing the remote logging app and 
nothing more. The 1.8 series would become just an update series just like the 
1.0.x series did.

In parallel with finishing remote logging, I would release a 2.0 version. 
Patches applied to 1.8 would also be applied to 2.0. A 2.1 release would 
signify the completion of remote logging that branch. I would recommend this 
branch for all distributions pulling new code in. 

The 2.0 branch will also have a couple more changes. I want to split up the 
audit source code a little bit. I want to drop the system-config-audit code 
and let it become standalone package updated and distributed separately. 

I also want to drop all audispd-plugins in the 2.0 branch and have them 
released separately. They cause unnecessary build dependencies for the audit 
package.

During the work for a 2.2 release, I would also like to pull the audispd 
program inside auditd. In the past, I tried to keep auditd lean and single 
purpose, but with adding remote logging and kerberos support, we already have 
something that is hard to analyze. So, to improve performance and decrease 
system load, the audit daemon will also do event dispatching.

Would this proposal impact anyone in a Bad Way?

Thanks,
-Steve

Re: Near Term Audit Road Map

[LC Bruzenak]

On Fri, 2009-02-27 at 10:33 -0500, Steve Grubb wrote:
> 
> During the work for a 2.2 release, I would also like to pull the
> audispd program inside auditd. In the past, I tried to keep auditd
> lean and single purpose, but with adding remote logging and kerberos
> support, we already have something that is hard to analyze. So, to
> improve performance and decrease system load, the audit daemon will
> also do event dispatching.
> 
> Would this proposal impact anyone in a Bad Way?

Steve,

Couple questions:
- Right now the audispd remote/prelude plugins have connection-related
concerns. For example, with the remote plugin there are timeouts,
retries, etc. and parameters to tweak that. With the prelude plugin, it
must have been registered with the running prelude-manager correctly or
it dies. Do you see that pulling in the dispatching will cause more
auditd complexity or is that not a problem? I thought that (one reason)
audispd was separate was to allow it to deal with the vagaries of
endpoint and delivery issues while the auditd kept doing its thing.

- In theory, if everything is still doing roughly the same amount of
work, I'd think that moving the functionality would not necessarily
decrease the system load. I'm sure you have thoughts on how this would
be realized and at some point I'd be interested to hear those.

- What do you see as the initial target platform for the 2.0 series in
Fedora? In RH I assume it would be the next RH release thereafter?

Overall, though, I'd vote that nothing you propose would harm my
efforts, since I think I'd be stuck with the 1.8 series for a while
anyway.

Thx,
LCB.

-- 
LC (Lenny) Bruzenak
lenny@magitekltd.com

Re: Near Term Audit Road Map

[LC Bruzenak]

On Fri, 2009-02-27 at 10:13 -0600, LC Bruzenak wrote:
> On Fri, 2009-02-27 at 10:33 -0500, Steve Grubb wrote:
> > ...
> audispd was separate was to allow it to deal with the vagaries of
> endpoint and delivery issues while the auditd kept doing its thing.

Maybe the dispatchers themselves are still separate.

LCB.

-- 
LC (Lenny) Bruzenak
lenny@magitekltd.com

Re: Near Term Audit Road Map

[Steve Grubb]

On Friday 27 February 2009 11:13:44 am LC Bruzenak wrote:
> On Fri, 2009-02-27 at 10:33 -0500, Steve Grubb wrote:
> > During the work for a 2.2 release, I would also like to pull the
> > audispd program inside auditd. In the past, I tried to keep auditd
> > lean and single purpose, but with adding remote logging and kerberos
> > support, we already have something that is hard to analyze. So, to
> > improve performance and decrease system load, the audit daemon will
> > also do event dispatching.
>
> Couple questions:
> Do you see that pulling in the dispatching will cause more auditd complexity
> or is that not a problem?

The current chain is:

kernel->auditd->audispd->audisp-prelude

What it would become is

kernel->auditd->audisp-prelude


> I thought that (one reason)  audispd was separate was to allow it to deal
> with the vagaries of endpoint and delivery issues while the auditd kept
> doing its thing. 

Sort of. It was mostly to do the demultiplexing, but in reality, people windup 
getting queue overflows between auditd and audispd.


> - In theory, if everything is still doing roughly the same amount of
> work, I'd think that moving the functionality would not necessarily
> decrease the system load. 

The problem is auditd gets backed up trying to send to audispd. Audispd rarely 
gets backed up and when it does you can increase the queue. But you can't do 
this between the audit daemon and dispatcher. Then auditd backs up the kernel 
queue.


> - What do you see as the initial target platform for the 2.0 series in
> Fedora? 

Either 11 or 12 depending on when I can get it done and pushed through.


> In RH I assume it would be the next RH release thereafter? 

Yes, it would aimed at RHEL6, with the 1.8 series becoming RHEL5 only.

-Steve

audisp-remote and audisp-prelude question

[LC Bruzenak]

I thought that we have :
    
(from another machine)
     audisp-remote 
          |
          v          (to collector)
kernel->auditd->audispd->audisp-prelude

and that I could pick off the prelude-bound events on the aggregated
data, but I don't get the events into the prelude DB.

For example, I see the client logins in the collector's log, so the
aggregation appears to be working.
Local logins on the collector machine do get sent to prelude, so the
audisp-prelude plugin is working.

However, logins on the remote machine which are sent to the collector
log do not make it into the prelude DB (at least prewikka doesn't show
them). I have no prewikka filters and I have the prewikka viewer set to
"1 day".

Any ideas? Using 1.7.12 audit rpms.

Here is a sample of "ausearch -ts today -i -m USER_LOGIN" on the
collector:
...
node=v157 type=USER_LOGIN msg=audit(03/24/2009 10:44:27.533:548759) :
user pid=11353 uid=root auid=root ses=328
subj=system_u:system_r:sshd_t:s0-s15:c0.c1023 msg='uid=root
exe=/usr/sbin/sshd (hostname=homeserver, addr=192.168.31.40,
terminal=/dev/pts/0 res=success)' 
----
node=audit type=USER_LOGIN msg=audit(03/24/2009 11:11:37.882:1412) :
user pid=3103 uid=root auid=root ses=54
subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='uid=root
exe=/usr/sbin/sshd (hostname=192.168.31.40, addr=192.168.31.40,
terminal=/dev/pts/3 res=success)' 

On the prewikka screen I only see the second event.

Thx,
LCB.

-- 
LC (Lenny) Bruzenak
lenny@magitekltd.com

Re: audisp-remote and audisp-prelude question

[Steve Grubb]

On Tuesday 24 March 2009 12:29:48 LC Bruzenak wrote:
> On the prewikka screen I only see the second event.

prelude is its own protocol and picks out certain data from its config files and 
puts in its packets. The intended use is each machine sends its prelude alerts 
to a common prelude manager. Each audit event is sent to its aggregator. The 
two systems diverge at audispd.

kernel->auditd->audispd-+->audisp-prelude->prelude-manager
                                               +->audisp-remote->auditd

-Steve

Re: audisp-remote and audisp-prelude question

[Sebastien Tricaud]

On Tue, 2009-03-24 at 11:29 -0500, LC Bruzenak wrote:
> 
> However, logins on the remote machine which are sent to the collector
> log do not make it into the prelude DB (at least prewikka doesn't show
> them). I have no prewikka filters and I have the prewikka viewer set to
> "1 day".

Hi, 

can you see events coming to prelude-manager when running:
prelude-manager --debug

?

Thanks,
Sebastien.

Re: audisp-remote and audisp-prelude question

[LC Bruzenak]

On Tue, 2009-03-24 at 17:55 +0100, Sebastien Tricaud wrote:
> On Tue, 2009-03-24 at 11:29 -0500, LC Bruzenak wrote:
> > 
> > However, logins on the remote machine which are sent to the collector
> > log do not make it into the prelude DB (at least prewikka doesn't show
> > them). I have no prewikka filters and I have the prewikka viewer set to
> > "1 day".
> 
> Hi, 
> 
> can you see events coming to prelude-manager when running:
> prelude-manager --debug
> 
> ?
> 
> Thanks,
> Sebastien.
> 
Sebastien,

Thanks for the reply. I do not see it getting into the prelude-manager.
I also did a local login so I could see one which worked and the alert
came through in that case.

LCB.

-- 
LC (Lenny) Bruzenak
lenny@magitekltd.com

Re: audisp-remote and audisp-prelude question

[Steve Grubb]

On Tuesday 24 March 2009 12:29:48 LC Bruzenak wrote:
> On the prewikka screen I only see the second event.

prelude is its own protocol and picks out certain data from its config files and 
puts in its packets. The intended use is each machine sends its prelude alerts 
to a common prelude manager. Each audit event is sent to its aggregator. The 
two systems diverge at audispd.

kernel->auditd->audispd-+->audisp-prelude->prelude-manager
                                               +->audisp-remote->auditd

-Steve

Re: audisp-remote and audisp-prelude question

[LC Bruzenak]

On Tue, 2009-03-24 at 13:06 -0400, Steve Grubb wrote: 
> On Tuesday 24 March 2009 12:29:48 LC Bruzenak wrote:
> > On the prewikka screen I only see the second event.
> 
> prelude is its own protocol and picks out certain data from its config files and 
> puts in its packets. The intended use is each machine sends its prelude alerts 

not MY intended use...
:) 

> to a common prelude manager. Each audit event is sent to its aggregator. The 
> two systems diverge at audispd.
> 
> kernel->auditd->audispd-+->audisp-prelude->prelude-manager
>                                                +->audisp-remote->auditd
> 
> -Steve

Steve; thanks.

I may not follow. Does the above preclude what I'm asking?
Asked another way, what stops the aggregated audit events from creating
a prelude event? 

Thx,
LCB.

-- 
LC (Lenny) Bruzenak
lenny@magitekltd.com

Re: audisp-remote and audisp-prelude question

[Steve Grubb]

On Tuesday 24 March 2009 14:01:39 LC Bruzenak wrote:
> Asked another way, what stops the aggregated audit events from creating
> a prelude event?

Prelude grabs things from its own config files to fill in certain fields. This 
means that if run from an aggregator, it will use the same values for all 
events. This affects the host names that show up in heartbeats and other 
events.

The two are meant to be separate but complimentary systems.

-Steve

Re: Near Term Audit Road Map

[Matthew Booth]

Steve Grubb wrote:
> Hi,
> 
> With the proposals sent to the list, I wanted to talk about how this might 
> play out code-wise. With regard to the current code base, I am working on a 
> 1.8 release. This would represent finishing the remote logging app and 
> nothing more. The 1.8 series would become just an update series just like the 
> 1.0.x series did.
> 
> In parallel with finishing remote logging, I would release a 2.0 version. 
> Patches applied to 1.8 would also be applied to 2.0. A 2.1 release would 
> signify the completion of remote logging that branch. I would recommend this 
> branch for all distributions pulling new code in. 
> 
> The 2.0 branch will also have a couple more changes. I want to split up the 
> audit source code a little bit. I want to drop the system-config-audit code 
> and let it become standalone package updated and distributed separately. 
> 
> I also want to drop all audispd-plugins in the 2.0 branch and have them 
> released separately. They cause unnecessary build dependencies for the audit 
> package.
> 
> During the work for a 2.2 release, I would also like to pull the audispd 
> program inside auditd. In the past, I tried to keep auditd lean and single 
> purpose, but with adding remote logging and kerberos support, we already have 
> something that is hard to analyze. So, to improve performance and decrease 
> system load, the audit daemon will also do event dispatching.
> 
> Would this proposal impact anyone in a Bad Way?

On the contrary. My austream tool was born because:

* Ensuring a dispatcher doesn't generate audit events is fragile
* The additional task switching and memory copying becomes onerous under
load

Additionally, auditd is clearly geared up for writing to disk: certainly
in RHEL 4, switching off all disk related activity is a whole lot of
typing to tell it not to do anything :)

Solaris's BSM implements custom behaviour with loadable modules. If our
auditd did that, hopefully I could deprecate austream. The dispatcher
architecture doesn't lend itself to sustained high volume.

Matt
-- 
Matthew Booth, RHCA, RHCSS
Red Hat, Global Professional Services

M:       +44 (0)7977 267231
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490