Introducing audit-explorer

Introducing audit-explorer

[Steve Grubb]

Hello,

I have released the audit-explorer shiny app that I have been demo'ing this 
spring:

https://github.com/stevegrubb/audit-explorer

I have been talking about some of the concepts in it from my blog. But this is 
much more interesting because its fully interactive.

Enjoy...

-Steve

Re: Introducing audit-explorer

[Vincas Dargis]

2017.06.19 23:55, Steve Grubb rašė:
> I have released the audit-explorer shiny app that I have been demo'ing this
> spring:
> 
> https://github.com/stevegrubb/audit-explorer

Very nice, thanks for sharing!

Now if we are talking about tools, are there somewhere (maybe in your shelf? :-) ) a conveniently configurable tool for 
generating daily plaintext (or HTML) reports, that could be sent via email from machine your are interested in?

For example, I had to build custom bash script at work, that uses ausearch, aureport and even grep (for AppArmor events 
since it has issues with it's audit messages) to aggregate most interesting audit records (for example, with -k 
apache_user_executed_binaries, non-root executed something as root, failed logins and such) and sends it via email every 
day.

Though it is not that complicated to fill your .sh with bunch of ausearch/aureport/grep calls, it feels like I'm 
reimplementing something...






--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit

Re: Introducing audit-explorer

[Steve Grubb]

Hello,

On Tuesday, June 20, 2017 12:28:16 PM EDT Vincas Dargis wrote:
> 2017.06.19 23:55, Steve Grubb rašė:
> > I have released the audit-explorer shiny app that I have been demo'ing
> > this spring:
> > 
> > https://github.com/stevegrubb/audit-explorer
> 
> Very nice, thanks for sharing!

Thanks.

> Now if we are talking about tools, are there somewhere (maybe in your shelf?
> :-) ) a conveniently configurable tool for generating daily plaintext (or
> HTML) reports, that could be sent via email from machine your are
> interested in?

I am working my way around to this. For one, its hard to imagine all the 
reports that might be of interest without overwhelming the report. If we could 
define a small list of what is expected or useful, then we can work towards 
this.

The aureport tool is good at doing summary reports. It's about 12 years old. 
There are newer technologies that might be better to use. For example, how 
would people like to see reporting in a Jupyter notebook? If you don't know 
what a Jupyter notebook is, then take a few minutes and google it. Or take a 
look here:

https://github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-Notebooks


> For example, I had to build custom bash script at work, that uses ausearch,
> aureport and even grep (for AppArmor events since it has issues with it's
> audit messages) to aggregate most interesting audit records (for example,
> with -k apache_user_executed_binaries, non-root executed something as root,
> failed logins and such) and sends it via email every day.
> 
> Though it is not that complicated to fill your .sh with bunch of
> ausearch/aureport/grep calls, it feels like I'm reimplementing something...

Yes. Another way forward might be to use the CSV extraction options in 
ausearch and send that into a SQL database. Then any SQL reporting tool out 
there can be used.

I'm going to be starting the 2.8 development cycle in the near future. The 
goal for it is to get the remote logging better and continue improving the 
auparse_normalizer API. This will directly lead to more and better reporting 
options.

Things that may be possible:
push events in realtime to datalake such as kafka
push events in realtime to SQL database
demux events out of rsyslog and push into audit aggregation server
enhance collectors for various SIEMs

Of course to do any of this means I need participation and collaboration from 
the community. I can't guess what people need to hook up the audit trail to 
reporting tools.

-Steve


--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit

Re: Introducing audit-explorer

[Vincas Dargis]

2017.06.20 20:22, Steve Grubb wrote:
> I am working my way around to this. For one, its hard to imagine all the
> reports that might be of interest without overwhelming the report. If we could
> define a small list of what is expected or useful, then we can work towards
> this.

Well, we have `aureport --summary`, and if any of these summary items has non-zero
event count, it would be useful to know the details.

If there is: "Number of users: 5", automatically output of `aureport --summary -u`
might be useful.

For flexibility, if there are:

"Number of executables: 14"

And if one has custom audit rule with it's filter key set, with (pseudo) configuration as:

alias="Commands executd as root:",
executables,
key=root,
type=EXECVE,
<something more fore aggregation?>

Administrator could receive something like this (it's snippet from my homebrew script):

--- Commands executed as root: ---

       1 a0="/usr/sbin/exim4" a1="-Mc" a2="1dPibe-000889-Jy"
       1 a0="/usr/sbin/exim4" a1="-Mc" a2="1dPn4O-0006hm-2W"
       1 a0="/usr/sbin/exim4" a1="-Mc" a2="1dPnE4-0002ux-4g"
       1 a0="/usr/sbin/exim4" a1="-Mc" a2="1dPyzk-00032N-PR"
       1 a0="/usr/sbin/exim4" a1="-Mc" a2="1dPz9Q-0005QM-4E"
       3 a0="sudo" a1="-i"
       4 a0="send-mail" a1="-i" a2="--" a3="root@localhost"
      48 a0="/usr/sbin/exim4" a1="-q"
     288 a0="/usr/bin/sudo" a1="-u" a2="root" a3="/usr/sbin/exim4" a4="-bpc"
     288 a0="/usr/sbin/exim4" a1="-bpu"
    1440 a0="sudo" a1="/usr/local/nagios/plugins/check_apparmor_unconfined_with_profile"



> The aureport tool is good at doing summary reports. It's about 12 years old.
> There are newer technologies that might be better to use. For example, how
> would people like to see reporting in a Jupyter notebook? If you don't know
> what a Jupyter notebook is, then take a few minutes and google it. Or take a
> look here:
> 
> https://github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-Notebooks

Can't comment it yet, needs free few minutes :-) .

> Yes. Another way forward might be to use the CSV extraction options in
> ausearch and send that into a SQL database. Then any SQL reporting tool out
> there can be used.

Yeah, good idea with CSV. I guess SQLite could be rather enough for such reporting.

> I'm going to be starting the 2.8 development cycle in the near future. The
> goal for it is to get the remote logging better and continue improving the
> auparse_normalizer API. This will directly lead to more and better reporting
> options.
> 
> Things that may be possible:
> push events in realtime to datalake such as kafka
> push events in realtime to SQL database
> demux events out of rsyslog and push into audit aggregation server
> enhance collectors for various SIEMs

Yes, sending to Graylog using GELF format would be useful too, for example.

> Of course to do any of this means I need participation and collaboration from
> the community. I can't guess what people need to hook up the audit trail to
> reporting tools.

Sure thing. It would be nice to have feedback about this topic from administrators
handling big shops, my comments are rather amateurish.