Re: New draft standards

[Steve Grubb <sgrubb@redhat.com>]

On Wed, 09 Dec 2015 12:43:37 +1100
Burn Alting <burn@swtf.dyndns.org> wrote:

> On Tue, 2015-12-08 at 19:28 -0500, Paul Moore wrote:
> > On Tuesday, December 08, 2015 03:25:22 PM Steve Grubb wrote:
> > > On Tuesday, December 08, 2015 02:58:18 PM Paul Moore wrote:
> > > > On Tue, Dec 8, 2015 at 2:22 PM, Steve Grubb <sgrubb@redhat.com>
> > > > wrote:
> > > > > Hello,
> > > > > 
> > > > > I would like to point out 2 new standards that have been
> > > > > posted to the linux audit web page. The first establishes the
> > > > > events around system start up and shutdown. This is important
> > > > > because it sets the session boundaries for when a system is
> > > > > up or down or crashed.
> > > > > 
> > > > > http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> > > > > 
> > > > > The second standard is more of a forward looking standard. It
> > > > > explains how
> > > > > the audit daemon and utilities will perform event enrichment
> > > > > before being
> > > > > stored long term in an aggregator. The target for
> > > > > implementation is the 2.5 release of the audit daemon.
> > > > > 
> > > > > http://people.redhat.com/sgrubb/audit/event-enrichment
> > > > > 
> > > > > Let me know if anyone has feedback on these standards,
> > > > > especially the second one.
> > > > 
> > > > Were these two specification documents created based on
> > > > published standards from an established standards body, e.g.
> > > > NIST, IETF, etc?
> > > 
> > > No. All of the standards published to date is documenting what
> > > exists and why. The needs are typically driven by common criteria
> > > and the need to detect certain kinds of events for intrusion
> > > detection or anomalous conditions.
> > 
> > Okay, let's not call these "standards" and just stick with
> > "specifications". The term standards has all sorts of connotations
> > associated with it, both good and bad, and I think we should be
> > clear when we start talking with other developers.  I think it
> > would also be *very* helpful if you could explain the motivation
> > behind these specs so we understand what problems you are trying to
> > solve and what requirements you are trying to meet; you talk about
> > this a bit in the conclusion, but more background would be nice.
> > 
> > Another nit-picky comment, in the future I would suggest sending
> > the specs inline in your mail; having to go to my browser to read
> > your document and then cut-and-paste it into my email to comment on
> > it means your request for feedback goes to the bottom of my todo
> > list.  Lower the bar for collaboration as much as possible, if you
> > inline the text all we have to do is hit "reply" to comment on the
> > specs.
> > 
> > Anyway, on to your docs ...
> > 
> > * https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> > 
> > > System Lifecycle Auditing
> > > =========================
> > > This document will go over the events that are associated with
> > > starting up a system and shutting it down. Knowing what events
> > > make up these actions allows an analytical application to know
> > > the boundaries of all sessions and actions a user may perform. It
> > > also allows identification of crashed systems or malfunctioning
> > > services. The following table lists the events that make up the
> > > system lifecycle in the audit trail:
> > > 
> > > AUDIT_SYSTEM_BOOT - System boot
> > > AUDIT_SYSTEM_RUNLEVEL - System runlevel change
> > > AUDIT_DAEMON_START - Audit daemon startup record
> > > AUDIT_DAEMON_ABORT - Audit daemon error stop record
> > > AUDIT_SERVICE_START - Service (daemon) start
> > > AUDIT_SERVICE_STOP - Service (daemon) stop
> > > AUDIT_SYSTEM_SHUTDOWN - System shutdown
> > > AUDIT_DAEMON_END - Audit daemon normal stop record
> > 
> > Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just
> > an AUDIT_DAEMON_STOP with a field to indicate if it was a normal
> > shutdown or a failure as outlined in the spec?  This would be more
> > consistent with the other daemons and the shutdown result field
> > could potentially be reused by the init systems for other daemons
> > (assuming the information was conveyed via return values or some
> > other mechanism).
> > 
> > > Lifecycle of the system
> > > =======================
> > > When the system is powered on, control is eventually turned over
> > > to an init daemon. This daemon is responsible for starting up all
> > > other system services and performing an order system shutdown
> > > when asked. This init daemon should send a AUDIT_SYSTEM_BOOT
> > > event after it has done its own initialization. Most init systems
> > > have different targets or modes of operation that the system is
> > > turned over for interactive sessions. Examples are multi-user
> > > console, multi-user graphical, etc. 
> > 
> > You mention it later, but it might be a good idea to mention in
> > this paragraph that the audit daemon should be started as early as
> > possible by the init daemon.
> > 
> > > Init will determine what runlevel the system is ultimately going
> > > to try to achieve. When gets there or it fails to get there, it
> > > shall issue an AUDIT_SYSTEM_RUNLEVEL event to denote which mode
> > > of operation it was going to be in. If an admin requests that the
> > > system switch to another runlevel, then it should issue another
> > > AUDIT_SYSTEM_RUNLEVEL event.
> > 
> > I think it would be good to have a discussion about runlevels that
> > don't follow the traditional integer numbering, e.g. string based
> > runlevels.
> > 
> > * https://people.redhat.com/sgrubb/audit/event-enrichment
> > 
> > > Audit Event Enrichment
> > > ======================
> > > 
> > > There are times when the audit events are stored in another
> > > machine and need to be searched at a later date. Some parts of
> > > the audit event are temporally limited in duration or unique to a
> > > system. This makes interpretting fields that are numbers into
> > > human readable fields hard or impossible without running a report
> > > at the time of the event or on the machine the event occurred on.
> > > 
> > > To address this issue, the audit daemon will get a new mode,
> > > enrich, where the audit trail will be amended as follows at the
> > > time an event is logged:
> > > 
> > > 1) Translations will be:
> > >   a) appended to the end of the event with the field's name in
> > > capital letters
> > 
> > Please no, let's leave field names case insensitive, perhaps an
> > agreed upon suffix, e.g. "-trans"?
> > 
> > >   b) encoded if user controlled data is used for enrichment
> > > 
> > > 2) The auparse library will:
> > >   a) preferentially use these fields whenever an interpretation
> > > is requested b) if none exist, look up the fields on the local
> > > machine if necessary
> > 
> > I think resolving these fields on the local machine is misleading,
> > and potentially dangerous; this is especially true with respect to
> > SELinux labels. If you can't resolve the field, simply display it
> > raw and let the operator determine how to handle it.
> > 
> 
> Steve,
> 
> Can you mock up some examples of an 'enriched' event showing how it is
> different from what we have now.

type=LOGIN msg=audit(1449782897.896:2496): pid=1768 uid=0
subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 old-auid=4294967295
auid=4325 old-ses=4294967295 ses=1 res=1 UID="root" OLD-AUID="unset"
AUID="sgrubb"

type=SYSCALL msg=audit(1449778741.412:4952): arch=c000003e syscall=40
success=no exit=-22 a0=3 a1=0 a2=0 a3=4000 items=0 ppid=7362 pid=7994
auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
tty=(none) ses=4294967295 comm="systemd-coredum"
exe="/usr/lib/systemd/systemd-coredump"
subj=system_u:system_r:init_t:s0 key="einval-retcode" ARCH=x86_64
SYSCALL=sendfile AUID="unset" UID="root" GID="root" EUID="root"
SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

-Steve

> Being one of those people who maintain a central repository of Linux
> audit, my main "ingest" concerns are to have data that is simple and,
> hence, quick to parse and hence normalize.
> 
> I think the risks associated with resolution of raw data can be
> mitigated by optionally maintaining the raw value as well when
> transmitting the event to a central repository.
> 
> Given the above is implemented, then I would recommend the
> modification of ausearch to optionally translate a complete raw
> enriched event into either a single json or xml record and optionally
> include raw and interpreted values. The decision to include both
> could be driven via pattern matched directive (eg *id|hostname). In
> reality, irrespective of whether the above is implemented or not, I
> would recommend (and will probably create the patch). 
> 
> To me the biggest benefit of Steve's proposal is the near real time
> resolution of some values. This is particularly useful for IP
> addresses (given one also notes somewhere, the name servers the
> system uses for resolution) given their potential reuse in short
> periods of time.