[audit] Upstream solution for auditing file system objects

[audit] Upstream solution for auditing file system objects

[Timothy Chavez]

Greetings, 
 
I'm writing this e-mail to facilitate some discussion on an audit
feature for inclusion to the mainline kernel's audit subsysystem. 
I've written out a "brief" description of the problem with some of the
associated problems it introduces and some of the outlines of
potential solutions below.  For the most part, the idea stays the
same, and the mechanism we use to implement it varies.

This message is fairly long.  My apologies.
 
----------- 
 
The Problem: 
~~~~~~~~~~~~ 
 
Over the last two months, I've been given the daunting task of
implementing a feature by which an administrator can specify from user
space, a list of file system objects (namely regular files and
directories) that he/she wishes to audit.  The administrator would
presumably like to,
 
1.  Specify a file, "/etc/shadow" 
 
2.  Filter based on some criteria, "I only care if /etc/shadow is
open()'d by uid=X" (Filtering could be more granular then this or
less, that's arbitrary at this point)
 
3.  Be reassured that one may not bypass the audit subsystem nor lose
an audit record.
 
Assuming we can do the filtering aspect in user space via auditd (or
whatever), we've introduced some kernel work.  For instance, when we
say we want to audit a file or directory, what we're really saying is,
 
"I'm interested in knowing when X syscall accesses/alters Y
file/directory, so send me a record of this activity."
 
Some notable design problems to introduce now are with the "identity"
of Y file/directory with respect to the kernel.  In fact, we only
really care about paths of which file/directory Y completes.  I.e: If
we want to audit /etc/shadow, we might not care if /etc/shadow is
moved to /tmp/shadow.  This means that any access/alteration of
/tmp/shadow would go unnoticed (unless otherwise specified).  However,
if /tmp/shadow were a hardlink to /etc/shadow, we would then still
care, because we are still effectively /etc/shadow... if that makes
sense :-)

These are things that our design must consider in some way. 
 
1.  Identifying a file, such as /etc/shadow, solely by a pathname,
enables someone to create a hardlink to /etc/shadow from a different
directory and by-pass the audit subsystem.  Also, we can't assume we
have this user space definition of a file/directory in the kernel.
 
2.  Identifying a file, such as /etc/shadow, solely by its inode,
enables a by-pass of the audit subsystem, when say, /etc/shadow gets
copied to /etc/shadow+ and moved back.  Because the inode has changed,
we're no longer the "same" file system object.  Similarly, if we
delete an audited file, /tmp/ultra_secret (in /tmp? sounds like my old
university), and recreate it, we'd like /tmp/ultra_secret to be
audited.  This falls back on the "we don't want audit records to be
lost" requirement.
 
3.  How do we treat mounts and --bind mounts?  I won't go into this in
this e-mail.  There's several design decisions to made with regards to
this issue.
 
4.  How can we obtain the full path of the file system object without
reconstructing it in the kernel?  Xattrs? Integer cookies/hashes that
key a table in user space?
 
5.  Whats the best approach at capturing information about the syscall
accessing / altering the file/directory?  We might lose records if we
attempt to capture this information from VFS hooks as we could
short-circuit before we ever reached that point in the function.  We'd
then need a way of related the audit of a system call that in place
now with the file/directory its accessing. (This particiular issue is
not talked about in this e-mail in any detail, really)

The Reason: 
~~~~~~~~~~~ 
 
The current audit subsystem does not contain this capability.  This
capability is essential for meeting Controlled Access Protection
Profile (CAPP) Evaluation Assurance Level (EAL) 4+ requirements.
 
The Solutions: 
~~~~~~~~~~~~~~ 
 
(These are brief notes.  More implementation detail can be provided it
asked for.  If someone is interested in discussing an actual solution,
we can get into that - I'd like to get into that.  I'm just trying to
give you a sense of some of the things that have been thought about).
 
1.  Inotify. Inotify implements a good portion of what we'd like to do
and looks like its well on its way into the mainline kernel.  What
would need to be added to inotify:
 
A.  A way in which it could, if desired, auto-watch the file/directory
associated with an IN_CREATE_FILE/SUBDIR event.  This would also
require we have some way to communicate , "We want this
<file/directory> to be auto-watched" -- There might be an issue here
because we're now giving meaning to the file/directory name associated
with the inode.  We're now saying, "If the inode changes, but the name
associated with it is the same, watch the new inode automatically."
 
B.  Inotify would have to be able to notify the audit subsystem of
events via a direct call internally, when the event is triggered. 
You'd have some device with an "audit" flag on it that says, "Hey I'm
interested in receiving events on behalf of audit, send them to me,
and I'll pass them directly on to the audit subsystem."  This would be
fairly easy to do, but would undermine the inotify event queuing
process because we could not take any chances of losing event records
in a queue that's overflowed.
 
2.  Linux Security Module.  This is perhaps the most daunting
approach. This solution would allow us to use the i_security field of
the inode to do our "book keeping."  We'd also have at our disposal
the the security hooks already in-place in the VFS.  This would allow
us to intercept the relevant syscalls via function pointers, to make
decisions on what we want to do.  Using the i_security field to hold
our table of auditable files/directories on parent directory inodes,
would enable us to ask the question, "Does this file/directory that's
just been created beneath me need to be marked as "auditable?" from
with in, say, security_inode_post_create().
 
The advantages to a module is that we can make this audit option
more... optional.  We're not introducing any new hooks and we're using
the LSM's inode i_security field, too.  This keeps the inode
structure's size flat and minimizes our impact in the VFS. 
Disadvantages are that for it to play nicely with other LSMs like
SELinux, we'd need the LSM stacker or else we'd need to disable
SELinux.  We'd also be forcing the question, "Is this an appropriate
use of the LSM framework?"  Although this feature is definitely
security-relevant and related, is what it does an appropriate context
for use of the LSM framework?  Some would say that the LSM framework
is for Access Control only, others would disagree.
 
3.  SELinux.  This feature is not currently in SELinux (as we've been
told), but we've had interest by the NSA in implementing this feature
for SELinux.  We'd have a similar solution to the LSM, except we'd
encapsulate ourselves within SELinux, and use its framework to drive
the solution.  This might prove less resilient, politically, because
we'd be under its umbrella.  I'm not clear on the exact implementation
of this in SELinux -- It sounds like it'd be a patch to the security
server and hooks, the creation of an audit label, and the invention of
some additional structure for "book keeping" purposes.  Also, there is
a general problem with using SELinux specifically with figuring out
how to certify with it enabled.  There's a lot of complications
(mostly unknowns) -- Complications we'd willingly work on, if need be.
 Obviously design decisions would have to be factor in as well.
 
4.  Audit subsystem.  To me, this is the best solution (maybe even
worst solution? we'll see what you have to say), but probably the
nastiest one.  Let the audit subsystem hook VFS in all the necessary
places and give audit an inode field to do the "book keeping."  The
advantages here is that we're encapsulating ourselves as much as
possible in the audit subsystem and not depending on any other
subsystem except for the one constant in all our solutions, VFS.  If
we were given the "go ahead" or some fuzzy feeling that this would be
the best approach/attempt, we could implement this feature in the
audit subsystem.  The disadvantages are fairly clear.  If this feature
were configured Y, the inode structure would grow in size and if this
were an inode field, we'd be keeping a list of "names," on it.  We'd
also be introducing yet another set of hooks to VFS which have a very
specific use.  And, on the kernel-side, we'd essentially be
duplicating some of inotify's behavior.
 
However, the most obvious gain here, is that the subsystem is
centralized and its intentions are clearer.
 
5.  Kprobes.  I know nothing about them.  These have not been
considered, but were mentioned to me just recently by two different
people.  I'm not sure if this approach could be useful or not.  Any
help here?
 
------- 
 
Your feedback is essential to figuring out what the best way to
approach this is.  Thanks for reading :-)
 
-- 
- Timothy R. Chavez

Re: [audit] Upstream solution for auditing file system objects

[Chris Wright]

* Timothy Chavez (chavezt@gmail.com) wrote:
> Over the last two months, I've been given the daunting task of
> implementing a feature by which an administrator can specify from user
> space, a list of file system objects (namely regular files and
> directories) that he/she wishes to audit.  The administrator would
> presumably like to,
>  
> 1.  Specify a file, "/etc/shadow" 
>  
> 2.  Filter based on some criteria, "I only care if /etc/shadow is
> open()'d by uid=X" (Filtering could be more granular then this or
> less, that's arbitrary at this point)
>  
> 3.  Be reassured that one may not bypass the audit subsystem nor lose
> an audit record.
>  
> Assuming we can do the filtering aspect in user space via auditd (or
> whatever), we've introduced some kernel work.  For instance, when we
> say we want to audit a file or directory, what we're really saying is,
>  
> "I'm interested in knowing when X syscall accesses/alters Y
> file/directory, so send me a record of this activity."
>  
> Some notable design problems to introduce now are with the "identity"
> of Y file/directory with respect to the kernel.  In fact, we only
> really care about paths of which file/directory Y completes.  I.e: If
> we want to audit /etc/shadow, we might not care if /etc/shadow is
> moved to /tmp/shadow.  This means that any access/alteration of

Of course you would.  A mv to /tmp/shadow includes an unlink, which
should be an auditable event.

> /tmp/shadow would go unnoticed (unless otherwise specified).  However,
> if /tmp/shadow were a hardlink to /etc/shadow, we would then still
> care, because we are still effectively /etc/shadow... if that makes
> sense :-)

Yes, at this point you care about the inode not the pathname (the latter
is just a handle to the former).

> These are things that our design must consider in some way. 
>  
> 1.  Identifying a file, such as /etc/shadow, solely by a pathname,
> enables someone to create a hardlink to /etc/shadow from a different
> directory and by-pass the audit subsystem.  Also, we can't assume we
> have this user space definition of a file/directory in the kernel.

What does that mean, "user space definition of a file/dir in kernel"?

> 2.  Identifying a file, such as /etc/shadow, solely by its inode,
> enables a by-pass of the audit subsystem, when say, /etc/shadow gets
> copied to /etc/shadow+ and moved back.  Because the inode has changed,
> we're no longer the "same" file system object.  Similarly, if we

OK, depends on the mechanism (O_TRUNC and write or unlink and rename).

> delete an audited file, /tmp/ultra_secret (in /tmp? sounds like my old
> university), and recreate it, we'd like /tmp/ultra_secret to be
> audited.  This falls back on the "we don't want audit records to be
> lost" requirement.

So the pathname is relative to the parent dir, which needs to know the
about these special names.

> 3.  How do we treat mounts and --bind mounts?  I won't go into this in
> this e-mail.  There's several design decisions to made with regards to
> this issue.

Well, it's an important one, can't just punt on it.

> 4.  How can we obtain the full path of the file system object without
> reconstructing it in the kernel?  Xattrs? Integer cookies/hashes that
> key a table in user space?

There's no such thing.  How do you store a single path in an xattr (which
is on an inode) for an inode that has multiple hardlinks, or multiple
vfsmounts (since you can't punt on --bind ;-).  The _best_ you could do
is go by the path that the user used (audit captures that already IIRC).

> 5.  Whats the best approach at capturing information about the syscall
> accessing / altering the file/directory?  We might lose records if we
> attempt to capture this information from VFS hooks as we could
> short-circuit before we ever reached that point in the function.  We'd
> then need a way of related the audit of a system call that in place
> now with the file/directory its accessing. (This particiular issue is
> not talked about in this e-mail in any detail, really)

I think you'd need to do more filtering in userspace, and generate a lot
of extra audit records.  At each point (audit, and lsm hook) there's not
enough context.  I can't recall when the record is created, if it's at
the syscall exit point, then the hook can tell you if you actually cared
about the transaction.

> The Reason: 
> ~~~~~~~~~~~ 
>  
> The current audit subsystem does not contain this capability.  This
> capability is essential for meeting Controlled Access Protection
> Profile (CAPP) Evaluation Assurance Level (EAL) 4+ requirements.
>  
> The Solutions: 
> ~~~~~~~~~~~~~~ 
>  
> (These are brief notes.  More implementation detail can be provided it
> asked for.  If someone is interested in discussing an actual solution,
> we can get into that - I'd like to get into that.  I'm just trying to
> give you a sense of some of the things that have been thought about).
>  
> 1.  Inotify. Inotify implements a good portion of what we'd like to do
> and looks like its well on its way into the mainline kernel.  What
> would need to be added to inotify:
>  
> A.  A way in which it could, if desired, auto-watch the file/directory
> associated with an IN_CREATE_FILE/SUBDIR event.  This would also
> require we have some way to communicate , "We want this
> <file/directory> to be auto-watched" -- There might be an issue here
> because we're now giving meaning to the file/directory name associated
> with the inode.  We're now saying, "If the inode changes, but the name
> associated with it is the same, watch the new inode automatically."

Not exactly, you're saying let me know if the name in this dir goes away
or gets created.  If it's created then you register for all it's access
activity. </handwavy>

> B.  Inotify would have to be able to notify the audit subsystem of
> events via a direct call internally, when the event is triggered. 
> You'd have some device with an "audit" flag on it that says, "Hey I'm
> interested in receiving events on behalf of audit, send them to me,
> and I'll pass them directly on to the audit subsystem."  This would be
> fairly easy to do, but would undermine the inotify event queuing
> process because we could not take any chances of losing event records
> in a queue that's overflowed.

I wonder if you could register with inotify and it could callback to
you for the events that you register for?  Then you're responsible
for dealing with audit subsystem, not inotify.  Classic consumer deal,
can you add yourself to being an event consumer?

> 2.  Linux Security Module.  This is perhaps the most daunting
> approach. This solution would allow us to use the i_security field of
> the inode to do our "book keeping."  We'd also have at our disposal
> the the security hooks already in-place in the VFS.  This would allow
> us to intercept the relevant syscalls via function pointers, to make
> decisions on what we want to do.  Using the i_security field to hold
> our table of auditable files/directories on parent directory inodes,
> would enable us to ask the question, "Does this file/directory that's
> just been created beneath me need to be marked as "auditable?" from
> with in, say, security_inode_post_create().
>  
> The advantages to a module is that we can make this audit option
> more... optional.  We're not introducing any new hooks and we're using
> the LSM's inode i_security field, too.  This keeps the inode
> structure's size flat and minimizes our impact in the VFS. 
> Disadvantages are that for it to play nicely with other LSMs like
> SELinux, we'd need the LSM stacker or else we'd need to disable
> SELinux.  We'd also be forcing the question, "Is this an appropriate
> use of the LSM framework?"  Although this feature is definitely
> security-relevant and related, is what it does an appropriate context
> for use of the LSM framework?  Some would say that the LSM framework
> is for Access Control only, others would disagree.

It's on the edge.  LSM is designed for access control, but does give the
hooking needed to do this security relevant operation.  We specifically
did not include audit functionality into LSM because of the audit needs
(syscall centric, early exit points are missed, etc.).

> 3.  SELinux.  This feature is not currently in SELinux (as we've been
> told), but we've had interest by the NSA in implementing this feature
> for SELinux.  We'd have a similar solution to the LSM, except we'd
> encapsulate ourselves within SELinux, and use its framework to drive
> the solution.  This might prove less resilient, politically, because
> we'd be under its umbrella.  I'm not clear on the exact implementation
> of this in SELinux -- It sounds like it'd be a patch to the security
> server and hooks, the creation of an audit label, and the invention of
> some additional structure for "book keeping" purposes.  Also, there is
> a general problem with using SELinux specifically with figuring out
> how to certify with it enabled.  There's a lot of complications
> (mostly unknowns) -- Complications we'd willingly work on, if need be.
>  Obviously design decisions would have to be factor in as well.
>  
> 4.  Audit subsystem.  To me, this is the best solution (maybe even
> worst solution? we'll see what you have to say), but probably the
> nastiest one.  Let the audit subsystem hook VFS in all the necessary
> places and give audit an inode field to do the "book keeping."  The
> advantages here is that we're encapsulating ourselves as much as
> possible in the audit subsystem and not depending on any other
> subsystem except for the one constant in all our solutions, VFS.  If
> we were given the "go ahead" or some fuzzy feeling that this would be
> the best approach/attempt, we could implement this feature in the
> audit subsystem.  The disadvantages are fairly clear.  If this feature
> were configured Y, the inode structure would grow in size and if this
> were an inode field, we'd be keeping a list of "names," on it.  We'd
> also be introducing yet another set of hooks to VFS which have a very
> specific use.  And, on the kernel-side, we'd essentially be
> duplicating some of inotify's behavior.

It's sounding like allowing the audit subsystem to subscribe to inotify
events is the best alternative at first glance.

> However, the most obvious gain here, is that the subsystem is
> centralized and its intentions are clearer.
>  
> 5.  Kprobes.  I know nothing about them.  These have not been
> considered, but were mentioned to me just recently by two different
> people.  I'm not sure if this approach could be useful or not.  Any
> help here?

Well, these just give you primarily ad-hoc hooking.

thanks,
-chris
-- 
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net

Re: [audit] Upstream solution for auditing file system objects

[Timothy Chavez]

Chris,

Thanks for the response.  Reply embedded...

(Also I removed serue from the CC, I guess I copied the wrong address for Serge)


On Thu, 9 Dec 2004 17:46:10 -0800, Chris Wright <chrisw@osdl.org> wrote:
> * Timothy Chavez (chavezt@gmail.com) wrote:
> 
> 
> > Over the last two months, I've been given the daunting task of
> > implementing a feature by which an administrator can specify from user
> > space, a list of file system objects (namely regular files and
> > directories) that he/she wishes to audit.  The administrator would
> > presumably like to,
> >
> > 1.  Specify a file, "/etc/shadow"
> >
> > 2.  Filter based on some criteria, "I only care if /etc/shadow is
> > open()'d by uid=X" (Filtering could be more granular then this or
> > less, that's arbitrary at this point)
> >
> > 3.  Be reassured that one may not bypass the audit subsystem nor lose
> > an audit record.
> >
> > Assuming we can do the filtering aspect in user space via auditd (or
> > whatever), we've introduced some kernel work.  For instance, when we
> > say we want to audit a file or directory, what we're really saying is,
> >
> > "I'm interested in knowing when X syscall accesses/alters Y
> > file/directory, so send me a record of this activity."
> >
> > Some notable design problems to introduce now are with the "identity"
> > of Y file/directory with respect to the kernel.  In fact, we only
> > really care about paths of which file/directory Y completes.  I.e: If
> > we want to audit /etc/shadow, we might not care if /etc/shadow is
> > moved to /tmp/shadow.  This means that any access/alteration of
> 
> Of course you would.  A mv to /tmp/shadow includes an unlink, which
> should be an auditable event.

Yes, you're absolutely right.  Anyway, the point I was trying to drive
home is that once moved to /tmp/shadow from an auditable path,
/etc/shadow, audit records will no longer be generated.  If we do mv
/tmp/shadow to /etc/shadow then audit will resume on /etc/shadow...
onward..

> > /tmp/shadow would go unnoticed (unless otherwise specified).  However,
> > if /tmp/shadow were a hardlink to /etc/shadow, we would then still
> > care, because we are still effectively /etc/shadow... if that makes
> > sense :-)
> 
> Yes, at this point you care about the inode not the pathname (the latter
> is just a handle to the former).
> 
> > These are things that our design must consider in some way.
> >
> > 1.  Identifying a file, such as /etc/shadow, solely by a pathname,
> > enables someone to create a hardlink to /etc/shadow from a different
> > directory and by-pass the audit subsystem.  Also, we can't assume we
> > have this user space definition of a file/directory in the kernel.
> 
> What does that mean, "user space definition of a file/dir in kernel"?

Oh, I suppose description would have been a better word then
definition.  I was trying to say that we can't deal with path names. 
How we handle a file/directory in the kernel will be with dentry's and
inodes.  Sorry about that.

> > 2.  Identifying a file, such as /etc/shadow, solely by its inode,
> > enables a by-pass of the audit subsystem, when say, /etc/shadow gets
> > copied to /etc/shadow+ and moved back.  Because the inode has changed,
> > we're no longer the "same" file system object.  Similarly, if we
> 
> OK, depends on the mechanism (O_TRUNC and write or unlink and rename).

Sure.  But the fact remains that if we say, "audit /etc/shadow" that
regardless of the underlying inode, relevant access / alterations to
/etc/shadow are audited.
 
> > delete an audited file, /tmp/ultra_secret (in /tmp? sounds like my old
> > university), and recreate it, we'd like /tmp/ultra_secret to be
> > audited.  This falls back on the "we don't want audit records to be
> > lost" requirement.
> 
> So the pathname is relative to the parent dir, which needs to know the
> about these special names.

Correct.  This is sufficient.

> > 3.  How do we treat mounts and --bind mounts?  I won't go into this in
> > this e-mail.  There's several design decisions to made with regards to
> > this issue.
> 
> Well, it's an important one, can't just punt on it.

I have carpel tunnel and ADD?? :-)  Can we treat the filesystem as
flat and just restart the auditd to recaliberate itself?  Is that
reasonable?  I'm not sure.  If designed right, I think we could slip
by --bind mounts unscathed.  If we mount --bind /etc/ /tmp/foo and we
do touch /tmp/foo/shadow, shadow will consult the table associated
with foo's inode, the same inode of /etc.  So, I might be missing
something, but this wouldn't give us any problem.

 
> > 4.  How can we obtain the full path of the file system object without
> > reconstructing it in the kernel?  Xattrs? Integer cookies/hashes that
> > key a table in user space?
> 
> There's no such thing.  How do you store a single path in an xattr (which
> is on an inode) for an inode that has multiple hardlinks, or multiple
> vfsmounts (since you can't punt on --bind ;-).  The _best_ you could do
> is go by the path that the user used (audit captures that already IIRC).

You're right.  But to be able to say something about the origin of the
audit....  If you're generating audit records for /etc/shadow, which
was specifically set "auditable," you at least will be able to get
this in the log.  The fact that /etc/shadow was accessed/altered is
whats really important and not necessarily from where it was
accessed/altered, should suffice.  Right?

> > 5.  Whats the best approach at capturing information about the syscall
> > accessing / altering the file/directory?  We might lose records if we
> > attempt to capture this information from VFS hooks as we could
> > short-circuit before we ever reached that point in the function.  We'd
> > then need a way of related the audit of a system call that in place
> > now with the file/directory its accessing. (This particiular issue is
> > not talked about in this e-mail in any detail, really)
> 
> I think you'd need to do more filtering in userspace, and generate a lot
> of extra audit records.  At each point (audit, and lsm hook) there's not
> enough context.  I can't recall when the record is created, if it's at
> the syscall exit point, then the hook can tell you if you actually cared
> about the transaction.

It's probably not wise to collect information in lsm hooks because we
could return out of the function before we ever hit the hook (I think
you talk about this further down) and we'll miss potentially important
audit records.  So auditd would just have to piece together the
complete message in userspace, based on a unique serial number or
something, for all the message that says, "such and such was accessed 
/ altered" with the their corresponding, "by this syscall under these
conditions" part.
> 
> 
> > The Reason:
> > ~~~~~~~~~~~
> >
> > The current audit subsystem does not contain this capability.  This
> > capability is essential for meeting Controlled Access Protection
> > Profile (CAPP) Evaluation Assurance Level (EAL) 4+ requirements.
> >
> > The Solutions:
> > ~~~~~~~~~~~~~~
> >
> > (These are brief notes.  More implementation detail can be provided it
> > asked for.  If someone is interested in discussing an actual solution,
> > we can get into that - I'd like to get into that.  I'm just trying to
> > give you a sense of some of the things that have been thought about).
> >
> > 1.  Inotify. Inotify implements a good portion of what we'd like to do
> > and looks like its well on its way into the mainline kernel.  What
> > would need to be added to inotify:
> >
> > A.  A way in which it could, if desired, auto-watch the file/directory
> > associated with an IN_CREATE_FILE/SUBDIR event.  This would also
> > require we have some way to communicate , "We want this
> > <file/directory> to be auto-watched" -- There might be an issue here
> > because we're now giving meaning to the file/directory name associated
> > with the inode.  We're now saying, "If the inode changes, but the name
> > associated with it is the same, watch the new inode automatically."
> 
> Not exactly, you're saying let me know if the name in this dir goes away
> or gets created.  If it's created then you register for all it's access
> activity. </handwavy>

Yes, but this decision and action would have to be taken in inotify. 

> 
> 
> 
> > B.  Inotify would have to be able to notify the audit subsystem of
> > events via a direct call internally, when the event is triggered.
> > You'd have some device with an "audit" flag on it that says, "Hey I'm
> > interested in receiving events on behalf of audit, send them to me,
> > and I'll pass them directly on to the audit subsystem."  This would be
> > fairly easy to do, but would undermine the inotify event queuing
> > process because we could not take any chances of losing event records
> > in a queue that's overflowed.
> 
> I wonder if you could register with inotify and it could callback to
> you for the events that you register for?  Then you're responsible
> for dealing with audit subsystem, not inotify.  Classic consumer deal,
> can you add yourself to being an event consumer?
> 

Some way for inotify to "notify" other parts of the kernel of file
system activity would be good.  This is the arguement I'd like to use.
 If inotify can notify userspace apps of activity/events, why can't it
notify kernel subsystems?  There might be a very good reason as to why
this can't be so.  "Just because" might be it :-) Whatever the reason,
it'd be good to hear.  I wouldn't want to destroy or degrade the
intended use of inotify, but expand it.  If that's not doable, then
there's no way we can use inotify.  This would have to be something
John and Rob and whoever else contributes to Inotify would like in
addition to the community as a whole.

> 
> > 2.  Linux Security Module.  This is perhaps the most daunting
> > approach. This solution would allow us to use the i_security field of
> > the inode to do our "book keeping."  We'd also have at our disposal
> > the the security hooks already in-place in the VFS.  This would allow
> > us to intercept the relevant syscalls via function pointers, to make
> > decisions on what we want to do.  Using the i_security field to hold
> > our table of auditable files/directories on parent directory inodes,
> > would enable us to ask the question, "Does this file/directory that's
> > just been created beneath me need to be marked as "auditable?" from
> > with in, say, security_inode_post_create().
> >
> > The advantages to a module is that we can make this audit option
> > more... optional.  We're not introducing any new hooks and we're using
> > the LSM's inode i_security field, too.  This keeps the inode
> > structure's size flat and minimizes our impact in the VFS.
> > Disadvantages are that for it to play nicely with other LSMs like
> > SELinux, we'd need the LSM stacker or else we'd need to disable
> > SELinux.  We'd also be forcing the question, "Is this an appropriate
> > use of the LSM framework?"  Although this feature is definitely
> > security-relevant and related, is what it does an appropriate context
> > for use of the LSM framework?  Some would say that the LSM framework
> > is for Access Control only, others would disagree.
> 
> It's on the edge.  LSM is designed for access control, but does give the
> hooking needed to do this security relevant operation.  We specifically
> did not include audit functionality into LSM because of the audit needs
> (syscall centric, early exit points are missed, etc.).
> 
> 
> > 3.  SELinux.  This feature is not currently in SELinux (as we've been
> > told), but we've had interest by the NSA in implementing this feature
> > for SELinux.  We'd have a similar solution to the LSM, except we'd
> > encapsulate ourselves within SELinux, and use its framework to drive
> > the solution.  This might prove less resilient, politically, because
> > we'd be under its umbrella.  I'm not clear on the exact implementation
> > of this in SELinux -- It sounds like it'd be a patch to the security
> > server and hooks, the creation of an audit label, and the invention of
> > some additional structure for "book keeping" purposes.  Also, there is
> > a general problem with using SELinux specifically with figuring out
> > how to certify with it enabled.  There's a lot of complications
> > (mostly unknowns) -- Complications we'd willingly work on, if need be.
> >  Obviously design decisions would have to be factor in as well.
> >
> > 4.  Audit subsystem.  To me, this is the best solution (maybe even
> > worst solution? we'll see what you have to say), but probably the
> > nastiest one.  Let the audit subsystem hook VFS in all the necessary
> > places and give audit an inode field to do the "book keeping."  The
> > advantages here is that we're encapsulating ourselves as much as
> > possible in the audit subsystem and not depending on any other
> > subsystem except for the one constant in all our solutions, VFS.  If
> > we were given the "go ahead" or some fuzzy feeling that this would be
> > the best approach/attempt, we could implement this feature in the
> > audit subsystem.  The disadvantages are fairly clear.  If this feature
> > were configured Y, the inode structure would grow in size and if this
> > were an inode field, we'd be keeping a list of "names," on it.  We'd
> > also be introducing yet another set of hooks to VFS which have a very
> > specific use.  And, on the kernel-side, we'd essentially be
> > duplicating some of inotify's behavior.
> 
> It's sounding like allowing the audit subsystem to subscribe to inotify
> events is the best alternative at first glance.
>

This would be good.  Inotify would need some definate changes to make
this work cleanly.  Right now its very specific in that it queues
events from the VFS to be written to a character device driver.  With
our method we assume everything being watched on a device with an
audit_notify=1 will call the audit_log() function (which writes out to
an audit buffer at which point the audit subsystem takes over).
 
> > However, the most obvious gain here, is that the subsystem is
> > centralized and its intentions are clearer.
> >
> > 5.  Kprobes.  I know nothing about them.  These have not been
> > considered, but were mentioned to me just recently by two different
> > people.  I'm not sure if this approach could be useful or not.  Any
> > help here?
> 
> Well, these just give you primarily ad-hoc hooking.
> 
> thanks,
> -chris
> --
> Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net
> 

-- 
- Timothy R. Chavez

Re: [audit] Upstream solution for auditing file system objects

[Robert Love]

On Fri, 2004-12-10 at 02:50 +0000, Timothy Chavez wrote:

Hi, Timothy.  You work for IBM?

> Some way for inotify to "notify" other parts of the kernel of file
> system activity would be good.  This is the arguement I'd like to use.
>  If inotify can notify userspace apps of activity/events, why can't it
> notify kernel subsystems?  There might be a very good reason as to why
> this can't be so.  "Just because" might be it :-) Whatever the reason,
> it'd be good to hear.  I wouldn't want to destroy or degrade the
> intended use of inotify, but expand it.  If that's not doable, then
> there's no way we can use inotify.  This would have to be something
> John and Rob and whoever else contributes to Inotify would like in
> addition to the community as a whole.

I do not think it makes any sense for inotify to be the mechanism that
implements auditing.  What you want is a general file event mechanism at
the level and time that we currently do the inotify hooks.  I agree,
that is good.  What you also want is to do is hack into inotify your
auditing code.  I don't like that--I don't want inotify to grow into a
generic file system tap.

What we both need, ultimately, is a generic file change notification
system.  This way inotify, dnotify, your audit thing, and whatever else
can hook into the filesystem as desired.

Subverting the inotify project to add this functionality now will only
hurt inotify.  We are not yet in the kernel and we need to streamline
and simplify ourselves, not bloat and featurize.  Besides, indeed, we
are not in the kernel yet--you can just as easily add the hooks
yourself.

So my position would be that I am all for moving the inotify hooks to
generic file change hooks, but that needs to be done either once inotify
is in the kernel proper or as a separate project.  Then inotify can be
one consumer of the hooks and auditing another.

If you want to move forward with a project to hook file system events,
go for it.  Regardless, I think that you should post to lkml your
intentions.

Best,

	Robert Love

Re: [audit] Upstream solution for auditing file system objects

[Timothy Chavez]

On Thu, 09 Dec 2004 22:42:18 -0500, Robert Love <rml@novell.com> wrote:
> On Fri, 2004-12-10 at 02:50 +0000, Timothy Chavez wrote:
> 
> Hi, Timothy.  You work for IBM?
>

Yep.
 
> 
> 
> > Some way for inotify to "notify" other parts of the kernel of file
> > system activity would be good.  This is the arguement I'd like to use.
> >  If inotify can notify userspace apps of activity/events, why can't it
> > notify kernel subsystems?  There might be a very good reason as to why
> > this can't be so.  "Just because" might be it :-) Whatever the reason,
> > it'd be good to hear.  I wouldn't want to destroy or degrade the
> > intended use of inotify, but expand it.  If that's not doable, then
> > there's no way we can use inotify.  This would have to be something
> > John and Rob and whoever else contributes to Inotify would like in
> > addition to the community as a whole.
> 
> I do not think it makes any sense for inotify to be the mechanism that
> implements auditing.  What you want is a general file event mechanism at
> the level and time that we currently do the inotify hooks.  I agree,
> that is good.  What you also want is to do is hack into inotify your
> auditing code.  I don't like that--I don't want inotify to grow into a
> generic file system tap.

Fair enough.  I agree that having something more generic at the hook
level would be ideal.  I'm interested in what others might think about
this.
 
> What we both need, ultimately, is a generic file change notification
> system.  This way inotify, dnotify, your audit thing, and whatever else
> can hook into the filesystem as desired.

> Subverting the inotify project to add this functionality now will only
> hurt inotify.  We are not yet in the kernel and we need to streamline
> and simplify ourselves, not bloat and featurize.  Besides, indeed, we
> are not in the kernel yet--you can just as easily add the hooks
> yourself.

Right, but we like inotify and want to see it succeed :-)!  We also
want an upstream solution, so playing nicely is essential.

> So my position would be that I am all for moving the inotify hooks to
> generic file change hooks, but that needs to be done either once inotify
> is in the kernel proper or as a separate project.  Then inotify can be
> one consumer of the hooks and auditing another.

It's a reasonable compromise and it'll have to be considered and discussed.
 
> If you want to move forward with a project to hook file system events,
> go for it.  Regardless, I think that you should post to lkml your
> intentions.

Most definately.  Thanks for the reply.
 
> Best,
> 
>         Robert Love
> 
> 

-- 
- Timothy R. Chavez

Re: [audit] Upstream solution for auditing file system objects

[Robert Love]

On Fri, 2004-12-10 at 04:38 +0000, Timothy Chavez wrote:

> Right, but we like inotify and want to see it succeed :-)!  We also
> want an upstream solution, so playing nicely is essential.

Awesome. ;)

I'm not adverse to doing the auditing in a generic hook mechanism, at
all, assuming that LSM hooks and the other options are not the preferred
and optimal solution.

> > So my position would be that I am all for moving the inotify hooks to
> > generic file change hooks, but that needs to be done either once inotify
> > is in the kernel proper or as a separate project.  Then inotify can be
> > one consumer of the hooks and auditing another.
> 
> It's a reasonable compromise and it'll have to be considered and discussed.

I've actually been thinking of doing this anyhow, because we currently
have both dnotify and inotify hooks in the filesystem code.  But one
thing at a time.

	Robert Love

Re: [audit] Upstream solution for auditing file system objects

[Chris Wright]

* Robert Love (rml@novell.com) wrote:
> What we both need, ultimately, is a generic file change notification
> system.  This way inotify, dnotify, your audit thing, and whatever else
> can hook into the filesystem as desired.

Yup, makes sense.  From my handwavy perspective inotify was the generic
file event notifcation mechanism, but I agree with your point.

thanks,
-chris
-- 
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net

Re: [audit] Upstream solution for auditing file system objects

[John McCutchan]

On Thu, 2004-12-09 at 22:42 -0500, Robert Love wrote:
> On Fri, 2004-12-10 at 02:50 +0000, Timothy Chavez wrote:
> 
> I do not think it makes any sense for inotify to be the mechanism that
> implements auditing.  What you want is a general file event mechanism at
> the level and time that we currently do the inotify hooks.  I agree,
> that is good.  What you also want is to do is hack into inotify your
> auditing code.  I don't like that--I don't want inotify to grow into a
> generic file system tap.
> 
> What we both need, ultimately, is a generic file change notification
> system.  This way inotify, dnotify, your audit thing, and whatever else
> can hook into the filesystem as desired.

Yes, hopefully we can use some inotify bits when it comes time to do it.

> 
> Subverting the inotify project to add this functionality now will only
> hurt inotify.  We are not yet in the kernel and we need to streamline
> and simplify ourselves, not bloat and featurize.  Besides, indeed, we
> are not in the kernel yet--you can just as easily add the hooks
> yourself.
>
> So my position would be that I am all for moving the inotify hooks to
> generic file change hooks, but that needs to be done either once inotify
> is in the kernel proper or as a separate project.  Then inotify can be
> one consumer of the hooks and auditing another.

I am in complete agreement here. Inotify was not designed with any of
this in mind. Inotify was designed with the desktop in mind. I think a
more realistic approach would be to get inotify in, then refactor from
there. Creating a generic layer that contains some of the inotify code
plus whatever is needed to implement this audit stuff. Then the audit
system and inotify could use this layer for the nitty gritty.

-- 
John McCutchan <ttb@tentacle.dhs.org>

Re: [audit] Upstream solution for auditing file system objects

[Jan Engelhardt]

>> Some notable design problems to introduce now are with the "identity"
>> of Y file/directory with respect to the kernel.  In fact, we only
>> really care about paths of which file/directory Y completes.  I.e: If
>> we want to audit /etc/shadow, we might not care if /etc/shadow is
>> moved to /tmp/shadow.  This means that any access/alteration of
>
>Of course you would.  A mv to /tmp/shadow includes an unlink, which
>should be an auditable event.

Does it really? If /etc and /tmp reside on the same filesystem, only a rename()
is done as to my knowledge (and possibly, an fs-specific rebalance). And if
they are on the same fs, they might even keep the inode number.

>> /tmp/shadow would go unnoticed (unless otherwise specified).  However,
>> if /tmp/shadow were a hardlink to /etc/shadow, we would then still
>> care, because we are still effectively /etc/shadow... if that makes
>> sense :-)

(Your mentioning of hardlinks proves my assumption of you having /etc and /tmp
on the same fs.)


Jan Engelhardt
-- 
ENOSPC

Re: [audit] Upstream solution for auditing file system objects

[James Morris]

On Fri, 10 Dec 2004, Timothy Chavez wrote:

> Greetings, 
>  
> I'm writing this e-mail to facilitate some discussion on an audit
> feature for inclusion to the mainline kernel's audit subsysystem. 

Note, there is a mailing list intended for audit discussion at 
https://www.redhat.com/archives/linux-audit/

It's been a bit quiet, but it may be useful for detailed discussions.

> 2.  Linux Security Module.

As Chris said, LSM is not suitable for CAPP auditing.

> 3.  SELinux.

I don't think it's a good idea to tie everyone in to using SELinux.
Audit and SELinux do need to play well together though (and SELinux is 
already integrated with some of the existing audit subsystem).

> 4.  Audit subsystem.

IMHO we need a distinct Audit subsystem, perhaps even something which can 
accomodate different implementations.

> However, the most obvious gain here, is that the subsystem is
> centralized and its intentions are clearer.

Indeed.


- James
-- 
James Morris
<jmorris@redhat.com>

Re: [audit] Upstream solution for auditing file system objects

[John McCutchan]

On Fri, 2004-12-10 at 00:02 +0000, Timothy Chavez wrote:
>  
> 2.  Filter based on some criteria, "I only care if /etc/shadow is
> open()'d by uid=X" (Filtering could be more granular then this or
> less, that's arbitrary at this point)
>  
> 3.  Be reassured that one may not bypass the audit subsystem nor lose
> an audit record.
>  
> Assuming we can do the filtering aspect in user space via auditd (or
> whatever), we've introduced some kernel work.  For instance, when we
> say we want to audit a file or directory, what we're really saying is,

This could amount to a lot of data being passed from the kernel to user
space. To do a complete audit from user space you would need to provide
as much context from the kernel as possible. 

>  
> "I'm interested in knowing when X syscall accesses/alters Y
> file/directory, so send me a record of this activity."
>  
> Some notable design problems to introduce now are with the "identity"
> of Y file/directory with respect to the kernel.  In fact, we only
> really care about paths of which file/directory Y completes.  I.e: If
> we want to audit /etc/shadow, we might not care if /etc/shadow is
> moved to /tmp/shadow.  This means that any access/alteration of
> /tmp/shadow would go unnoticed (unless otherwise specified).  However,
> if /tmp/shadow were a hardlink to /etc/shadow, we would then still
> care, because we are still effectively /etc/shadow... if that makes
> sense :-)
> 
> These are things that our design must consider in some way. 
>  
> 1.  Identifying a file, such as /etc/shadow, solely by a pathname,
> enables someone to create a hardlink to /etc/shadow from a different
> directory and by-pass the audit subsystem.  Also, we can't assume we
> have this user space definition of a file/directory in the kernel.
>  

hardlinks share the same inode. 

> 2.  Identifying a file, such as /etc/shadow, solely by its inode,
> enables a by-pass of the audit subsystem, when say, /etc/shadow gets
> copied to /etc/shadow+ and moved back.  Because the inode has changed,
> we're no longer the "same" file system object.  Similarly, if we
> delete an audited file, /tmp/ultra_secret (in /tmp? sounds like my old
> university), and recreate it, we'd like /tmp/ultra_secret to be
> audited.  This falls back on the "we don't want audit records to be
> lost" requirement.
>  

Using inotify you can handle this. You watch /tmp for SUBFILE/SUBDIR
creations, and then when you see ultra_secret again, start watching it.
This is very racy though.

> 3.  How do we treat mounts and --bind mounts?  I won't go into this in
> this e-mail.  There's several design decisions to made with regards to
> this issue.

More detail please..



> A.  A way in which it could, if desired, auto-watch the file/directory
> associated with an IN_CREATE_FILE/SUBDIR event.  This would also
> require we have some way to communicate , "We want this
> <file/directory> to be auto-watched" -- There might be an issue here
> because we're now giving meaning to the file/directory name associated
> with the inode.  We're now saying, "If the inode changes, but the name
> associated with it is the same, watch the new inode automatically."
>  

Inotify can provide a racy solution to this problem [see above]. I don't
know if it could be possible to provide an atomic solution to this
problem. Al Viro has lots of nasty things to say about providing any
kind of accurate view of the filesystem to user space.

> B.  Inotify would have to be able to notify the audit subsystem of
> events via a direct call internally, when the event is triggered. 
> You'd have some device with an "audit" flag on it that says, "Hey I'm
> interested in receiving events on behalf of audit, send them to me,
> and I'll pass them directly on to the audit subsystem."  This would be
> fairly easy to do, but would undermine the inotify event queuing
> process because we could not take any chances of losing event records
> in a queue that's overflowed.

Inotify could easily produce events for any part of the system. 

This being said, I don't think that inotify should be the solution.
Inotify was not designed with any of this in mind. A more generic
solution is needed and inotify could sit on top of it.

> 4.  Audit subsystem.  To me, this is the best solution (maybe even
> worst solution? we'll see what you have to say), but probably the
> nastiest one.  Let the audit subsystem hook VFS in all the necessary
> places and give audit an inode field to do the "book keeping."  The
> advantages here is that we're encapsulating ourselves as much as
> possible in the audit subsystem and not depending on any other
> subsystem except for the one constant in all our solutions, VFS.  If
> we were given the "go ahead" or some fuzzy feeling that this would be
> the best approach/attempt, we could implement this feature in the
> audit subsystem.  The disadvantages are fairly clear.  If this feature
> were configured Y, the inode structure would grow in size and if this
> were an inode field, we'd be keeping a list of "names," on it.  We'd
> also be introducing yet another set of hooks to VFS which have a very
> specific use.  And, on the kernel-side, we'd essentially be
> duplicating some of inotify's behavior.
>  
> However, the most obvious gain here, is that the subsystem is
> centralized and its intentions are clearer.

This is probably your best course of action. Later on we could merge
generic bits from this and inotify.
 
-- 
John McCutchan <ttb@tentacle.dhs.org>

Re: [audit] Upstream solution for auditing file system objects

[Paul Rolland]

Hello,

You could also have a look at LIDS. Though the target is not the same,
some useful idea may be gathered there...

http://www.lids.org/

Regards,
Paul

Paul Rolland, rol(at)as2917.net
ex-AS2917 Network administrator and Peering Coordinator

--

Please no HTML, I'm not a browser - Pas d'HTML, je ne suis pas un navigateur

"Some people dream of success... while others wake up and work hard at it" 

  

> -----Message d'origine-----
> De : linux-kernel-owner@vger.kernel.org 
> [mailto:linux-kernel-owner@vger.kernel.org] De la part de 
> Timothy Chavez
> Envoyé : vendredi 10 décembre 2004 01:03
> À : linux-kernel@vger.kernel.org
> Cc : serue@us.ltcfwd.linux.ibm.com; sds@epoch.ncsc.mil; 
> rml@novell.com; ttb@tentacle.dhs.org
> Objet : [audit] Upstream solution for auditing file system objects
> 
> Greetings, 
>  
> I'm writing this e-mail to facilitate some discussion on an audit
> feature for inclusion to the mainline kernel's audit subsysystem. 
> I've written out a "brief" description of the problem with some of the
> associated problems it introduces and some of the outlines of
> potential solutions below.  For the most part, the idea stays the
> same, and the mechanism we use to implement it varies.

Re: [audit] Upstream solution for auditing file system objects

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

On Fri, 10 Dec 2004 00:02:31 GMT, Timothy Chavez said:

> Over the last two months, I've been given the daunting task of
> implementing a feature by which an administrator can specify from user
> space, a list of file system objects (namely regular files and
> directories) that he/she wishes to audit. 

One *big* question that you don't address at all in your mail:

Do you need *real time* tracking of changes/etc, in which case inotify or
something based on it are probably an approach to follow (note that I don't
think inotify currently deals with read-only access to a file).

Or do you not care about real-time tracking, but have a requirement to be
able to go back and say "Oh, at 9:03:38.99342 last Tuesday, user XYZ
tried to open this file" - if that's what you want, you probably want to
look at the audit subsystem and its support for syscall auditing.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [audit] Upstream solution for auditing file system objects

[Timothy Chavez]

On Fri, 10 Dec 2004 09:29:18 -0500, Valdis.Kletnieks@vt.edu
<Valdis.Kletnieks@vt.edu> wrote:
> On Fri, 10 Dec 2004 00:02:31 GMT, Timothy Chavez said:
> 
> > Over the last two months, I've been given the daunting task of
> > implementing a feature by which an administrator can specify from user
> > space, a list of file system objects (namely regular files and
> > directories) that he/she wishes to audit.
> 
> One *big* question that you don't address at all in your mail:
> 
> Do you need *real time* tracking of changes/etc, in which case inotify or
> something based on it are probably an approach to follow (note that I don't
> think inotify currently deals with read-only access to a file).
> 
> Or do you not care about real-time tracking, but have a requirement to be
> able to go back and say "Oh, at 9:03:38.99342 last Tuesday, user XYZ
> tried to open this file" - if that's what you want, you probably want to
> look at the audit subsystem and its support for syscall auditing.
> 
Valdis,

Thank you for the reply.  There is no need for real-time tracking to
meet this requirement.  You have the right idea with the timestamped
message.

And yes, the audit subsystem that's in existence now is something we
want to work with and improve.  Like I had mentioned earlier, it could
just be a matter of logging the VFS relevant portion ("this
<file/directory> was accessed / altered by") with the same serial
number as the syscall ("open() called under these <conditions>") and
then piecing the entire audit record together in userspace and filter
it accordingly.

The main goal here is to not be redundant and make use of our
resources.  I personally like Robert's idea with the general hooking
framework.  Making the audit subsystem, itself, modular also sounds
like a pretty good idea.

Thanks again!

-- 
- Timothy R. Chavez