Auditing File Changes

Auditing File Changes

[eklinger]

Good morning. Please forgive me if this has been asked, but will the file
watch functionality be able to intercept writes and/or be able to
intercept the actual changes to the file and report those, in addition to
the fact that the file was modified?

Thank you
Evan

Re: Auditing File Changes

[Valdis.Kletnieks]

[-- Attachment #1.1: Type: text/plain, Size: 655 bytes --]

On Mon, 10 Jul 2006 10:29:38 PDT, eklinger@uci.edu said:
> Good morning. Please forgive me if this has been asked, but will the file
> watch functionality be able to intercept writes and/or be able to
> intercept the actual changes to the file and report those, in addition to
> the fact that the file was modified?

No.  In particular, logging the actual changes is out of the question,
because of the volume-of-data issue.  Even logging the fact that writes
occurred is painful - that's why the filter usually used is "file being
opened with write permission" and make the assumption that if it opened
it for write, it will/did actually write to it....

[-- Attachment #1.2: Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Auditing File Changes

[Steve Grubb]

On Monday 10 July 2006 13:29, eklinger@uci.edu wrote:
> Please forgive me if this has been asked, but will the file watch
> functionality

We only go after an open command with write permission turned on. The case 
being that you can fill up your logs quickly by intercepting all writes.

> be able to intercept writes and/or be able to intercept the 
> actual changes to the file and report those, in addition to the fact that
> the file was modified? 

No, it will not. If you need to see actual changes, then you need to 
instrument the program in question to log changes. You can look at passwd or 
hwclock as an example of this.

-Steve

Re: Auditing File Changes

[Casey Schaufler]

--- eklinger@uci.edu wrote:

> Good morning. Please forgive me if this has been
> asked, but will the file
> watch functionality be able to intercept writes
> and/or be able to
> intercept the actual changes to the file and report
> those, in addition to
> the fact that the file was modified?

As others have mentioned, the answer is no.

It might be an interesting project to create
a file system that does this level of audit.
Start with efs3's journaling mechanism and
retain all of the update information. True,
you'll run out of space in a hurry, but there
may be environments that would accept that.


Casey Schaufler
casey@schaufler-ca.com

Re: Auditing File Changes

[Valdis.Kletnieks]

[-- Attachment #1.1: Type: text/plain, Size: 595 bytes --]

On Mon, 10 Jul 2006 12:32:14 PDT, Casey Schaufler said:

> It might be an interesting project to create
> a file system that does this level of audit.
> Start with efs3's journaling mechanism and
> retain all of the update information. True,
> you'll run out of space in a hurry, but there
> may be environments that would accept that.

I'm not sure whether the right answer here is VMS style file versions,
or IBM big iron style database transaction logs (replaying the IMS logs
on restart, yippee! :)

Probably depends on what actual problem he's trying to solve by recording
all the changes.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Auditing File Changes

[Klaus Weidner]

On Mon, Jul 10, 2006 at 12:32:14PM -0700, Casey Schaufler wrote:
> It might be an interesting project to create
> a file system that does this level of audit.
> Start with efs3's journaling mechanism and
> retain all of the update information. True,
> you'll run out of space in a hurry, but there
> may be environments that would accept that.

How about using FUSE (userspace file system) on top of a version control
system (git, subversion, ...), and using the version control system's
change records to keep track of the full history?

-Klaus

Re: Auditing File Changes

[LC Bruzenak]

On Mon, 2006-07-10 at 15:42 -0400, Valdis.Kletnieks@vt.edu wrote:
...
> 
> Probably depends on what actual problem he's trying to solve by recording
> all the changes.

Most likely the same one I have been working on all my career:

Security guy: Please deliver system with maximum security.
System guy (me): What do you need to know?
Security guy: Any and all changes to security-relevant files.
System guy: Which ones are those?
Security guy: All of 'em.

Basically my plan is this:
As Steve Grubb said, instrument the processes with trusted access.
Have file watches which note when certain "critical" files are opened
for write/append.
Have an audit analysis program which compares the trusted accesses to
the total accesses; the delta shows potentially interesting mods.

LCB.

-- 
LC Bruzenak
lenny@bruzenak.com

Re: Auditing File Changes

[Casey Schaufler]

--- Klaus Weidner <klaus@atsec.com> wrote:

> On Mon, Jul 10, 2006 at 12:32:14PM -0700, Casey
> Schaufler wrote:
> > It might be an interesting project to create
> > a file system that does this level of audit.
> > Start with efs3's journaling mechanism and
> > retain all of the update information. True,
> > you'll run out of space in a hurry, but there
> > may be environments that would accept that.
> 
> How about using FUSE (userspace file system) on top
> of a version control
> system (git, subversion, ...), and using the version
> control system's
> change records to keep track of the full history?

I donno, those small writes are going to be
kind of slow.


Casey Schaufler
casey@schaufler-ca.com

Re: Auditing File Changes

[Valdis.Kletnieks]

[-- Attachment #1.1: Type: text/plain, Size: 1097 bytes --]

On Mon, 10 Jul 2006 14:56:47 CDT, LC Bruzenak said:

> Security guy: Please deliver system with maximum security.

At this point, we already know the request is a crock.  Even fascist
military-style security recognizes that there exist tradeoffs (which is
why "Secret" and "Top Secret" have different requirements to prevent
disclosure, and so on).

> System guy (me): What do you need to know?
> Security guy: Any and all changes to security-relevant files.

You missed the obvious questions here:

Do you need the exact change, or is the fact that an unauthorized change
happened sufficient (you can always just get the over version off backups
and diff them)?

Do you need to know about unexplained failed attempts as well?

> System guy: Which ones are those?
> Security guy: All of 'em.

*all* files are security relevant?  Yowza. :)

There's not much that the audit code can do to support an unrealistic
design.  There may not be much it can do to support a *realistic* one that
has certain requirements - but at least at that point we can point you
at other tools to address the issues...


[-- Attachment #1.2: Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Auditing File Changes

[Valdis.Kletnieks]

[-- Attachment #1.1: Type: text/plain, Size: 1329 bytes --]

On Mon, 10 Jul 2006 14:56:47 CDT, LC Bruzenak said:

(Addressing the actual design seperately)

> As Steve Grubb said, instrument the processes with trusted access.
> Have file watches which note when certain "critical" files are opened
> for write/append.
> Have an audit analysis program which compares the trusted accesses to
> the total accesses; the delta shows potentially interesting mods.

Ahh... but to find that delta, you don't really need to record the actual
changes, do you?  You can (hopefully/presumably) then recover the old version
of the modified file and diff it.

And "comparing trusted accesses to total accesses" is quite possibly flawed as
well - I've lost count of times that the audit trail has clearly said that a
"trusted program" did something, and the *actual* security issue was the user
went to the bathroom and a locking screensaver wasn't engaged, allowing
somebody else to run the program surreptitiously....

Yes, a proper and correct audit trail is required - but on *any* realistic
system, the result is a fire hose of data, and no amount of data mining will
produce anything but wrong answers if you ask it the wrong questions....

(Sorry for those on the list who have a grasp on that - I find an incredible
number of people in the security industry need reminding on a regular basis..)



[-- Attachment #1.2: Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Auditing File Changes

[Timothy R. Chavez]

On Mon, 2006-07-10 at 14:56 -0500, LC Bruzenak wrote:
> On Mon, 2006-07-10 at 15:42 -0400, Valdis.Kletnieks@vt.edu wrote:
> ...
> > 
> > Probably depends on what actual problem he's trying to solve by recording
> > all the changes.
> 
> Most likely the same one I have been working on all my career:
> 
> Security guy: Please deliver system with maximum security.
> System guy (me): What do you need to know?
> Security guy: Any and all changes to security-relevant files.
> System guy: Which ones are those?
> Security guy: All of 'em.

If you issue 100 writes() in between an open() and an exit(), what does
write() #97 tell you that write() #26 doesn't?  You'd pretty much have
to log what changed per-write, which would consume a ridiculous amount
of space in addition to what would be consumed by logging write()'s in
general.  I think, from a security stand point, it's more interesting to
know if, when, and by whom the file was open'ed() for write.

-tim

Re: Auditing File Changes

[Klaus Weidner]

On Mon, Jul 10, 2006 at 04:51:31PM -0400, Valdis.Kletnieks@vt.edu wrote:
> And "comparing trusted accesses to total accesses" is quite possibly flawed as
> well - I've lost count of times that the audit trail has clearly said that a
> "trusted program" did something, and the *actual* security issue was the user
> went to the bathroom and a locking screensaver wasn't engaged, allowing
> somebody else to run the program surreptitiously....

That one is easy to fix by including a current webcam picture of the user
in each audit record in addition to the auid ;-)

-Klaus

Re: Auditing File Changes

[Timothy R. Chavez]

On Mon, 2006-07-10 at 16:51 -0400, Valdis.Kletnieks@vt.edu wrote:
> On Mon, 10 Jul 2006 14:56:47 CDT, LC Bruzenak said:
> 
> (Addressing the actual design seperately)
> 
> > As Steve Grubb said, instrument the processes with trusted access.
> > Have file watches which note when certain "critical" files are opened
> > for write/append.
> > Have an audit analysis program which compares the trusted accesses to
> > the total accesses; the delta shows potentially interesting mods.
> 
> Ahh... but to find that delta, you don't really need to record the actual
> changes, do you?  You can (hopefully/presumably) then recover the old version
> of the modified file and diff it.

Assuming you were wanting to audit write()'s, maybe the record could
include the offset into the file where writing began and how many bytes
were written.  This obviously has some pretty major limitations in
usefulness, but would be more feasible than actually logging the
differences!

<snip>

-tim

Re: Auditing File Changes

[Timothy R. Chavez]

On Mon, 2006-07-10 at 15:55 -0500, Timothy R. Chavez wrote:
> If you issue 100 writes() in between an open() and an exit(), what does
                                                        ^^^^^
*cough* close() *cough* oops ;-)

-tim

Re: Auditing File Changes

[eklinger]

> On Mon, 2006-07-10 at 15:42 -0400, Valdis.Kletnieks@vt.edu wrote:
> ...
>>
>> Probably depends on what actual problem he's trying to solve by
>> recording
>> all the changes.
>
> Most likely the same one I have been working on all my career:
>
Actually I'm trying to prevent certain files from leaving the computer,
specifically source code. However, that means I need to watch for file
copies, renames, cut and pastes, emails, etc. The idea was to encapsulate
the actual file data in an encrypted wrapper that would have to be
opened/decrypted by our program. The wrapper would also contain the
allowed operations on the file data itself, which is where auditing would
come in so that we can see what the user is attempting to do with the
file. After we decrypt the file and remove the wrapper, the raw data would
be opened in the appropriate application on the system (e.g.
OpenOffice.org). However, at the save we would want to add that wrapper
back in so they could not simply circumvent the wrapper protection. Of
course, we don't want to have to modify any of the user level applications
to achieve this functionality.

Thanks
Evan

Re: Auditing File Changes

[Klaus Weidner]

On Mon, Jul 10, 2006 at 01:24:05PM -0700, Casey Schaufler wrote:
> --- Klaus Weidner <klaus@atsec.com> wrote:
> > How about using FUSE (userspace file system) on top of a version
> > control system (git, subversion, ...), and using the version control
> > system's change records to keep track of the full history?
> 
> I donno, those small writes are going to be kind of slow.

Hmm, I wonder how much would break if the filesystem treated the close()
as a commit in the version control system, with the changes not being
visible to other processes until that happens...

(or exit() instead of close() for a more transactional approach)

-Klaus

Re: Auditing File Changes

[Valdis.Kletnieks]

[-- Attachment #1.1: Type: text/plain, Size: 1037 bytes --]

On Mon, 10 Jul 2006 14:11:42 PDT, eklinger@uci.edu said:

> file. After we decrypt the file and remove the wrapper, the raw data would
> be opened in the appropriate application on the system (e.g.
> OpenOffice.org). However, at the save we would want to add that wrapper
> back in so they could not simply circumvent the wrapper protection. Of
> course, we don't want to have to modify any of the user level applications
> to achieve this functionality.

OpenOffice 'Save As...' will be a can of worms, as will the fact that it
can have multiple documents open at once, and you can slice-n-mice between
them.   Do you have an X server that properly enforces MLS/MCS restrictions
so cut-n-paste between documents won't work?  Otherwise, they can just
cut it from their OpenOffice document, paste it into Evilution, and hit 'send'.

You're much better off just fixing the machines so you can't attach removable
media to them (no USB/firewire/etc), and then restrict the network access
so they can't send the bits anyplace interesting....

[-- Attachment #1.2: Type: application/pgp-signature, Size: 226 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Auditing File Changes

[Timothy R. Chavez]

On Mon, 2006-07-10 at 14:11 -0700, eklinger@uci.edu wrote:
> > On Mon, 2006-07-10 at 15:42 -0400, Valdis.Kletnieks@vt.edu wrote:
> > ...
> >>
> >> Probably depends on what actual problem he's trying to solve by
> >> recording
> >> all the changes.
> >
> > Most likely the same one I have been working on all my career:
> >
> Actually I'm trying to prevent certain files from leaving the computer,
> specifically source code. However, that means I need to watch for file
> copies, renames, cut and pastes, emails, etc. The idea was to encapsulate
> the actual file data in an encrypted wrapper that would have to be
> opened/decrypted by our program. The wrapper would also contain the
> allowed operations on the file data itself, which is where auditing would
> come in so that we can see what the user is attempting to do with the
> file. After we decrypt the file and remove the wrapper, the raw data would
> be opened in the appropriate application on the system (e.g.
> OpenOffice.org). However, at the save we would want to add that wrapper
> back in so they could not simply circumvent the wrapper protection

Maybe it's the way you've described it, but this sounds like a very
contrived and fickle security mechanism.  I really don't understand the
purpose of your encryption, can you elaborate any?  Maybe I'm just
confused with the example you gave.  Further more, if you want to
restrict operations on a given a file, why reinvent the wheel, it's
already doable.  Also, the audit subsystem does log events describing
"copy" events, renames, linking, unlinking, open's, close's, file
attribute modifications, etc, without the need for modifying specific
programs.  Decompose the "abstract" event of cut and paste into its
system-calls and there you go.

> . Of
> course, we don't want to have to modify any of the user level applications
> to achieve this functionality.

You'd have to modify OpenOffice to decrypt and re-encrypt documents,
right?

-tim

Re: Auditing File Changes

[Casey Schaufler]

--- Klaus Weidner <klaus@atsec.com> wrote:

 
> That one is easy to fix by including a current
> webcam picture of the user
> in each audit record in addition to the auid ;-)

This was in the spec for Irix 6.6 auditing.
Really, I'm not kidding.


Casey Schaufler
casey@schaufler-ca.com

Re: Auditing File Changes

[eklinger]

> Maybe it's the way you've described it, but this sounds like a very
> contrived and fickle security mechanism.  I really don't understand the
> purpose of your encryption, can you elaborate any?  Maybe I'm just
> confused with the example you gave.  Further more, if you want to
> restrict operations on a given a file, why reinvent the wheel, it's
> already doable.  Also, the audit subsystem does log events describing
> "copy" events, renames, linking, unlinking, open's, close's, file
> attribute modifications, etc, without the need for modifying specific
> programs.  Decompose the "abstract" event of cut and paste into its
> system-calls and there you go.
>
The original idea was to prevent the user from opening the file in any
text or hex editor and changing the file or the file's allowed operations,
which would be stored in the file itself. However, if we can capture the
open call we may not need the the encryption afterall. All of this is just
a proof of concept. It will need to be refined much more before we do the
actual implementation, which is why I'm here to get these comments and
ideas from the community. :) We do not want to reinvent the wheel but the
permissions need to go beyond the basic read-write-exec since engineers
will need to modify the source code files but we may not want them to copy
them to a usb drive or email them, and we want the permissions to be in
place across platforms.

>
> You'd have to modify OpenOffice to decrypt and re-encrypt documents,
> right?

The idea was that we would capture the writes and the data and then
encrypt it ourselves so OO would not need to care.

Evan

Re: Auditing File Changes

[Darren Cole]

> The original idea was to prevent the user from opening the file in any
> text or hex editor and changing the file or the file's allowed  
> operations,
> which would be stored in the file itself. However, if we can  
> capture the
> open call we may not need the the encryption afterall. All of this  
> is just
> a proof of concept. It will need to be refined much more before we  
> do the
> actual implementation, which is why I'm here to get these comments and
> ideas from the community. :) We do not want to reinvent the wheel  
> but the
> permissions need to go beyond the basic read-write-exec since  
> engineers
> will need to modify the source code files but we may not want them  
> to copy
> them to a usb drive or email them, and we want the permissions to  
> be in
> place across platforms.

selinux can do this using policy (MLS policy specifically) and of  
course auditting.  The idea is that you can label a file to be  
Secret.  To access this file the user must be at Secret.  When at  
Secret they can only write at Secret, so even if they change  
permissions or copy the data, it will still be at Secret and only  
users allowed Secret access can see these files.  You can then define  
at what level a usb device is mounted, or what level the email system  
runs at, so that Secret data cannot be written to the device or sent  
through email.  This is greatly simplified explanation of MLS and  
more detail is probably outside the scope of the audit list.  You  
might try looking at the lssp-redhat list <http://www.redhat.com/ 
mailman/listinfo/redhat-lspp>, or Fedora MLS wiki <http:// 
www.fedoraproject.org/wiki/SELinux/FedoraMLSHowto>.

The above is of course specific to MLS systems (selinux, Trusted  
Solaris, HP-UX 10.26 and several others), so it isn't really cross  
platform in they way you probably want.  Still it might give you a  
place to start so you don't reinvent the wheel.

>>
>> You'd have to modify OpenOffice to decrypt and re-encrypt documents,
>> right?
>
> The idea was that we would capture the writes and the data and then
> encrypt it ourselves so OO would not need to care.

In an MLS system OO should not care either.

>
> Evan

Re: Auditing File Changes

[Stephen John Smoogen]

On 7/10/06, eklinger@uci.edu <eklinger@uci.edu> wrote:
> > Maybe it's the way you've described it, but this sounds like a very
> > contrived and fickle security mechanism.  I really don't understand the
> > purpose of your encryption, can you elaborate any?  Maybe I'm just
> > confused with the example you gave.  Further more, if you want to
> > restrict operations on a given a file, why reinvent the wheel, it's
> > already doable.  Also, the audit subsystem does log events describing
> > "copy" events, renames, linking, unlinking, open's, close's, file
> > attribute modifications, etc, without the need for modifying specific
> > programs.  Decompose the "abstract" event of cut and paste into its
> > system-calls and there you go.
> >
> The original idea was to prevent the user from opening the file in any
> text or hex editor and changing the file or the file's allowed operations,
> which would be stored in the file itself. However, if we can capture the
> open call we may not need the the encryption afterall. All of this is just
> a proof of concept. It will need to be refined much more before we do the
> actual implementation, which is why I'm here to get these comments and
> ideas from the community. :) We do not want to reinvent the wheel but the
> permissions need to go beyond the basic read-write-exec since engineers
> will need to modify the source code files but we may not want them to copy
> them to a usb drive or email them, and we want the permissions to be in
> place across platforms.
>

Well a lot of things I think you need are not in place yet (MLS/MCS X
server, and some other parts (webcam audit :)))

If you need a solution now then you need to go to the standard
physical beats technical listings:

1) Dont put the boxes on the internet. If they need internet access it
is done via a two system enclave and/or a one way transfer of data via
diode.

2) KVM with a trusted KVM system and put the CPU/hard-disks in a
controlled vault type enviroment. If a system has to be in the
physical control of the engineer then epoxy any port that isnt in use
(USB, Firewire) and physically tie down the keyboard/mouse etc to the
box.

3) Design processes for handling data between enclaves, handling data,
how to handle removable media, etc.


-- 
Stephen J Smoogen.
CSIRT/Linux System Administrator

Re: Auditing File Changes

[Steve Grubb]

On Monday 10 July 2006 18:09, eklinger@uci.edu wrote:
> The original idea was to prevent the user from opening the file in any
> text or hex editor and changing the file or the file's allowed operations,
> which would be stored in the file itself.

The access has already occured by the time the audit system tells you about 
it. You are simply too late. What you need its access control. The MCS 
capabilities in SE Linux/FC5 may help you. You can google for MCS.

-Steve