[RFC]Kernel Built-in IDS extension

[RFC]Kernel Built-in IDS extension

[horietk]

Hi,

I'm glad to announce the release of experimental kernel extensions to 
SELinux access control, which enables intrusion detection and system 
protection functionalities.

The extensions have following features.
1. Detecting the access violation and can log that trial.
2. Operating booleans of 'Conditional Policy' when the access violation 
is detected.
3. Kernel built-in implementation. All the detections and the operations 
are performed within kernel.
4. To specify access contexts to be detected and boolean variables to be 
turned on, extended policy notation was introduced, which resembles the 
'allow' statement.

   trap <subject> <object>:<class> <permissions> <booleans> ;

Also, the AVC 'detected' log is recorded. (see below)

"Jun 13 10:05:47 molder kernel: audit(1118624747.611:0): avc:  detected  
{ execute } for  pid=4126 exe=/usr/sbin/in.ftpd name=sh dev=dm-0 
ino=3572008 scontext=scontext=system_u:system_r:ftpd_t 
tcontext=system_u:object_r:shell_exec_t tclass=file"

I believe that these features are useful in some cases.
For examaple, it is possible to detect harzardous access context which 
appears only when malicious operation is tried and to apply more 
restricted policy by means of turning on booleans automatically.

example) buffer overflow detection and system protection.
  bool etc_locked false ;
  bool bool_locked false ;
  trap ftpd_t shell_exec_t:file { execute } { etc_locked bool_locked } ;
  if (etc_locked) {
     /* read only, no one can change /etc files  */
     allow sysadm_t etc_t:file { r_file_perms }; 
  } else {
     /* sysadm can edit /etc files */
     allow sysadm_t etc_t:file { rw_file_perms }; 
  }
  if (!bool_locked) {
     /* sysadm can operate booleans */
     allow sysadm_t security_t:security setbool;
  }

If an unauthorized shell execution tried by ftpd is detected, kernel 
itself performs policy modification.

Of course, not only 'unconditional trap' but also 'conditional trap' can 
be used too.
  bool bool_a false ;
  bool bool_b false ;
  bool bool_c false ;
  if (bool_a) {
     trap xxx_t yyy_t:file { perms } { bool_b } ;
  } else {
     trap xxx_t yyy_t:file { perms } { bool_a bool_c } ;
  }
So, rather complicated boolean operations can be determined.

The patches and the documents are available at SourceForge site.
  http://sourceforge.net/projects/lk-ids

There are three patches.
[1] kernel patch (to version 2.6.11)
 - Member correspoding to 'trap' vector is added to avtab and policydb 
structures. (Mainly avc.h and avtab.h modifications)
 - Requested permission is compared with 'trap' vector, separate from 
SELinux's permission check. (Mainly avc.c modifications)
 - Internal boolean operation by kernel itself. (Mainly conditional.c 
and selinuxfs.c)
[2] checkpolicy patch (to version 1.22)
[3] libsetools patch (to version 1.4)
 - Policy parser, compiler and related library are extended to meet 
'trap' notation.

I hope I could have a lot of comments, suggestions and feedbacks.

Best regards,

-----
NTT DATA CORPORATION
Open Source Software Development Center
Takashi Horie
E-Mail: horie-t@users.sourceforge.net
E-Mail: horietk@nttdata.co.jp

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: [RFC]Kernel Built-in IDS extension

[Luke Kenneth Casson Leighton]

On Fri, Jun 17, 2005 at 06:16:47PM +0900, horietk@nttdata.co.jp wrote:

> I believe that these features are useful in some cases.
> For examaple, it is possible to detect harzardous access context which 
> appears only when malicious operation is tried and to apply more 
> restricted policy by means of turning on booleans automatically.

 COOL!


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: [RFC]Kernel Built-in IDS extension

[Lorenzo Hernández García-Hierro]

[-- Attachment #1: Type: text/plain, Size: 4669 bytes --]

El vie, 17-06-2005 a las 18:16 +0900, horietk@nttdata.co.jp escribió:
> The extensions have following features.
> 1. Detecting the access violation and can log that trial.
> 2. Operating booleans of 'Conditional Policy' when the access violation 
> is detected.
> 3. Kernel built-in implementation. All the detections and the operations 
> are performed within kernel.
> 4. To specify access contexts to be detected and boolean variables to be 
> turned on, extended policy notation was introduced, which resembles the 
> 'allow' statement.

As far as I have looked over it, it looks nice and useful, but as
everything does, it has some shortcomings.

> I believe that these features are useful in some cases.
> For examaple, it is possible to detect harzardous access context which 
> appears only when malicious operation is tried and to apply more 
> restricted policy by means of turning on booleans automatically.

There's a main problem regarding this implementation: false positives
and trapping limited to the end of the execution flow.

On the first it's just that, used in a messy manner and configured
improperly it would cause what any IDS can do: take the least
possibility as a great attack vector (ie. some commercial products that
try to catch up strings inside the requests against a web server, some
like "exec", "bash", etc).

On the latter: you're looking only at your side (in this case, you catch
the end of the attack). Not all the vulnerabilities will lead to
spawning a shell, there are many other areas of interest for the
attacker (ie. in format string handling bugs retrieving potentially
confidential information and, if well prepared and designed, lead the
application to return itself to the point where the execution flow was
crafted to catch up such confidential information).

In some cases, it won't be necessary for the attacker to execute an
arbitrary instruction in the right time, but only to force the
application itself to execute *code of it's own* at the wrong time, with
the wrong information.

If the above statement sounds confusing, let's try to clarify: if the
XYZ application is compromised by using a specifically designed attack
vector (ie. the attacker knows well the relevant information to exploit
the vulnerability at issue and has enough access and time to verify that
it will be successful), which doesn't make use of external
instructions/code/what-ever-else you call it, but leads the execution
flow to return to a foo() function which may rewrite a critical memory
area, and such area will be read by the function which was compromised,
does your kernel-land IDS trap the trick? Not at all (if there's no
policy that determines at runtime which memory areas the application
will need to protect, at the exact and right time, and how it will need
to un-protect them depending on the action to be performed. That is,
something pretty complex which would lead to have in mind the whole
specification of the architecture being used, the memory management code
and the segmentation for user-land application, and a long etc of other
really-weird-and-painful-complex stuff which makes it no worth effort,
but another reason for taking care at coding the user-land applications
themselves).

I don't say this a common case, but there are many cases, either
publicly available or not, that demonstrate that in such cases, IDS and
other so-called "Proactive security" technologies are useless.

I've written a little patch [1] for making IBM Stack Smashing Protector
(aka ProPolice) able to use kernel-land helpers for trapping the stack
smashing events caused by the implementation itself (currently it's only
implemented in libssp [2]), hence making possible to integrate it in a
near future with SELinux or any other thing relying in the Linux
Security Modules framework. but it's just an example of how "defense in
depth" would need to be deployed widely, and not only in small bits
inside the kernel (no mention to the possible overhead caused by
implementing an IDS in kernel space, on top of the SELinux overhead,
which is minimal indeed).

Anyways, great work. Keep it up.

[1]:http://pearls.tuxedo-es.org/patches/ssp-propolice/propolice_kernel_helpers.patch
(It might not work correctly for sparc, as I didn't take a lot of care
regarding the implementation for any other arch. rather than ia32/i386
and I couldn't test it under other architectures)
[2]: http://pearls.tuxedo-es.org/papers/libssp.pdf

Cheers,
-- 
Lorenzo Hernández García-Hierro <lorenzo@gnu.org>
[1024D/6F2B2DEC] & [2048g/9AE91A22][http://tuxedo-es.org]

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC]Kernel Built-in IDS extension

[Stephen Smalley]

On Fri, 2005-06-17 at 18:16 +0900, horietk@nttdata.co.jp wrote:
> I'm glad to announce the release of experimental kernel extensions to 
> SELinux access control, which enables intrusion detection and system 
> protection functionalities.

Thanks for making this available.

> The extensions have following features.
> 1. Detecting the access violation and can log that trial.
> 2. Operating booleans of 'Conditional Policy' when the access violation 
> is detected.
> 3. Kernel built-in implementation. All the detections and the operations 
> are performed within kernel.
> 4. To specify access contexts to be detected and boolean variables to be 
> turned on, extended policy notation was introduced, which resembles the 
> 'allow' statement.
> 
>    trap <subject> <object>:<class> <permissions> <booleans> ;

Did you consider a userspace implementation, e.g. extend auditd to match
against avc messages it receives from the kernel (likely using a
libsepol function to perform the matching), and then having auditd
adjust boolean settings in response to particular avc messages?
Naturally, that would yield a slower response time to events, and you
would have to ensure that avc messages were generated for all events of
interest (e.g. using appropriate auditallow rules if you wanted to trap
on an allowed permission, and being careful not to dontaudit a denied
permission if you wanted to trap it), but would avoid having to further
complicate the kernel code.  There has been quite a bit of work recently
on the kernel's audit framework and the userspace auditd, and a fair
amount of those enhancements are included in FC4.

Also, it appears that your trap syntax only allows you to enable
booleans upon events, whereas I could envision one wanting to disable
booleans as well.  Otherwise, people have to contort their boolean
definitions to correspond with this limitation.

> The patches and the documents are available at SourceForge site.
>   http://sourceforge.net/projects/lk-ids

Posting patches to the list is preferred for review and feedback.
Requiring people to separately download a tarball reduces the set of
people who are likely to bother commenting.

> There are three patches.
> [1] kernel patch (to version 2.6.11)
>  - Member correspoding to 'trap' vector is added to avtab and policydb 
> structures. (Mainly avc.h and avtab.h modifications)
>  - Requested permission is compared with 'trap' vector, separate from 
> SELinux's permission check. (Mainly avc.c modifications)
>  - Internal boolean operation by kernel itself. (Mainly conditional.c 
> and selinuxfs.c)
> [2] checkpolicy patch (to version 1.22)
> [3] libsetools patch (to version 1.4)
>  - Policy parser, compiler and related library are extended to meet 
> 'trap' notation.

When developing new SELinux functionality, it is preferable to work
against:
1) the latest upstream kernel for kernel code (presently 2.6.12-rc6 or
the latest git snapshot or the latest -mm patchset),
2) the selinux sourceforge CVS tree for checkpolicy, libsepol or other
userland components (presently up to 1.23.4 and 1.5.10 respectively).
Fedora Core CVS tree also tends to track this closely.

Otherwise, your patches may not apply very well.

> I hope I could have a lot of comments, suggestions and feedbacks.

A few quick comments based on a cursory read of the kernel patch:

- A transient memory failure could cause the trap information to be
dropped from an AVC node in avc_node_populate(), with not even a log
message warning about it - thus you could fail to properly trap an
access under memory pressure.  In contrast, allocation failure for the
node itself by avc_insert() does no harm - the existing
avc_has_perm_noaudit() code just falls back to using the temporary entry
on the stack that was filled by security_compute_av() to finish the
checking.

- Possibly should be storing your boolean value array as a pair of
bitmaps instead (one bitmap to indicate which booleans to enable, one
bitmap to indicate which booleans to disable) to reduce the need to
allocate and copy around potentially large integer arrays.

- avc_has_perm() cannot invoke any potentially blocking (sleeping)
operations, because it can be called from irq or while locks are held by
the kernel.  Hence, your code (e.g. sel_set_booles_by_trap) likewise
cannot invoke any potentially blocking operations, like down.  Of
course, taking that particular semaphore and using the shared
bool_pending_values is suspect anyway, and you already have a kludge to
avoid double locking in the load_policy case (and the setbool case would
also be a problem for you).  Why not just create and use your own
temporary value array so that you don't have any locking issue there?

- avc_detect_perm_noaudit() calls security_compute_av() and avc_insert()
after updating the booleans.  The boolean update will have flushed the
cache entirely (which is necessary, as the booleans may have affected
multiple entries), so you might as well just wait and let the cache
re-fill in the usual manner upon subsequent permission checks.  It also
doesn't make sense for it to change avd because it is only used by
avc_audit(), which has already been called, and you don't want to adjust
that result for the new booleans (because avc_has_perm made its
decisions based on the old boolean settings).

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: [RFC]Kernel Built-in IDS extension

[James Morris]

On Fri, 17 Jun 2005, Stephen Smalley wrote:

> Did you consider a userspace implementation, e.g. extend auditd to match
> against avc messages it receives from the kernel (likely using a
> libsepol function to perform the matching), and then having auditd

Why limit the input to the IDS to AVC messages?  There are many ways of
collecting attack signatures, and not necessarily even from the local 
system.

And yes, this definitely does not belong in the kernel.

Also see http://personal.utulsa.edu/~brandon-pollet/DynamicPolicy-BP.pdf


- James
-- 
James Morris
<jmorris@redhat.com>




--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: [RFC]Kernel Built-in IDS extension

[Russell Coker]

On Saturday 18 June 2005 02:08, James Morris <jmorris@redhat.com> wrote:
> On Fri, 17 Jun 2005, Stephen Smalley wrote:
> > Did you consider a userspace implementation, e.g. extend auditd to match
> > against avc messages it receives from the kernel (likely using a
> > libsepol function to perform the matching), and then having auditd
>
> Why limit the input to the IDS to AVC messages?  There are many ways of
> collecting attack signatures, and not necessarily even from the local
> system.

I have been considering writing an extension to the watchdog daemon (which 
currently only seems to be in Debian) to have SE Linux support.

Usually when a server has an unreasonably high load average the administrator 
can make a good guess at what the likely culprit is likely to be.  For 
example a university shell server with a high load average is probably caused 
by a badly written program written by someone studying Unix/C 101.  The 
current functionality of the watchdog daemon is to reboot the machine if the 
load stays high (or some other criteria are met).  Wheras it's quite likely 
that merely killing a certain class of process would do the job.

In the university shell server example causing all programs run by 
undergraduate students to cease operation (by flipping a SE Linux boolean) 
would be greatly preferrable to rebooting the machine and therefore killing 
processes from staff members and post-grads.  If stopping the processes from 
under-grads doesn't do the job then the second step would be to reboot the 
machine.

It would probably be best to have a user-space program that locks it's memory 
in core and sets real-time priority to monitor audit.log, the load average, 
and other sources of information that may relate to an attack.


In the example that was given changing a boolean is not something that needs 
to happen urgently.  The execution of the shell was denied so the 
administrator is apparently concerned about what operations may be done with 
the compromised ftpd other than executing a shell.  The administrator can't 
rely on the attacker trying to execute a shell as their first action, so the 
attacker may do that after launching other attacks.

Finally I have doubts about the IDS use of a single trigger.  On today's 
Internet you would not consider your server to be under attack if a single 
suspect packet comes in, the volume of such strange traffic is so great that 
strange things happen all the time.  It's only if you have a series of 
suspicious packets coming in that you would consider it to be evidence of an 
attack.  I believe that the same rules apply for SE Linux audit messages.  
One message may be a corner case that the author of the policy didn't 
consider (for example it was quite a long time after I initially wrote the 
Postfix policy that I discovered the Postfix functionality that occurs when a 
message in the queue times out while the daemon is not running).  It's only a 
series of accesses that may be considered evidence of attack.

I know that some people have programs to slowly send probe packets to servers 
over the course of days or weeks to get past IDS systems that watch for a 
barrage of packets.  There's not much we can do about that, you just have to 
make sure that your machine is reasonably secure at all times.

-- 
http://www.coker.com.au/selinux/   My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/  Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/    Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/  My home page

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: [RFC]Kernel Built-in IDS extension

[Stephen Smalley]

On Fri, 2005-06-17 at 18:16 +0900, horietk@nttdata.co.jp wrote:
> example) buffer overflow detection and system protection.
>   bool etc_locked false ;
>   bool bool_locked false ;
>   trap ftpd_t shell_exec_t:file { execute } { etc_locked bool_locked } ;
>   if (etc_locked) {
>      /* read only, no one can change /etc files  */
>      allow sysadm_t etc_t:file { r_file_perms }; 
>   } else {
>      /* sysadm can edit /etc files */
>      allow sysadm_t etc_t:file { rw_file_perms }; 
>   }
>   if (!bool_locked) {
>      /* sysadm can operate booleans */
>      allow sysadm_t security_t:security setbool;
>   }

This particular example is not particularly motivating, since:
a) you aren't doing anything to lock down ftpd_t itself upon this trap,
and
b) you are instead locking down sysadm_t, so this presents an obvious
denial-of-service attack against legitimate admins.

Note that self:process execmem makes a good candidate for a trap, as
that reflects an attempt to make anonymous memory executable, e.g. an
attempt to make the stack executable.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

RE: [RFC]Kernel Built-in IDS extension

[horietk]

Thanks a lot.
I appreciate helpful comments, especially, about in-depth AVC 
implementations.

> -----Original Message-----
> 
> Did you consider a userspace implementation, e.g. extend 
> auditd to match
> against avc messages it receives from the kernel (likely using a
> libsepol function to perform the matching), and then having auditd
> adjust boolean settings in response to particular avc messages?

Certainly, I agree that the logging daemon based implementation is much 
simpler than kernel based one. But I think syslogd may not be reliable 
enough for this kind of purpose, i.e. printk() loses some of messages 
under very high load. So this is one of benefits which kernel based 
implementation gives.

> Also, it appears that your trap syntax only allows you to enable
> booleans upon events, whereas I could envision one wanting to disable
> booleans as well.  Otherwise, people have to contort their boolean
> definitions to correspond with this limitation.

I should have considered impacts on boolean definition. TRAP notation, 
as well as data structure, will be modified in next release, such like 
this,
example) 
   trap <subject> <object>:<class> <permissions> <bools_operations> ;

> A few quick comments based on a cursory read of the kernel patch:
> 
> - A transient memory failure could cause the trap information to be
> dropped from an AVC node in avc_node_populate(), with not even a log
> message warning about it - thus you could fail to properly trap an
> access under memory pressure.  In contrast, allocation failure for the
> node itself by avc_insert() does no harm - the existing
> avc_has_perm_noaudit() code just falls back to using the 
> temporary entry
> on the stack that was filled by security_compute_av() to finish the
> checking.

Current implementation resulted from booleans management data structure 
used by trap. It may be changed in next release and also some codes will 
be added to handle memory allocation failures properly.

> - avc_has_perm() cannot invoke any potentially blocking (sleeping)
> operations, because it can be called from irq or while locks 
> are held by
> the kernel.  Hence, your code (e.g. sel_set_booles_by_trap) likewise
> cannot invoke any potentially blocking operations, like down.  Of
> course, taking that particular semaphore and using the shared
> bool_pending_values is suspect anyway, and you already have a 
> kludge to
> avoid double locking in the load_policy case (and the setbool 
> case would
> also be a problem for you).  Why not just create and use your own
> temporary value array so that you don't have any locking issue there?

I checked out the issue you pointed out. load_policy problem seems 
resolved by changing blocking operation.

> - avc_detect_perm_noaudit() calls security_compute_av() and 
> avc_insert()
> after updating the booleans.  The boolean update will have flushed the
> cache entirely (which is necessary, as the booleans may have affected
> multiple entries), so you might as well just wait and let the cache
> re-fill in the usual manner upon subsequent permission 
> checks.  It also
> doesn't make sense for it to change avd because it is only used by
> avc_audit(), which has already been called, and you don't 
> want to adjust
> that result for the new booleans (because avc_has_perm made its
> decisions based on the old boolean settings).

I misread kernel codes related to avd handling. Of-course, it's 
unnecessary operation. Thanks for pointing out!

> Note that self:process execmem makes a good candidate for a trap, as
> that reflects an attempt to make anonymous memory executable, e.g. an
> attempt to make the stack executable.

I agree that execmem is more robust/generic target to detect suspicious 
operations. It's more difficult to choose appropriate access context to 
trap than configuring policy.


Currently, I have a plan for integrating IDS functionality with MLS 
instead of boolean. This implementation will make trap description more 
flexible and stateful. The following usage is intended.

example) Progressive access policy
By means of detecting symbolic access context which indicates transition 
to the next processing phase, the privileges of running daemon can be 
changed according to its progress. 

It provides similar functionality to domain trasition. But no execve() 
is needed. I think it's more secure way because the daemon's privileges 
can be configured such that the process is only allowed the privileges 
which are needed exactly at that point.

How about this plan?

Regards,

-----
NTT DATA CORPORATION
Open Source Software Development Center
Takashi Horie
E-Mail: horie-t@users.sourceforge.net
E-Mail: horietk@nttdata.co.jp

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

RE: [RFC]Kernel Built-in IDS extension

[Stephen Smalley]

On Tue, 2005-06-21 at 19:24 +0900, horietk@nttdata.co.jp wrote:
> Certainly, I agree that the logging daemon based implementation is much 
> simpler than kernel based one. But I think syslogd may not be reliable 
> enough for this kind of purpose, i.e. printk() loses some of messages 
> under very high load. So this is one of benefits which kernel based 
> implementation gives.

The kernel audit framework allows redirection of audit messages to a
separate audit daemon, with the communication handled via netlink.  In
FC4, this is now the default; there has been a lot of work on the kernel
audit framework and auditd for FC4.

> Currently, I have a plan for integrating IDS functionality with MLS 
> instead of boolean. This implementation will make trap description more 
> flexible and stateful. The following usage is intended.
> 
> example) Progressive access policy
> By means of detecting symbolic access context which indicates transition 
> to the next processing phase, the privileges of running daemon can be 
> changed according to its progress. 
> 
> It provides similar functionality to domain trasition. But no execve() 
> is needed. I think it's more secure way because the daemon's privileges 
> can be configured such that the process is only allowed the privileges 
> which are needed exactly at that point.
> 
> How about this plan?

I'm not sure I follow your description.  Items to note:

- SELinux includes a setcon(3) operation since 2.6.11 (introduced by
Trusted Computer Solutions), so programs can be modified to change
contexts (if allowed by policy) during their operation for such
privilege bracketing.  But such privilege bracketing within the program
is not as good as decomposing the program into multiple programs and
performing exec-based transitions, because it requires that the entire
application be trusted to maintain any desired separation between the
old and new contexts and the binding between the code and the set of
privileges at any given point is weak and unreliable.

- Dynamically changing the process context in the kernel without an
explicit setcon(3) call seems even worse, because at least with an
explicit setcon(3) call, the program had to be aware of the transition
and would know to adjust its state prior to switching contexts to
maintain any desired separation.

I'm not sure what you mean by integrating it with MLS rather than
booleans; the booleans seem like a more natural fit and allow the policy
writer to define exactly how he wants the policy rules to evolve.

But again, it isn't clear that it is justified to provide this
functionality in the kernel, especially given the use of auditd these
days for SELinux auditing.  It would seem to make more sense to extend
auditd to trap specific SELinux audit messages (among other things) and
possibly run a helper to adjust boolean settings to tighten or deny
access for the process that is exhibiting questionable behavior without
interfering with normal operation by legitimate users.
 
-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.