Re: [AppArmor 00/41] AppArmor security module overview

David Wagner
2007-04-12T21:31:41Z

Pavel Machek wrote:
>You can do the same with ptrace. If that's not fast enough... improve
>ptrace?

I did my Master's thesis on a system called Janus that tried using ptrace
for this goal. The bottom line is that ptrace sucks for this purpose.
It is a kludge. It is not the right approach. I do not know of any
satisfactory way to improve ptrace for this purpose; you have to throw
away ptrace and start over.

At the time I did the work, ptrace has all sorts of serious problems.
Here are some of them. There was no way to follow fork securely.
There was no way to deny a single system call without killing the process
entirely. Performance was poor, because ptrace context-switches on every
read() and write(). Handling of signals is a mess: ptrace overloads the
signal mechanism to deliver its events, which in retrospect was a lousy
design decision it makes the tracer complex and error-prone and makes
it hard to maintain the transparency of tracing. ptrace breaks wait(),
and consequently handling wait() and other signal-related system calls
transparently and securely is ugly at best. Handling signals is probably
feasible but a total mess, and that's the last thing you want in the
security-critical part of your system. In addition, ptrace operates
at the wrong level of abstraction and forces the user-level tracer to
maintain a lot of shadow state that must be kept in sync with state held
by the kernel. That's an opportunity for security holes. Also, ptrace
has no way to force the tracee to die if the tracer unexpectedly dies,
which is risky when using ptrace for security confinement. I haven't
checked whether these problems are still present in the current
implementation of ptrace, but I'd guess that many probably still are,
because many are fundamental consequences of how ptrace works.

Before advocating ptrace for this purpose, I encourage you to study some
of the relevant literature. Start with Chapter 4 of my Master's thesis.
http://www.cs.berkeley.edu/~daw/papers/janus-masters.ps
Then, read Tal Garfinkel's paper on system call interposition.
http://www.stanford.edu/~talg/papers/traps/abstract.html
Then, read about Ostia.
http://www.stanford.edu/~talg/papers/NDSS04/abstract.html
I think these may change your mind about the suitability of ptrace for
this task.