Re: [PATCH 2/2] x86-64: seccomp: fix 32/64 syscall hole

[daw@cs.berkeley.edu (David Wagner)]

Roland McGrath  wrote:
>> Ptrace has performance and/or reliability problems when used to
>> sandbox threaded applications due to potential race conditions when
>> inspecting system call arguments. We hope that we can avoid this
>> problem with seccomp.
>
>ptrace certainly has performance issues.  I take it the only "reliability
>problems" you are talking about are MT races with modifications to user
>memory that is relevant to a system call.  (Is there something else?)

As of 1999, I believe there were some other issues for using ptrace
securely:

1. I do not know of a good way to reliably ensure that all children of
a traced program will be traced as well.  If you wait for the fork()
call to return, check the pid, and start tracing the child process,
you are subject to race conditions.  (strace's solution is to modify
the code of the traced program to put a trapping instruction immediately
after the call site to fork().  This is a grody hack and I had a hard
time convincing myself that this will be secure in all cases.)

2. ptrace disrupts the process hierarchy and Unix signals.  Because of
the way ptrace overloads signals to deliver tracing events, tracing is
not transparent.  For instance, if the parent and child are both traced,
and the parent waits for a signal from the child, things may no longer
work the same way while being traced.  Working around this requires
non-trivial code.  Complexity is the enemy of security and makes it hard
to gain confidence this doesn't introduce subtle issues.

3. If the tracing application should happen to die unexpectedly
(OOM, anyone?), I believe the traced application continues running,
now without any security checks.

4. I seem to recall that when I looked at this in 1999, if the traced
app makes a syscall that should not be allowed, I couldn't find a good
way to prevent that syscall from executing.  I don't know if current
ptrace has solved this problem.

Disclaimer: I haven't checked whether these all still apply today.