Re: Network isolation with RLIMIT_NETWORK, cont'd.

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Eric W. Biederman]

I have added the container's list to the cc as there is some overlap.

Michael Stone <michael@laptop.org> writes:

> Dear lkml,
> 
> A few months ago [1], I asked for feedback on a new network isolation primitive
> named "RLIMIT_NETWORK" designed for use with Unix sandboxing utilities like
> Rainbow, Plash, and friends [2]. Thank you to all those CC'ed for your helpful
> early remarks.
> 
> Here is an updated patchset with responses to the following criticisms:

Overall what you have looks addhoc, and very special case which is
likely to impair maintenance in the future.

Furthermore you have not addressed the primary issue that keeps
unshare(CLONE_NEWNET) requiring root privileges.  You can in theory
confuse a suid root application and cause it to take action with it's
elevated privileges that violate the security policy.  The network
namespace has more potential to confuse existing applications than
your mechanism, but the problem seems to remain.

>   1. ptrace() 
>      
>      It was pointed out by Alan Cox, Andi Kleen, and others that processes
>      which dropped their RLIMIT_NETWORK rlimit were still able to directly
>      perform networking through a ptrace()'d victim.
> 
>      The new patchset adds an access check to __ptrace_may_access() to prevent
>      this behavior.

Solve that with an unused uid.  That ptrace_may_access check is
completely non-intuitive, and a problem if we ever remove the current
== task security module bug avoidance.

>   2. unshare(CLONE_NEWNET)
> 
>      It was pointed out by James Morris that network namespaces could be used
>      to implement behavior similar to the behavior this patchset is designed to
>      implement. To address this criticism, I added support for network
>      namespaces to my sandboxing utility (Rainbow).
> 
>      Unfortunately, I have discovered that network namespaces in their current
>      form are not appropriate for my use cases because they prevent the
>      namespace'd apps from connecting to the X server, even over plain old
>      AF_UNIX sockets.

We discussed that a while ago, and there is no fundamental reason to
disallow opening unix domain sockets from another network namespace.
The reason this has not been done, is that no one has taken a good
hard look at the packet transmit path and said there are no technical
problems for packets traversing between two network namespaces.

It is probably time to revisit that.


>      The RLIMIT_NETWORK facility I propose contains a specific exception for
>      AF_UNIX filesystem sockets since those sockets are already bound by
>      regular Unix discretionary access control.

What is more significant that unix discretionary access control is the
fact that the set of available af_unix sockets you can bind to is filtered
by the mount namespace.


With respect to the problem of handling suid root applications my long
term plan is to finish the security credentials namespace aka
unshare(NEWUSER).  Making the capabilities namespace local and
changing all uid based checks from uid1 == uid2 to (ns1, uid1) ==
(ns2, uid2).  At which point suid root applications will not be a
problem because the problem root capabilities will not be available
for them to acquire.

Eric

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Michael Stone]

Eric Biederman wrote:

> I have added the container's list to the cc as there is some overlap.

Good idea; thanks.

> Overall what you have looks ad-hoc, and very special case which is
> likely to impair maintenance in the future.

Unfortunately, these are the semantics which are necessary to make further
progress on sandboxing real Linux apps with the discretionary access control
facilities which are available today.

> You can in theory confuse a suid root application and cause it to take action
> with it's elevated privileges that violate the security policy. 

You're right, in theory. In practice, the setuid-root facility is a rather
special escape hatch which *everyone* in this field knows must be carefully
audited and maintained when building or updating trustworthy systems.

Also, in practice, I'm not expecting perfection today. Nor was I last year, nor
am I next year. What I am expecting is that the kernel will supply me (perhaps
with my assistance along the way) with the access control facilities that I
need to do my job in userland. This is one of them.

> The network namespace has more potential to confuse existing applications
> than your mechanism, but the problem seems to remain.

I'm glad to hear that you find this mechanism to be comparatively less
confusing.

>>   1. ptrace() 
>>      
>>      It was pointed out by Alan Cox, Andi Kleen, and others that processes
>>      which dropped their RLIMIT_NETWORK rlimit were still able to directly
>>      perform networking through a ptrace()'d victim.
>> 
>>      The new patchset adds an access check to __ptrace_may_access() to prevent
>>      this behavior.
> 
> Solve that with an unused uid.  

I already do, in general. (As do the other people requesting this facility.)

The reason for the __ptrace_may_access() check is that the logical way for
*application authors* whose code is *already* running in a fresh uid to further
improve system security is to separate their network I/O from their parsing
code a process boundary and to drop networking privileges in the parser.

>>   2. unshare(CLONE_NEWNET)
>> 
>>      It was pointed out by James Morris that network namespaces could be used
>>      to implement behavior similar to the behavior this patchset is designed to
>>      implement. To address this criticism, I added support for network
>>      namespaces to my sandboxing utility (Rainbow).
>> 
>>      Unfortunately, I have discovered that network namespaces in their current
>>      form are not appropriate for my use cases because they prevent the
>>      namespace'd apps from connecting to the X server, even over plain old
>>      AF_UNIX sockets.
>
>We discussed that a while ago, and there is no fundamental reason to
>disallow opening unix domain sockets from another network namespace.

I disagree. I like that the network namespaces have (fairly) clear semantics.
They are excellent semantics for some of my other use cases, like testing
networked software [1]. They're probably quite nice for full-blown
containerization. They're just not right for the kind of lightweight sandboxing
of complicated legacy apps that I'm doing.

[1]: http://dev.laptop.org/git/users/mstone/dnshash/tree/docs/unit_testing.txt

>> The RLIMIT_NETWORK facility I propose contains a specific exception for
>> AF_UNIX filesystem sockets since those sockets are already bound by
>> regular Unix discretionary access control.
> 
> What is more significant than unix discretionary access control is the
> fact that the set of available af_unix sockets you can bind to is filtered
> by the mount namespace.

Actually, the Unix DAC is far more important for my purposes. The reason is
that it's unprivileged, already understood by literally *everyone* involved in
Unix security, and it has the best tools support of any access control
mechanism.

For comparison, I do use CLONE_NEWNS mount namespaces and they've been a real
pain because

   a) unlike in Plan 9, they're privileged,

   b) they greatly complicate debugging the isolated app because you see
      different things inside and outside the namespace,

   c) there's no good way to manipulate them from userland, and

   d) they're poorly documented outside of the mount man page.

Regards,

Michael

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Mark Seaborn]

On Sun, Dec 13, 2009 at 2:21 PM, Michael Stone <michael-2X9k7bc8m7Mdnm+yROfE0A@public.gmane.org> wrote:

> For comparison, I do use CLONE_NEWNS mount namespaces and they've been a
> real
> pain because
>
>  a) unlike in Plan 9, they're privileged,
>
>  b) they greatly complicate debugging the isolated app because you see
>     different things inside and outside the namespace,
>
>  c) there's no good way to manipulate them from userland, and
>
>  d) they're poorly documented outside of the mount man page.
>

Maybe we could try to fix those problems.

The reason chroot() and clone()/CLONE_NEWNS are privileged is that they
provide a way to violate the assumptions of setuid/setgid executables.  If
we add a per-process flag that prevents a process from exec'ing setuid
executables, we could allow chroot() and CLONE_NEWNS when that flag is set.
That fixes (a).

Maybe we could fix (b) by making mount namespaces into first class objects
that can be named through a file descriptor, so that one process can
manipulate another process's namespace without itself being subject to the
namespace.

Cheers,
Mark

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Bryan Donlan]

On Thu, Dec 17, 2009 at 12:31 PM, Mark Seaborn <mrs@mythic-beasts.com> wrote:

> Maybe we could fix (b) by making mount namespaces into first class objects
> that can be named through a file descriptor, so that one process can
> manipulate another process's namespace without itself being subject to the
> namespace.

Can this be done using openat() and friends currently? It would seem
the natural way to implement this; open /proc/(pid)/root, then
openat() things from there (or even chdir to it and see the mounts
that it sees from there...)

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Bernie Innocenti]

On Thu, 2009-12-17 at 13:24 -0500, Bryan Donlan wrote:
> Can this be done using openat() and friends currently? It would seem
> the natural way to implement this; open /proc/(pid)/root, then
> openat() things from there (or even chdir to it and see the mounts
> that it sees from there...)

Yeah, but /proc/<pid>/root is just a symlink. It's correct for chroots,
but I doubt it can be meaningful for per-process namespaces.

If we were to implement Mark Seaborn's idea of naming
namespaces, /proc/<pid>/rootfd would be a file descriptor providing
access to the namespace through some fancy ioctls.

Or maybe not. Could such a file-descriptor be used as the source
argument to mount(), perhaps along with a new MS_NS flag?

Alternatively, perhaps one could come up with a userspace solution:
read /proc/<pid>/mounts and repeat all mounts, perhaps with a prefix.
The downsides are that it would require superuser privs and wouldn't
automatically stay synchronized with the real namespace.

-- 
   // Bernie Innocenti - http://codewiz.org/
 \X/  Sugar Labs       - http://sugarlabs.org/

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Bryan Donlan]

On Thu, Dec 17, 2009 at 2:35 PM, Bernie Innocenti <bernie@codewiz.org> wrote:
> On Thu, 2009-12-17 at 13:24 -0500, Bryan Donlan wrote:
>> Can this be done using openat() and friends currently? It would seem
>> the natural way to implement this; open /proc/(pid)/root, then
>> openat() things from there (or even chdir to it and see the mounts
>> that it sees from there...)
>
> Yeah, but /proc/<pid>/root is just a symlink. It's correct for chroots,
> but I doubt it can be meaningful for per-process namespaces.

The files in /proc/<pid>/fs are 'just symlinks', but opening them can
provide access to objects (eg, deleted files) not accessible through
the normal filesystem namespace. I see no reason, API-wise, why
/proc/<pid>/root couldn't be extended similarly - but I've not looked
at the namespaces implementation, so maybe there's some reason it'd be
difficult to implement...

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Bernie Innocenti]

On Thu, 2009-12-17 at 17:31 +0000, Mark Seaborn wrote:


> The reason chroot() and clone()/CLONE_NEWNS are privileged is that
> they provide a way to violate the assumptions of setuid/setgid
> executables.  If we add a per-process flag that prevents a process
> from exec'ing setuid executables, we could allow chroot() and
> CLONE_NEWNS when that flag is set.  That fixes (a).

I think this would be great.

> 
> Maybe we could fix (b) by making mount namespaces into first class
> objects that can be named through a file descriptor, so that one
> process can manipulate another process's namespace without itself
> being subject to the namespace.

I think Michael's problem with debugging is much more fundamental:
application programmers get confused when some filesystem operations
fail in the debugged process, while it works fine from the shell.

It would help if the kernel provided a way for a process to switch to
another process' namespace. Even better, it would be great if existing
namespaces could be mounted at an arbitrary position within another
namespace. Then one could use traditional shell tools to inspect it, or
even chroot into it.

</delirium>

-- 
   // Bernie Innocenti - http://codewiz.org/
 \X/  Sugar Labs       - http://sugarlabs.org/

Re: Network isolation with RLIMIT_NETWORK, cont'd.

[Andi Kleen]

> Solve that with an unused uid.  That ptrace_may_access check is
> completely non-intuitive, and a problem if we ever remove the current
> == task security module bug avoidance.

I thought he wanted to do that without suid?

If he can change uids he can as well just use full network namespaces.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.