[kernel-hardening] Container Hardening

[kernel-hardening] Container Hardening

[Jessica Frazelle]

Hi,

I made this one page site[1] to detail trying to harden namespaces in
the kernel. The other primitives containers use are included as well,
but if we are honest we all know namespaces need the most help.

Solar mentioned just using this mailing list for this initiative as
well. That's great with me because I would love all your feedback and
help.

I think the first focus should be on preventing priviledge escalations
in user namespaces. This has the largest attack surface. The
fundamental problem seems to be that not many people have inspected
user namespaces and the various interactions with other parts of the
kernel. I will be trying to do this and would love any help from
anyone interested. We could split up the various systems and do some
research to find out just how far this rabbit hole goes.

In the past, one of the ways to fix vulnerabilities with user
namespaces was to disallow the interaction, for instance CLONE_FS.

I'm sure we can't have that as a solution for everything, but I'm
hoping by working together we can come up with a well-informed
solution.

Jess

[1] https://containerhardening.org

Re: [kernel-hardening] Container Hardening

[Thomas Garnier]

That seems like a good idea!

It would be useful to gather a list of bugs that affected namespaces
or usual mistakes in using namespaces.

I will see if I can free some time to help.

On Fri, Feb 3, 2017 at 8:13 AM, Jessica Frazelle <me@jessfraz.com> wrote:
> Hi,
>
> I made this one page site[1] to detail trying to harden namespaces in
> the kernel. The other primitives containers use are included as well,
> but if we are honest we all know namespaces need the most help.
>
> Solar mentioned just using this mailing list for this initiative as
> well. That's great with me because I would love all your feedback and
> help.
>
> I think the first focus should be on preventing priviledge escalations
> in user namespaces. This has the largest attack surface. The
> fundamental problem seems to be that not many people have inspected
> user namespaces and the various interactions with other parts of the
> kernel. I will be trying to do this and would love any help from
> anyone interested. We could split up the various systems and do some
> research to find out just how far this rabbit hole goes.
>
> In the past, one of the ways to fix vulnerabilities with user
> namespaces was to disallow the interaction, for instance CLONE_FS.
>
> I'm sure we can't have that as a solution for everything, but I'm
> hoping by working together we can come up with a well-informed
> solution.
>
> Jess
>
> [1] https://containerhardening.org



-- 
Thomas

Re: [kernel-hardening] Container Hardening

[Jessica Frazelle]

Yeah I can definitely come up with a list. The interesting thing is
some vulnerabilities don't even need for the process to be _in_ a user
namespace, just that CONFIG_USERNS=y. So as far as I currently know, a
lot has to do with hitting these obscure-ish code paths. But will work
on a list :)

On Fri, Feb 3, 2017 at 8:54 AM, Thomas Garnier <thgarnie@google.com> wrote:
> That seems like a good idea!
>
> It would be useful to gather a list of bugs that affected namespaces
> or usual mistakes in using namespaces.
>
> I will see if I can free some time to help.
>
> On Fri, Feb 3, 2017 at 8:13 AM, Jessica Frazelle <me@jessfraz.com> wrote:
>> Hi,
>>
>> I made this one page site[1] to detail trying to harden namespaces in
>> the kernel. The other primitives containers use are included as well,
>> but if we are honest we all know namespaces need the most help.
>>
>> Solar mentioned just using this mailing list for this initiative as
>> well. That's great with me because I would love all your feedback and
>> help.
>>
>> I think the first focus should be on preventing priviledge escalations
>> in user namespaces. This has the largest attack surface. The
>> fundamental problem seems to be that not many people have inspected
>> user namespaces and the various interactions with other parts of the
>> kernel. I will be trying to do this and would love any help from
>> anyone interested. We could split up the various systems and do some
>> research to find out just how far this rabbit hole goes.
>>
>> In the past, one of the ways to fix vulnerabilities with user
>> namespaces was to disallow the interaction, for instance CLONE_FS.
>>
>> I'm sure we can't have that as a solution for everything, but I'm
>> hoping by working together we can come up with a well-informed
>> solution.
>>
>> Jess
>>
>> [1] https://containerhardening.org
>
>
>
> --
> Thomas



-- 


Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC  511E 18F3 685C 0022 BFF3
pgp.mit.edu

Re: [kernel-hardening] Container Hardening

[Eric W. Biederman]

Jessica Frazelle <me@jessfraz.com> writes:

> Yeah I can definitely come up with a list. The interesting thing is
> some vulnerabilities don't even need for the process to be _in_ a user
> namespace, just that CONFIG_USERNS=y. So as far as I currently know, a
> lot has to do with hitting these obscure-ish code paths. But will work
> on a list :)

I believe you are a little misinformed about the current situation,
but one thing I can agree with is more people and more eyeballs on the
code can not hurt.

My best estimate of where things are at is at this point most of the
design issues have been fixed, and that user namespaces and namespaces
in general are about as buggy as the rest of the kernel.

As any process can create a user namespace a system does not have to be
using user namespaces to be vulnerable to their issues.  At the same
time there are a set of sysctls under /proc/sys/user/ that can be used
to reduce the attack surface if you are not using the features.

I will be happy to help resolve and merge any bugs you happen to find.

Although if they are ordinary kernel bugs in the network stack it is
probably easiest just to go through David Miller, and the netdev mailing
list.  I won't mind being Cc'd in that case.

Eric

Re: [kernel-hardening] Container Hardening

[Jessica Frazelle]

Thank you for your help.

On Fri, Feb 3, 2017 at 11:25 AM Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> Jessica Frazelle <me@jessfraz.com> writes:
>
> > Yeah I can definitely come up with a list. The interesting thing is
> > some vulnerabilities don't even need for the process to be _in_ a user
> > namespace, just that CONFIG_USERNS=y. So as far as I currently know, a
> > lot has to do with hitting these obscure-ish code paths. But will work
> > on a list :)
>
> I believe you are a little misinformed about the current situation,
> but one thing I can agree with is more people and more eyeballs on the
> code can not hurt.
>
> My best estimate of where things are at is at this point most of the
> design issues have been fixed, and that user namespaces and namespaces
> in general are about as buggy as the rest of the kernel.
>
> As any process can create a user namespace a system does not have to be
> using user namespaces to be vulnerable to their issues.  At the same
> time there are a set of sysctls under /proc/sys/user/ that can be used
> to reduce the attack surface if you are not using the features.
>

This sounds neat, I will read up on it!

>
> I will be happy to help resolve and merge any bugs you happen to find.
>
> Although if they are ordinary kernel bugs in the network stack it is
> probably easiest just to go through David Miller, and the netdev mailing
> list.  I won't mind being Cc'd in that case.
>
> Eric

Re: [kernel-hardening] Container Hardening

[Vincent Batts]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

Jess,

In the vein of your proposal (https://gist.github.com/jessfraz/3a84023ff85471696ee33a20031b9e7b),
there was recently a systemtap (http://sourceware.org/systemtap/) script
written to output some of this data that is not generally accessible
from userspace.

Will Cohen was nice enough to upload this and a quick write-up on it's
usage.
https://github.com/wcohen/linux-instrumentation/blob/master/container_check.md

Where this can show when a "badcap" is encountered, or just to see the
profile of capabilities and syscalls used.

vb



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [kernel-hardening] Container Hardening

[Jessica Frazelle]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

Thanks, I'll check it out.

On Fri, Feb 3, 2017 at 12:48 PM Vincent Batts <vbatts@hashbangbash.com>
wrote:

> Jess,
>
> In the vein of your proposal (
> https://gist.github.com/jessfraz/3a84023ff85471696ee33a20031b9e7b),
> there was recently a systemtap (http://sourceware.org/systemtap/) script
> written to output some of this data that is not generally accessible
> from userspace.
>
> Will Cohen was nice enough to upload this and a quick write-up on it's
> usage.
>
> https://github.com/wcohen/linux-instrumentation/blob/master/container_check.md
>
> Where this can show when a "badcap" is encountered, or just to see the
> profile of capabilities and syscalls used.
>
> vb
>
>
>

[-- Attachment #2: Type: text/html, Size: 1679 bytes --]

Re: [kernel-hardening] Container Hardening

[Jessica Frazelle]

As far as a list of CVEs I forgot I had already made one[1]. A lot of
those are considered "non-events" due to the fact the default seccomp
profile for docker containers blocks them. It outlines as far as the
mitigation.

Another that should probably be added there, that was not blocked by
the default seccomp profile was Dirty cow.


[1] https://github.com/jessfraz/docker/blob/6837cfc13cba842186a7261aa9bbd3a8755fd11e/docs/security/non-events.md

On Fri, Feb 3, 2017 at 1:13 PM, Jessica Frazelle <me@jessfraz.com> wrote:
> Thanks, I'll check it out.
>
> On Fri, Feb 3, 2017 at 12:48 PM Vincent Batts <vbatts@hashbangbash.com>
> wrote:
>>
>> Jess,
>>
>> In the vein of your proposal
>> (https://gist.github.com/jessfraz/3a84023ff85471696ee33a20031b9e7b),
>> there was recently a systemtap (http://sourceware.org/systemtap/) script
>> written to output some of this data that is not generally accessible
>> from userspace.
>>
>> Will Cohen was nice enough to upload this and a quick write-up on it's
>> usage.
>>
>> https://github.com/wcohen/linux-instrumentation/blob/master/container_check.md
>>
>> Where this can show when a "badcap" is encountered, or just to see the
>> profile of capabilities and syscalls used.
>>
>> vb
>>
>>
>



-- 


Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC  511E 18F3 685C 0022 BFF3
pgp.mit.edu