[kernel-hardening] overview of PaX features

[kernel-hardening] overview of PaX features

[Vasiliy Kulikov]

Hi,

This is a quick overview of PaX features from the upstream inclusion
point of view.


PaXTeam said there is almost nothing I can do with userspace NX.
Currently it is implemented as a "stack can be X unless GNU_STACK is
enabled" instead of secure by default policy.  It's impossible to fix it
not breaking the compatibility with old apps.  So, I'm skipping NOEXEC
options.


PAX_MPROTECT

As said in pax documentation [4]:

   The restrictions prevent
   - creating executable anonymous mappings
   - creating executable/writable file mappings
   - making an executable/read-only file mapping writable except for
     performing relocations on an ET_DYN ELF file (non-PIC shared library)
   - making a non-executable mapping executable

In Pax' terminology, it protects against one type of malforming code
execution flaw: introducing new code and executing it without
interaction with external files, (1) part 2 in [5].

Assuming no new executable vm mappings can be added (e.g.  via mmaping
a file with precreated shellcode), there is no WX areas, and no writable
areas can be turned into executable, ret2libc is the only attacker's
choise.

It definitely makes things harder for the attacker if the process is
seccomp'ed.  If execve()/fork() cannot be done (which is the seccomp's case)
arbitrary code execution becomes impossible, leaving the attacker only
using the already existing code.

I think it can be done via kernel.mprotect_rigid per pid namespace
(different containers might have different policies or one may have such
dummy app that doesn't work with PAX_MPROTECT).


PAX_KERNEXEC

As PaXTeam said, there are problems with DEBUG_RODATA in the upstream
kernel, not everything that can be made RO is actually made RO [1].

Constification - a significant portion of constification patches is not
yet applied.  One should patiently try to push it, listen to
maintainers' complains, re-divide the patch, and so forth ;-)

Verify what's done for NX, what's missing, what's wrong with large page
mappings.

Identify what structs may be RO or be RO for almost all the time (it is
implemented as pax_open()/pax_close() in PaX).


PAX_ASLR

Dan Rosenberg has started to work on it [0], I see no reason for
duplication the work (at least now).


PAX_RANDKSTACK

It randomizes the kernel stack by subtracting random value 0x0-0x100
from SP gotten from rdtsc.  Technically it is simple, but there are 2 issues:

1) it might slowdown the border case (very fast syscalls).

2) it makes kernel stack overflow situation worse, at least potentially.

However, it worth trying to propose.


PAX_RANDUSTACK
PAX_RANDMMAP

Looks like these 2 are actually implemented in the upstream kernel.


PAX_MEMORY_SANITIZE

It zeroes a memory page when it is freed.  IMO if it is worth including,
it also should be expanded by kernel caches zeroing.  Some sensitive
information might be stored not only in buffer/cache pages, but also in
kmalloc'ed areas or cryptography related structs like in
security/keys/trusted.h.  As it has some performance drawbacks, it
should be done sysctl'able (but not per namespace, sinse it is global to
the kernel).


PAX_MEMORY_STACKLEAK

Zeroes kernel stack memory on every syscall.

Good feature, but (a) it is very young (it might mature and have even
smaller performance penalty) and (b) has rather big performance penalty
in the border case anyway.


PAX_MEMORY_UDEREF

Unlikely to be applied because it rather significantly interferes in the
very core architecture code.  However, the feature is very interesting.


PAX_REFCOUNT

Technically it is very simple - if the resulted refcount value is too
large (2 times less than the overflow value) then redo atomic_inc/dec
and exec "int 4h".  It adds 3 asm commands to all atomic operations, it
shouldn't be performance visible.  I'd only change "int 4h" to
pr_emerg(), SIGKILL and maybe oops.

On the other hand atomic_t is used not only as a reference counter, but
also as a statistics storage, and even a bitmask [2].  Obviously, panic on
statistics counter overflow doesn't makes sense.  More correct naming
scheme would use refcount_t that checks overflows, but use atomic_t as
unmanaged type, and changing already implemented atomic_t refcounter to
refcount_t.  It is complicated by the fact that atomic_t usage as a
reference counter 99,9% all over the code with very small number other
applications.

So, if implement separate refcount_t and sequentially adapt atomic_t to
refcount_t, it would take too much time ("find -name '*.c' -o -name
'*.h' | wc" shows 2.5K lines including comments) and might end like
constification patch series, but it will be transparent to developers
and shouldn't cause much protest.  If change existing atomic_t
implementation, it should be done in one step not to break existing
statistics, bitmasks, etc., it would greatly annoy maintainers, but if
succeed would be perfect ;-)

As it is arch-dependent I'll implement it for ia32/amd64, leaving other
architectures with unmanaged atomic_t.


PAX_USERCOPY

This is a great overflow protection, it checks objects' size when the
size is checkable:

1) if it is on the stack, check whether it is fully packed in a single
stack frame.

2) if it is on the heap, check whether it is fully packed in an
allocated area.

If upstream is worried by the slowdown, it can be made sysctl'able or
CONFIG'urable.


In summa, I'd expect to try to push MPROTECT, review/fix KERNEXEC,
REFCOUNT, USERCOPY, KSTACK and maybe MEMORY_SANITIZE if there is some
time for it.  I'm afraid about one fundamental thing - upstream is
unlikely to favour anti-exploitation security features if they have some
significant slowdown.


[0] - http://lkml.org/lkml/2011/5/24/520
[1] - http://www.grsecurity.net/~paxguy1/kmaps.c
[2] - http://forums.grsecurity.net/viewtopic.php?f=1&t=2639
[3] - https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening
[4] - http://pax.grsecurity.net/docs/mprotect.txt
[5] - http://pax.grsecurity.net/docs/pax.txt

-- 
Vasiliy

Re: [kernel-hardening] overview of PaX features

[Solar Designer]

Vasiliy,

On Sun, Jun 26, 2011 at 10:33:21PM +0400, Vasiliy Kulikov wrote:
> PaXTeam said there is almost nothing I can do with userspace NX.
> Currently it is implemented as a "stack can be X unless GNU_STACK is
> enabled" instead of secure by default policy.  It's impossible to fix it
> not breaking the compatibility with old apps.  So, I'm skipping NOEXEC
> options.

Please do break compatibility with a handful of old apps - but as a
non-default option (which we're likely to make the default on Owl, just
like we had non-executable stack by default with 2.4.x-ow kernels).

With OpenVZ, it needs to be configurable per-container.

This is an important security hardening feature for us to have, so
please proceed to implement it sooner rather than later.

I'm sorry that I haven't reviewed the rest of your message yet (and am
in fact a bit late with this comment as well...)

Thanks,

Alexander

Re: [kernel-hardening] overview of PaX features

[Vasiliy Kulikov]

Solar,

On Wed, Jun 29, 2011 at 22:25 +0400, Solar Designer wrote:
> On Sun, Jun 26, 2011 at 10:33:21PM +0400, Vasiliy Kulikov wrote:
> > PaXTeam said there is almost nothing I can do with userspace NX.
> > Currently it is implemented as a "stack can be X unless GNU_STACK is
> > enabled" instead of secure by default policy.  It's impossible to fix it
> > not breaking the compatibility with old apps.  So, I'm skipping NOEXEC
> > options.
> 
> Please do break compatibility with a handful of old apps

That's not only about old apps, but also a default relaxed policy for
the toolchain:

http://www.gentoo.org/proj/en/hardened/gnu-stack.xml

> - but as a
> non-default option (which we're likely to make the default on Owl, just
> like we had non-executable stack by default with 2.4.x-ow kernels).
> 
> With OpenVZ, it needs to be configurable per-container.

For upstream linux the default policy is if no GNU_STACK present, the
stack flags is defined by a constant.  I think it makes sense for
the upsteam to change it to per pid namespace, with the same default.

A good idea!

Thanks,

-- 
Vasiliy

Re: [kernel-hardening] overview of PaX features

[Solar Designer]

Vasiliy,

On Wed, Jun 29, 2011 at 10:37:28PM +0400, Vasiliy Kulikov wrote:
> That's not only about old apps, but also a default relaxed policy for
> the toolchain:
> 
> http://www.gentoo.org/proj/en/hardened/gnu-stack.xml

Of course.  In my experience, most programs that currently get
executable stack actually don't need it.

And for gcc trampolines we can include the emulation code in the kernel.

> For upstream linux the default policy is if no GNU_STACK present, the
> stack flags is defined by a constant.  I think it makes sense for
> the upsteam to change it to per pid namespace, with the same default.

Sounds good.  Then we'll have less code to maintain in our patch.

Thanks,

Alexander

Re: [kernel-hardening] overview of PaX features

[Vasiliy Kulikov]

Solar,

On Wed, Jun 29, 2011 at 23:43 +0400, Solar Designer wrote:
> On Wed, Jun 29, 2011 at 10:37:28PM +0400, Vasiliy Kulikov wrote:
> > That's not only about old apps, but also a default relaxed policy for
> > the toolchain:
> > 
> > http://www.gentoo.org/proj/en/hardened/gnu-stack.xml
> 
> Of course.  In my experience, most programs that currently get
> executable stack actually don't need it.
> 
> And for gcc trampolines we can include the emulation code in the kernel.

I've looked over -ow and PaX' implementations of trampolines emulation.
Two notes:

1) Are trampolines the only widespread user of executable stack?
(widespread among executable stack needings ;)

2) In -ow patch the trampolines emulation is very tolerant: it supports
up to 8 movs and then one of 2 jmps.  PaX' version distinguishes only 2
specific trampolines implementations and alerts if the code doesn't fit
into these strict patterns.  Taking into consideration how long PaX
patch exists, I suppose the restricted version cover all (or almost all)
realworld trampolines implementations.   The -ow variant would relax the
stack too much.


Btw, there is a tool to change executable stack settings per binary,
written by Jakub Jelinek (Red Hat):

http://linux.die.net/man/8/execstack


Thanks,

-- 
Vasiliy

Re: [kernel-hardening] overview of PaX features

[Solar Designer]

On Thu, Jun 30, 2011 at 08:03:45PM +0400, Vasiliy Kulikov wrote:
> I've looked over -ow and PaX' implementations of trampolines emulation.
> Two notes:
> 
> 1) Are trampolines the only widespread user of executable stack?
> (widespread among executable stack needings ;)

Apps/libs relying on executable stack are very uncommon these days, even
more so than they were in 1990s when I started playing with this.  Of
those few that do exist, I am not sure which are "less uncommon".  There
was a time when glibc's localedef program had a nested function and thus
required either executable stack or gcc trampoline emulation.  Moreover,
there was a time when glibc's dynamic linker required executable stack.
But it was a decade ago (IIRC, glibc 2.1 and glibc 2.0, respectively).
Since then, I am not aware of widespread users of executable stack,
because of trampolines or otherwise.

Oh, of course the kernel itself also put a signal handler return
trampoline on the stack.  However, glibc started installing its own code
for that, not on the stack.  Also, -ow patch included a fault-based
signal return mechanism, which worked even with libc5.  I don't recall
what happened to it in exec-shield.

> 2) In -ow patch the trampolines emulation is very tolerant: it supports
> up to 8 movs and then one of 2 jmps.  PaX' version distinguishes only 2
> specific trampolines implementations and alerts if the code doesn't fit
> into these strict patterns.  Taking into consideration how long PaX
> patch exists, I suppose the restricted version cover all (or almost all)
> realworld trampolines implementations.   The -ow variant would relax the
> stack too much.

I recall fixing limitations and bugs in -ow's trampoline emulation in
response to actual problem reports with different versions of gcc, so I
think there are/were more than two kinds of gcc trampolines in
existence.  Maybe there are only two or even only one widespread version
of the trampoline code, though.

You may want to check the code in linux-2.2.12-ow6.diff.  It turned out
to be insufficient to cover some newer gcc versions, so it was enhanced
in later 2.2.x-ow versions.

http://download.openwall.net/pub/patches/linux/v2.2/historical/

That said, I don't have strong feelings one way or the other.  Feel free
to use the stricter code from PaX if you like.  You can also ask for PaX
Team's advice on this.

> Btw, there is a tool to change executable stack settings per binary,
> written by Jakub Jelinek (Red Hat):
> 
> http://linux.die.net/man/8/execstack

I think it makes sense for us to get it into Owl.

Thanks,

Alexander

Re: [kernel-hardening] overview of PaX features

[Vasiliy Kulikov]

Solar,

On Sat, Jul 02, 2011 at 21:21 +0400, Solar Designer wrote:
> Oh, of course the kernel itself also put a signal handler return
> trampoline on the stack.

As the kernel actually use NX for the stack on amd64 and on x86-32 with
PAE support, the signal handler is already rewritten to respect
the nonexecutable stack.


> You may want to check the code in linux-2.2.12-ow6.diff.  It turned out
> to be insufficient to cover some newer gcc versions, so it was enhanced
> in later 2.2.x-ow versions.
> 
> http://download.openwall.net/pub/patches/linux/v2.2/historical/

I'll take a look at it, thanks.


> That said, I don't have strong feelings one way or the other.  Feel free
> to use the stricter code from PaX if you like.  You can also ask for PaX
> Team's advice on this.

He told me that the PaX' version is based on the actual gcc code, so it
should be sufficient ;)


> > Btw, there is a tool to change executable stack settings per binary,
> > written by Jakub Jelinek (Red Hat):
> > 
> > http://linux.die.net/man/8/execstack
> 
> I think it makes sense for us to get it into Owl.

Also there is a paxtest utility, it shows some information related to
noexec, ASLR and NULL presence in some libc functions:

http://grsecurity.net/~spender/paxtest-0.9.9.tgz


Anyway, I expect to work on this patch just after PAX_USERCOPY
discussion with upstream (and trying to push it, of course!).

Thanks,

-- 
Vasiliy

Re: [kernel-hardening] overview of PaX features

[Anthony G. Basile]

On 06/29/2011 03:43 PM, Solar Designer wrote:
> Vasiliy,
> 
> On Wed, Jun 29, 2011 at 10:37:28PM +0400, Vasiliy Kulikov wrote:
>> That's not only about old apps, but also a default relaxed policy for
>> the toolchain:
>>
>> http://www.gentoo.org/proj/en/hardened/gnu-stack.xml
> 
> Of course.  In my experience, most programs that currently get
> executable stack actually don't need it.
> 

Ditto.  When we couldn't fix the source code, we used to use execstack
from the prelink package to remove the X flag from PT_GNU_STACK phdrs.
Recently I wrote fix-gnustack.c to avoid all the extra prelink stuff [1]
and we're using that now.

Ref.

   [1] http://git.overlays.gentoo.org/gitweb/?p=proj/elfix.git;a=summary

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535