Is kernel optimized with dead store removal?

Is kernel optimized with dead store removal?

[Roel Kluin]

According to http://cwe.mitre.org/data/slices/2000.html#14 due to optimization
A call to memset() can be removed as a dead store when the buffer is not used
after its value is overwritten. Does this optimization also occur during
compilation of the Linux kernel? Then I think I may have found some
vulnerabilities. One is sha1_update() where memset(temp, 0, sizeof(temp)); may
be removed.

Roel

Re: Is kernel optimized with dead store removal?

[Mikael Pettersson]

Roel Kluin writes:
 > According to http://cwe.mitre.org/data/slices/2000.html#14 due to optimization
 > A call to memset() can be removed as a dead store when the buffer is not used
 > after its value is overwritten. Does this optimization also occur during
 > compilation of the Linux kernel? Then I think I may have found some
 > vulnerabilities. One is sha1_update() where memset(temp, 0, sizeof(temp)); may
 > be removed.

Any such dead store removal is up to the compiler and the lifetime
of the object being clobbered. For 'auto' objects the optimization
is certainly likely.

This is only a problem if the memory (a thread stack, say) is recycled
and leaked uninitialized to user-space, but such bugs are squashed
fairly quickly upon discovery.

(checking gcc-4.4.3)
It seems that memset((volatile void*)&some_local_var, 0, sizeof(...))
just provokes a warning about the invalid type of memset()'s first
parameter, but then still optimizes the operation away.

You might need to call an out-of-line helper function for this to work.

Re: Is kernel optimized with dead store removal?

[Roel Kluin]

>  >  Does this optimization also occur during compilation of the Linux
>  >  kernel?

> Any such dead store removal is up to the compiler and the lifetime
> of the object being clobbered. For 'auto' objects the optimization
> is certainly likely.
> 
> This is only a problem if the memory (a thread stack, say) is recycled
> and leaked uninitialized to user-space, but such bugs are squashed
> fairly quickly upon discovery.

Thanks for comments,

In the sha1_update() case I don't know whether the stack is recycled and
leaked - it may be dependent on the calling function, but isn't it
vulnerable?

I tested this with the snippet below. If compiled with -O1 or -O2 and
ON_STACK defined 1, I can read "Secret" a second time. With ON_STACK
defined 0 I do not.

Roel

---
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#define ON_STACK 1

void foo()
{
        char password[] = "secret";
        password[0]='S';
        printf ("Don't show again: %s\n", password);
        memset(password, 0, sizeof(password));
}

void foo2()
{
        char* password = malloc(7);
        strncpy (password, "secret" , 7);
        password[6] = '\0';
        password[0] = 'S';
        printf ("Don't show again: %s\n", password);
        memset(password, 0, 7);
        free(password);

}

int main(int argc, char* argv[])
{

#if ON_STACK == 1
        foo();
#else
        foo2();
#endif
        int i;
        char foo3[] = "hoi";
        printf ("foo1:%s\n", foo3);
        char* bar = &foo3[0];
        for (i = -50; i < 50; i++)
                printf ("%c.", bar[i]);
        printf("\n");
        return 0;
}

Re: Is kernel optimized with dead store removal?

[Mikael Pettersson]

Roel Kluin writes:
 > 
 > >  >  Does this optimization also occur during compilation of the Linux
 > >  >  kernel?
 > 
 > > Any such dead store removal is up to the compiler and the lifetime
 > > of the object being clobbered. For 'auto' objects the optimization
 > > is certainly likely.
 > > 
 > > This is only a problem if the memory (a thread stack, say) is recycled
 > > and leaked uninitialized to user-space, but such bugs are squashed
 > > fairly quickly upon discovery.
 > 
 > Thanks for comments,
 > 
 > In the sha1_update() case I don't know whether the stack is recycled and
 > leaked - it may be dependent on the calling function, but isn't it
 > vulnerable?

It's only vulnerable if the data leaks to a less trusted domain.
There is no domain crossing in your user-space example.
In the kernel case, the domain crossing would be as I wrote: leaking
recycled and uninitialized memory to user-space (and those leaks of
uninitialized memory are frowned upon and quickly fixed when discovered).

Re: Is kernel optimized with dead store removal?

[Stefan Richter]

Mikael Pettersson wrote:
> It's only vulnerable if the data leaks to a less trusted domain.
> There is no domain crossing in your user-space example.
> In the kernel case, the domain crossing would be as I wrote: leaking
> recycled and uninitialized memory to user-space (and those leaks of
> uninitialized memory are frowned upon and quickly fixed when discovered).

Quickly fixed when discovered, but not necessary quickly discovered...
-- 
Stefan Richter
-=====-==-=- --=- ==--=
http://arcgraph.de/sr/

Re: Is kernel optimized with dead store removal?

[Henrique de Moraes Holschuh]

On Thu, 25 Feb 2010, Mikael Pettersson wrote:
>  > In the sha1_update() case I don't know whether the stack is recycled and
>  > leaked - it may be dependent on the calling function, but isn't it
>  > vulnerable?
> 
> It's only vulnerable if the data leaks to a less trusted domain.

If it is anything you wanted to protect badly enough that you already have
code to clobber it later, this *is* a security bug.

Not only you remove one layer of security, you also widen a lot the window
of opportunity to, e.g., http://en.wikipedia.org/wiki/Cold_boot_attack

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh