mprotect() failed: Cannot allocate memory

mprotect() failed: Cannot allocate memory

[Paweł Sikora]

hi,

i'm trying to debug an ugly application with ElectricFence.
in fact, on x86-64 box with 8GB ram and 16GB swap i'm getting following error:

"ElectricFence Exiting: mprotect() failed: Cannot allocate memory"

the program has been compiled with gcc-4.5, glibc-2.11.1, kernel-2.6.32.
did you ever come across such (kernel/glibc) limitations?

here's a simple testcase which triggs -ENOMEM in mprotect().

#define N 100
#include<stdlib.h>

double **bm;
int main(){
    int i;
    long NN = 4*N*N;
    int kmax=100;

        bm = (double **)malloc((time_t)NN*sizeof(double *));
        for(i=0; i<NN; ++i){
                bm[i] = (double*)malloc((time_t)kmax*sizeof(double));
        }

        for(i=0; i<NN; ++i){
                free(bm[i]);
        }
        free(bm);
        return 0;
}

thanks for any hint,

BR,
Pawel.

Re: mprotect() failed: Cannot allocate memory

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 386 bytes --]

On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:
> i'm trying to debug an ugly application with ElectricFence.

electricfence does a lot of ugly memory tricks to do its thing, including, but 
not limited to, overriding memory related symbols.  best to seek help from the 
electricfence authors.

unless of course, things still fail when you dont use electricfence.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: mprotect() failed: Cannot allocate memory

[Paweł Sikora]

On Wednesday 21 April 2010 01:17:22 Mike Frysinger wrote:
> On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:
> > i'm trying to debug an ugly application with ElectricFence.
> 
> electricfence does a lot of ugly memory tricks to do its thing, including,
> but not limited to, overriding memory related symbols.  best to seek help
> from the electricfence authors.

so, let's avoid EF and run following test:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

void* my_alloc( size_t n )
{
        size_t ps = getpagesize();
        printf( "request for %Zd bytes => ", n );
        /* alloc PAGE_SIZE + n */
        char* p = mmap( 0, ps + n, PROT_READ | PROT_WRITE, MAP_SHARED | 
MAP_ANONYMOUS, -1, 0 );
        if ( p == MAP_FAILED )
                __builtin_abort();
        /* block guard page */
        int rc = mprotect( p, ps, PROT_NONE );
        if ( rc != 0 )
                __builtin_abort();
        char* q = p + ps;
        printf( "guard page @ %p, allocated region @ %p\n", p, q );
        return q;
}

int main()
{
        #define N 100  
        size_t NN = 4*100*100;
        size_t kmax = 100;
        int i;

        double **bm = (double **)my_alloc( NN * sizeof( double* ) );
        for( i = 0; i < NN; ++i )
        {
                bm[ i ] = (double*)my_alloc( kmax * sizeof( double ) );
        }
        // leak...
        return 0;
}

and the result is...

(...)
mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 
0x7f5fd97df000
mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate memory)

Re: mprotect() failed: Cannot allocate memory

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 557 bytes --]

On Tuesday 20 April 2010 19:44:18 Paweł Sikora wrote:
> so, let's avoid EF and run following test:

good, the info is useful now

> mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) =
> 0x7f5fd97df000
> mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate
> memory)

since you're using funcs that go straight to the kernel, this most likely isnt 
related to glibc at all, so sticking to LKML would be best.  the next step 
would be to look at the linux internals and find out why this is failing.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: mprotect() failed: Cannot allocate memory

[Paweł Sikora]

Dnia 21-04-2010 o 11:22:19 Yann Droneaud <yann@droneaud.fr> napisał(a):

> Le mercredi 21 avril 2010 à 01:44 +0200, Paweł Sikora a écrit :
>> On Wednesday 21 April 2010 01:17:22 Mike Frysinger wrote:
>> > On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:
>> > > i'm trying to debug an ugly application with ElectricFence.
>> >
>> > electricfence does a lot of ugly memory tricks to do its thing,  
>> including,
>> > but not limited to, overriding memory related symbols.  best to seek  
>> help
>> > from the electricfence authors.
>>
>> so, let's avoid EF and run following test:
>>
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <unistd.h>
>> #include <sys/mman.h>
>>
>> void* my_alloc( size_t n )
>> {
>>         size_t ps = getpagesize();
>>         printf( "request for %Zd bytes => ", n );
>>         /* alloc PAGE_SIZE + n */
>>         char* p = mmap( 0, ps + n, PROT_READ | PROT_WRITE, MAP_SHARED |
>> MAP_ANONYMOUS, -1, 0 );
>>         if ( p == MAP_FAILED )
>>                 __builtin_abort();
>>         /* block guard page */
>>         int rc = mprotect( p, ps, PROT_NONE );
>>         if ( rc != 0 )
>>                 __builtin_abort();
>>         char* q = p + ps;
>>         printf( "guard page @ %p, allocated region @ %p\n", p, q );
>>         return q;
>> }
>>
>> int main()
>> {
>>         #define N 100
>>         size_t NN = 4*100*100;
>>         size_t kmax = 100;
>>         int i;
>>
>>         double **bm = (double **)my_alloc( NN * sizeof( double* ) );
>>         for( i = 0; i < NN; ++i )
>>         {
>>                 bm[ i ] = (double*)my_alloc( kmax * sizeof( double ) );
>>         }
>>         // leak...
>>         return 0;
>> }
>>
>> and the result is...
>>
>> (...)
>> mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0)  
>> =
>> 0x7f5fd97df000
>> mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate  
>> memory)
>
> Have you checked available memory on your system ? Or user limit ?
>
> You test program is going to allocate
>  79 + 1 pages for bm
>  1 + 1 for each double arrays (x 40000)
>
> So in the end your program is allocating 80080 pages, so about
> 312MBytes.
>
> It not that big for a 64bits system.
>
> Check limits such as
> -d	the maximum size of a process's data segment
> -l	the maximum size a process may lock into memory
> -m	the maximum resident set size

$ ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     unlimited
-s: stack size (kbytes)        8192
-c: core file size (blocks)    0
-m: resident set size (kbytes) unlimited
-u: processes                  unlimited
-n: file descriptors           1024
-l: locked-in-memory size (kb) 64
-v: address space (kb)         unlimited
-x: file locks                 unlimited
-i: pending signals            64024
-q: bytes in POSIX msg queues  819200
-e: max nice                   0
-r: max rt priority            0

$ free -m
              total       used       free     shared    buffers     cached
Mem:          8004       1779       6224          0         36       1228
-/+ buffers/cache:        514       7489
Swap:        15625          0      15625


imho the testcase has enough hardware resources.

Re: mprotect() failed: Cannot allocate memory

[Paweł Sikora]

Dnia 21-04-2010 o 11:22:19 Yann Droneaud <yann@droneaud.fr> napisał(a):

>> (...)
>> mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0)  
>> =
>> 0x7f5fd97df000
>> mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate  
>> memory)
>
> Have you checked available memory on your system ? Or user limit ?
>
> You test program is going to allocate
>  79 + 1 pages for bm
>  1 + 1 for each double arrays (x 40000)
>
> So in the end your program is allocating 80080 pages, so about
> 312MBytes.
>
> It not that big for a 64bits system.

afaics in gdb, the mprotect fails at i=32756. it's near to 2^15.
maybe some kernel data structures are full?

Re: mprotect() failed: Cannot allocate memory

[Yann Droneaud]

Le mercredi 21 avril 2010 à 01:44 +0200, Paweł Sikora a écrit :
> On Wednesday 21 April 2010 01:17:22 Mike Frysinger wrote:
> > On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:
> > > i'm trying to debug an ugly application with ElectricFence.
> > 
> > electricfence does a lot of ugly memory tricks to do its thing, including,
> > but not limited to, overriding memory related symbols.  best to seek help
> > from the electricfence authors.
> 
> so, let's avoid EF and run following test:
> 
> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>
> #include <sys/mman.h>
> 
> void* my_alloc( size_t n )
> {
>         size_t ps = getpagesize();
>         printf( "request for %Zd bytes => ", n );
>         /* alloc PAGE_SIZE + n */
>         char* p = mmap( 0, ps + n, PROT_READ | PROT_WRITE, MAP_SHARED | 
> MAP_ANONYMOUS, -1, 0 );
>         if ( p == MAP_FAILED )
>                 __builtin_abort();
>         /* block guard page */
>         int rc = mprotect( p, ps, PROT_NONE );
>         if ( rc != 0 )
>                 __builtin_abort();
>         char* q = p + ps;
>         printf( "guard page @ %p, allocated region @ %p\n", p, q );
>         return q;
> }
> 
> int main()
> {
>         #define N 100  
>         size_t NN = 4*100*100;
>         size_t kmax = 100;
>         int i;
> 
>         double **bm = (double **)my_alloc( NN * sizeof( double* ) );
>         for( i = 0; i < NN; ++i )
>         {
>                 bm[ i ] = (double*)my_alloc( kmax * sizeof( double ) );
>         }
>         // leak...
>         return 0;
> }
> 
> and the result is...
> 
> (...)
> mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 
> 0x7f5fd97df000
> mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate memory)

Have you checked available memory on your system ? Or user limit ?

You test program is going to allocate 
 79 + 1 pages for bm
 1 + 1 for each double arrays (x 40000)

So in the end your program is allocating 80080 pages, so about
312MBytes.

It not that big for a 64bits system.

Check limits such as 
-d	the maximum size of a process's data segment
-l	the maximum size a process may lock into memory
-m	the maximum resident set size

Regards.

-- 
Yann Droneaud

Re: mprotect() failed: Cannot allocate memory

[Paweł Sikora]

Dnia 21-04-2010 o 13:16:50 Peter Zijlstra <peterz@infradead.org>  
napisał(a):

> On Wed, 2010-04-21 at 01:05 +0200, Paweł Sikora wrote:
>> hi,
>>
>> i'm trying to debug an ugly application with ElectricFence.
>> in fact, on x86-64 box with 8GB ram and 16GB swap i'm getting following  
>> error:
>>
>> "ElectricFence Exiting: mprotect() failed: Cannot allocate memory"
>>
>> the program has been compiled with gcc-4.5, glibc-2.11.1, kernel-2.6.32.
>> did you ever come across such (kernel/glibc) limitations?
>>
>> here's a simple testcase which triggs -ENOMEM in mprotect().
>
> You probably depleted the max map count, see:
>   /proc/sys/vm/max_map_count

yes, that is the clue :)

the limit in /proc/sys/vm/max_map_count was set to 65530.
with `echo 128000 > /proc/sys/vm/max_map_count` the testcase passes.

thanks for hint.

Re: mprotect() failed: Cannot allocate memory

[Peter Zijlstra]

On Wed, 2010-04-21 at 01:05 +0200, Paweł Sikora wrote:
> hi,
> 
> i'm trying to debug an ugly application with ElectricFence.
> in fact, on x86-64 box with 8GB ram and 16GB swap i'm getting following error:
> 
> "ElectricFence Exiting: mprotect() failed: Cannot allocate memory"
> 
> the program has been compiled with gcc-4.5, glibc-2.11.1, kernel-2.6.32.
> did you ever come across such (kernel/glibc) limitations?
> 
> here's a simple testcase which triggs -ENOMEM in mprotect().

You probably depleted the max map count, see:
  /proc/sys/vm/max_map_count

We have a limit on the number of maps you can have, those mprotect()
calls split you maps like crazy, see also /proc/$pid/maps.

eg. change your second test program to include something like:

  char buf[128];
  snprintf(buf, sizeof(buf), "cat /proc/%d/maps", (int)getpid());
  system(buf);

at the end after lowering your NN count to fit, and observe the result
of those mprotect() calls.