[Xenomai-help] memset of heap crashes Xenomai-Task

[Xenomai-help] memset of heap crashes Xenomai-Task

[Roderik_Wildenburg]

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

Xenomai Version : 2.2-rc2
Skin : native
Kernel : 2.4.25
Arch.: PPC

I try to allocate as much memory as possible with the functions :
rt_heap_create and
rt_heap_alloc.
(see also "Xenomai heap services" in this mailing list; see source
attached)

When I try to use the allocated memory with memset, the Xenomai-task
crashes with a "Segmentation fault".

Is memset allowed to be used with Xenomai heaps ?
If so, could somebody try to reproduce this (preferable with a PPC), to
see whether it is a common problem or just one of my configuration .
I have the feeling it is timing dependend, as it does not crash every
time. So try to start it several times, please.


Thank you for your help 
Roderik

[-- Attachment #2: heap.c --]
[-- Type: application/octet-stream, Size: 4568 bytes --]

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <netdb.h>
#include <signal.h>
#include <getopt.h>
#include <pthread.h>
#include <malloc.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/mman.h>
#include <xenomai/native/task.h>
#include <xenomai/native/queue.h>
#include <xenomai/native/timer.h>
#include <xenomai/native/heap.h>



#define DISPLAY_TASK_PRI      10
#define DISPLAY_STACK_SIZE    8192
#define MEMORYCHUNKSIZE       16000000
#define MAXHEAPBLOCKS         20
#define DISP_PERIOD           5000000000

static RT_TASK display_task;
RT_HEAP bigheap[MAXHEAPBLOCKS];
void    *bigbuf[MAXHEAPBLOCKS];
int     bigsize[MAXHEAPBLOCKS];
int     memoryallocated;

void display(void *cookie)
{

   long memsize;
   long addsize;
   int i,ret;
   char nbuf[100];
   int err;
   RT_HEAP theap;

   err = rt_task_set_periodic(NULL, TM_NOW, rt_timer_ns2ticks(DISP_PERIOD));
   if (err)
   {
      fprintf(stderr, "Display : failed to set periodic, code %d\n", err);
      return;
   }


   /*delete left over heaps*/
   for(i=0;i<MAXHEAPBLOCKS;i++)
   {
      sprintf(nbuf,"bigheap%d",i);
      if(!rt_heap_bind(&theap,nbuf,1000000))
      {
         rt_heap_delete(&theap);
         printf("Heap %s deleted\n",nbuf);
      }
   }

   sleep(1);

   printf("Available Memory %ld (Pagesize %ld)\n",sysconf(_SC_AVPHYS_PAGES) * sysconf(_SC_PAGESIZE),sysconf(_SC_PAGESIZE));

   addsize=memsize=MEMORYCHUNKSIZE; /*max Heapsize 16MB*/
   memoryallocated=0;

   for(i=0;i<MAXHEAPBLOCKS;i++)
   {

      /*create heap*/
      sprintf(nbuf,"bigheap%d",i);
      if(rt_heap_create(&bigheap[i],nbuf,memsize,H_PRIO|H_MAPPABLE))
         break;
      printf("heap %d of size %ld created\n",i,memsize);


      /*allocate memoy*/
      if((ret=rt_heap_alloc(&bigheap[i],memsize,TM_NONBLOCK,&bigbuf[i])))
      {
         printf("Failed to allocate heap %d of size %ld (Ret:%d)\n",i,memsize,ret);
         bigbuf[i]=NULL;
         bigsize[i]=0;
      }
      else
      {
         memoryallocated+=memsize;
         printf("Heap %d allocated size %ld\n",i,memsize);
         bigsize[i]=memsize;
      }
   }

   printf("Available Memory after allocation %ld\n",sysconf(_SC_AVPHYS_PAGES) * sysconf(_SC_PAGESIZE));

   sleep(1);

   /*some more spare memory less than 16MB ?*/
   memsize=addsize=10*1024;
   do
   {
      memsize+=addsize;
      if(!(ret=rt_heap_create(&bigheap[i],nbuf,memsize,H_PRIO|H_MAPPABLE)))
         rt_heap_delete(&bigheap[i]);
   }while(ret==0);

   memsize>>=1;
   if(rt_heap_create(&bigheap[i],nbuf,memsize,H_PRIO|H_MAPPABLE))
   {
      printf("Heap %d creation failed for size %ld\n",i,memsize);
   }
   else
   {
      printf("Heap %d created with size %ld\n",i,memsize);
      if((ret=rt_heap_alloc(&bigheap[i],memsize,10000000000,&bigbuf[i])) )
      {
         printf("Failed to allocate heap %d of size %ld (Ret:%d).\n",i,memsize,ret);
         bigbuf[i]=NULL;
         bigsize[i]=0;
      }
      else
      {
         bigsize[i]=memsize;
         printf("Heap %d allocated size %ld\n",i,memsize);
         memoryallocated+=memsize;
      }
   }



   printf("Memory allocated in total : %d\n", memoryallocated);

   for (;;)
   {
      err = rt_task_wait_period(NULL);

      for(i=0;i<MAXHEAPBLOCKS;i++)
      {
         if(bigbuf[i]!=NULL)
         {
            /*printf("Setting heap %d. Size %d\n",i,bigsize[i]);*/
            memset(bigbuf[i],0xAF,bigsize[i]);
         }

      }

   }
}




int root_thread_init(void)
{
   printf("root_thread_init :\n");

   rt_timer_set_mode(TM_ONESHOT);

   printf("timer startedt\n");


   rt_task_spawn(&display_task,"DisplayTask", DISPLAY_STACK_SIZE, DISPLAY_TASK_PRI, 0, &display, NULL);
   printf("display task created\n");

   return 0;
}

void root_thread_exit(void)
{
   int i;
   RT_HEAP_INFO *hpinfo;

   printf("UDP->root_thread_exit\n");

   for(i=0;i<MAXHEAPBLOCKS;i++)
   {
      if( !rt_heap_inquire(&bigheap[i],hpinfo) )
         rt_heap_delete(&bigheap[i]);
   }

   rt_task_delete(&display_task);
}

void cleanup_upon_sig(int sig __attribute__ ((unused)))
{
   root_thread_exit();
}

int main(int argc, char *argv[])
{

   printf("Start Heap\n");
   sleep(1);

   mlockall(MCL_CURRENT | MCL_FUTURE);

   signal(SIGINT, cleanup_upon_sig);
   signal(SIGTERM, cleanup_upon_sig);
   signal(SIGHUP, cleanup_upon_sig);
   signal(SIGALRM, cleanup_upon_sig);


   if(root_thread_init()<0)
      goto exit;

   printf("root_thread_init beendet\n");
   sleep(1), pause();

exit:
   root_thread_exit();

   return 0;
}


[-- Attachment #3: Makefile.hp2 --]
[-- Type: application/octet-stream, Size: 672 bytes --]

xenodir:=/home/user/mc45/xenomai/_install
CC:=ppc-linux-gcc

# Allow overriding xeno-config on make command line
XENO_CONFIG=$(xenodir)/bin/xeno-config
prefix := $(shell $(XENO_CONFIG) --prefix)

ifeq ($(prefix),)
$(error Please add <xenomai-install-path>/bin to your PATH variable)
endif

STD_CFLAGS  := $(shell $(XENO_CONFIG) --xeno-cflags)
STD_LDFLAGS := $(shell $(XENO_CONFIG) --xeno-ldflags) -lnative -lm -lrtdm
STD_TARGETS := heap


# "std" makes all demos running natively in user-space.

std: $(STD_TARGETS)


all: std


$(STD_TARGETS): $(STD_TARGETS:%=%.c)
	$(CC) -o $@ $@.c $(STD_CFLAGS) $(STD_LDFLAGS)


clean:
	$(RM) -f *.o *_s.o $(STD_TARGETS) $(SIM_TARGETS)

Re: [Xenomai-help] memset of heap crashes Xenomai-Task

[Gilles Chanteperdrix]

Roderik_Wildenburg@domain.hid wrote:
 > Xenomai Version : 2.2-rc2
 > Skin : native
 > Kernel : 2.4.25
 > Arch.: PPC
 > 
 > I try to allocate as much memory as possible with the functions :
 > rt_heap_create and
 > rt_heap_alloc.
 > (see also "Xenomai heap services" in this mailing list; see source
 > attached)
 > 
 > When I try to use the allocated memory with memset, the Xenomai-task
 > crashes with a "Segmentation fault".
 > 
 > Is memset allowed to be used with Xenomai heaps ?

memset should work with Xenomai heaps, I suspect your problem is rather
that the memory is not really allocated until you memset it, which fails
when no memory is available. In this case, calling memset on memory
allocated with malloc should segfault the same way when the system
memory is exhausted. IIRC, this behaviour is documented in mlockall
manual page.

Be careful with sysconf(_SC_AVPHYS_PAGES), it may include the swap size,
but when mlocking memory, your application can not use swap pages, so,
you should substract the size of swap, if any.
Also, what is the value of /proc/sys/vm/overcommit_memory on your system
?

-- 


					    Gilles Chanteperdrix.

Re: [Xenomai-help] memset of heap crashes Xenomai-Task

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1634 bytes --]

Roderik_Wildenburg@domain.hid wrote:
> Xenomai Version : 2.2-rc2
> Skin : native
> Kernel : 2.4.25
> Arch.: PPC
> 
> I try to allocate as much memory as possible with the functions :
> rt_heap_create and
> rt_heap_alloc.
> (see also "Xenomai heap services" in this mailing list; see source
> attached)
> 
> When I try to use the allocated memory with memset, the Xenomai-task
> crashes with a "Segmentation fault".
> 
> Is memset allowed to be used with Xenomai heaps ?
> If so, could somebody try to reproduce this (preferable with a PPC), to
> see whether it is a common problem or just one of my configuration .
> I have the feeling it is timing dependend, as it does not crash every
> time. So try to start it several times, please.
> 

Nice test case with a tendency to jump out of every half-opened window
around. :)

First of all, I'm not sure if that 8K of stack is enough on PPC with
2.4, on x86 over 2.6 it isn't. The native skin picks PTHREAD_STACK_MIN*4
for you by default. Is this too much? Unless dealing with dozens of
*simple* threads, reducing this makes no sense to me.

Then there was this bug:

@@ -168,13 +169,13 @@
 void root_thread_exit(void)
 {
    int i;
-   RT_HEAP_INFO *hpinfo;
+   RT_HEAP_INFO hpinfo;

    printf("UDP->root_thread_exit\n");

    for(i=0;i<MAXHEAPBLOCKS;i++)
    {
-      if( !rt_heap_inquire(&bigheap[i],hpinfo) )
+      if( !rt_heap_inquire(&bigheap[i],&hpinfo) )
          rt_heap_delete(&bigheap[i]);
    }

Now I get some other strange effects (e.g. kernel oopses after OOM
oopses) that need a closer look under qemu later.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-help] memset of heap crashes Xenomai-Task

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

Gilles Chanteperdrix wrote:
> Roderik_Wildenburg@domain.hid wrote:
>  > Xenomai Version : 2.2-rc2
>  > Skin : native
>  > Kernel : 2.4.25
>  > Arch.: PPC
>  > 
>  > I try to allocate as much memory as possible with the functions :
>  > rt_heap_create and
>  > rt_heap_alloc.
>  > (see also "Xenomai heap services" in this mailing list; see source
>  > attached)
>  > 
>  > When I try to use the allocated memory with memset, the Xenomai-task
>  > crashes with a "Segmentation fault".
>  > 
>  > Is memset allowed to be used with Xenomai heaps ?
> 
> memset should work with Xenomai heaps, I suspect your problem is rather
> that the memory is not really allocated until you memset it, which fails
> when no memory is available. In this case, calling memset on memory
> allocated with malloc should segfault the same way when the system
> memory is exhausted. IIRC, this behaviour is documented in mlockall
> manual page.

I wonder if this "virtual allocation" also applies to vmalloc'ed memory
like in this case. I don't think so, or the kernel would oops as well.

> 
> Be careful with sysconf(_SC_AVPHYS_PAGES), it may include the swap size,
> but when mlocking memory, your application can not use swap pages, so,
> you should substract the size of swap, if any.
> Also, what is the value of /proc/sys/vm/overcommit_memory on your system
> ?
> 

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-help] memset of heap crashes Xenomai-Task

[Gilles Chanteperdrix]

Jan Kiszka wrote:
 > Gilles Chanteperdrix wrote:
 > > Roderik_Wildenburg@domain.hid wrote:
 > >  > Xenomai Version : 2.2-rc2
 > >  > Skin : native
 > >  > Kernel : 2.4.25
 > >  > Arch.: PPC
 > >  > 
 > >  > I try to allocate as much memory as possible with the functions :
 > >  > rt_heap_create and
 > >  > rt_heap_alloc.
 > >  > (see also "Xenomai heap services" in this mailing list; see source
 > >  > attached)
 > >  > 
 > >  > When I try to use the allocated memory with memset, the Xenomai-task
 > >  > crashes with a "Segmentation fault".
 > >  > 
 > >  > Is memset allowed to be used with Xenomai heaps ?
 > > 
 > > memset should work with Xenomai heaps, I suspect your problem is rather
 > > that the memory is not really allocated until you memset it, which fails
 > > when no memory is available. In this case, calling memset on memory
 > > allocated with malloc should segfault the same way when the system
 > > memory is exhausted. IIRC, this behaviour is documented in mlockall
 > > manual page.
 > 
 > I wonder if this "virtual allocation" also applies to vmalloc'ed memory
 > like in this case. I don't think so, or the kernel would oops as well.

On some architectures vmalloced memory is only added to a process page
directory when the fault occurs. I do not know if the RAM allocation is
delayed until then, though.

-- 


					    Gilles Chanteperdrix.