[Xenomai-help] SIGSEGV Segmentation fault during malloc within Xenomai POSIX thread

[Xenomai-help] SIGSEGV Segmentation fault during malloc within Xenomai POSIX thread

[Steve]

[-- Attachment #1: Type: text/plain, Size: 3321 bytes --]

Hi,

I am new to Xenomai (and mailling lists), and have been porting a working
c++ RTAI program so that it runs on Xenomai using POSIX threads, sempahores,
etc.  I have succeeded in getting the new version of the application running
and starting to initialise its various components, part of which involves
creating threads through a common function.  However it seg faults part way
through the main thread when instantiating an object with the 'new' keyword,
and the stack trace shows that malloc is the last identifiable call before
the fault:

Call Stack:
0xb7ac6f3c ??() /lib/tls/i686/cmov/libc.so.6
0xb7ac8cad malloc() /lib/tls/i686/cmov/libc.so.6
0xb7da3097 ObjectName::CurrentFunction(this=0x0b70a531c)
/path-to-file/file.cpp
...

I can't post the entire code here, but have not been able to replicate this
with a simpler example.  However, I can reproduce it exactly using malloc in
the lines before the call.  I have put an example that creates threads and
schedules them in a way similar to the full program at the end of this email
for reference.  The actual call that faults is:
Object *Instance = new Object(StringVar.c_str(), Pointer);

The same fault was replicated on the line before by inserting:
char *testWord;
testWord = (char *) malloc(1000 * sizeof(char));

Or by inserting:
char *testWord;
testWord = (char *) malloc(500 * sizeof(char));
testWord = (char *) malloc(500 * sizeof(char));

Or by inserting:
char *testWord;
testWord = (char *) malloc(500 * sizeof(char));
free(testWord);
testWord = (char *) malloc(500 * sizeof(char));

The last of these confuses me the most, because in theory the memory has
been freed and so the second call is no worse off than the first, but it seg
faults.  This is made worse by the fact that I can call malloc and free as
many times as I like (i.e. more than 10) on memory blocks of size
100*(sizeof(char)), and can call malloc on 11 blocks of size
100*(sizeof(char)) without freeing it before the 12th call seg faults.

I am out of ideas - I cannot use Valgrind because Xenomai and Valgrind don't
play well yet - and am positing this issue here because the faulty piece of
code worked fine before the Xenomai port.  My only thought is that it might
be caused by some sort of paging error as a result of using MLockAll(), but
if so I do not know what to do about it.

I am using Xenomai 2.4.6.1 on a vanilla 2.6.24.6 Linux kernel, and am
running Ubuntu 8.04  on 512MB RAM, 2.8 GHz Pentium 4.  I am compiling with
gcc, and debugging with gdb, both through the Code::Blocks IDE.  My glibc
version is 2.7.

Any insight would be very much appreciated, and if you need more information
then let me know.

Steve

******
The example thread creation functions:

void createThread()
{
    pthread_t thread;
    pthread_create(&thread, NULL, function, (void *)"Anonymous Thread");
    pthread_detach(thread);
}

void *function(void *ptr)
{
    pthread_t self;
    sched_param *schedParams = new sched_param;

    self = pthread_self();
    schedParams->sched_priority = 5;

    // Make the thread real time by settings its priority (and any other
scheduling parameters)
    pthread_setschedparam(self, SCHED_FIFO, schedParams);

    while(1) // Loops forever, but only if there is a requested period
    {
        perform((char *)ptr);
        delay(300000000L);
    }

}

[-- Attachment #2: Type: text/html, Size: 3574 bytes --]

Re: [Xenomai-help] SIGSEGV Segmentation fault during malloc within Xenomai POSIX thread

[Gilles Chanteperdrix]

Steve wrote:
> Hi,
> 
> I am new to Xenomai (and mailling lists), and have been porting a working
> c++ RTAI program so that it runs on Xenomai using POSIX threads, sempahores,
> etc.  I have succeeded in getting the new version of the application running
> and starting to initialise its various components, part of which involves
> creating threads through a common function.  However it seg faults part way
> through the main thread when instantiating an object with the 'new' keyword,
> and the stack trace shows that malloc is the last identifiable call before
> the fault:
> 
> Call Stack:
> 0xb7ac6f3c ??() /lib/tls/i686/cmov/libc.so.6
> 0xb7ac8cad malloc() /lib/tls/i686/cmov/libc.so.6
> 0xb7da3097 ObjectName::CurrentFunction(this=0x0b70a531c)
> /path-to-file/file.cpp
> ...
> 
> I can't post the entire code here, but have not been able to replicate this
> with a simpler example.  However, I can reproduce it exactly using malloc in
> the lines before the call.  I have put an example that creates threads and
> schedules them in a way similar to the full program at the end of this email
> for reference.  The actual call that faults is:
> Object *Instance = new Object(StringVar.c_str(), Pointer);
> 
> The same fault was replicated on the line before by inserting:
> char *testWord;
> testWord = (char *) malloc(1000 * sizeof(char));
> 
> Or by inserting:
> char *testWord;
> testWord = (char *) malloc(500 * sizeof(char));
> testWord = (char *) malloc(500 * sizeof(char));
> 
> Or by inserting:
> char *testWord;
> testWord = (char *) malloc(500 * sizeof(char));
> free(testWord);
> testWord = (char *) malloc(500 * sizeof(char));
> 
> The last of these confuses me the most, because in theory the memory has
> been freed and so the second call is no worse off than the first, but it seg
> faults.  This is made worse by the fact that I can call malloc and free as
> many times as I like (i.e. more than 10) on memory blocks of size
> 100*(sizeof(char)), and can call malloc on 11 blocks of size
> 100*(sizeof(char)) without freeing it before the 12th call seg faults.
> 
> I am out of ideas - I cannot use Valgrind because Xenomai and Valgrind don't
> play well yet - and am positing this issue here because the faulty piece of
> code worked fine before the Xenomai port.  My only thought is that it might
> be caused by some sort of paging error as a result of using MLockAll(), but
> if so I do not know what to do about it.
> 
> I am using Xenomai 2.4.6.1 on a vanilla 2.6.24.6 Linux kernel, and am
> running Ubuntu 8.04  on 512MB RAM, 2.8 GHz Pentium 4.  I am compiling with
> gcc, and debugging with gdb, both through the Code::Blocks IDE.  My glibc
> version is 2.7.
> 
> Any insight would be very much appreciated, and if you need more information
> then let me know.

A segfault of malloc is typical of memory corruption. And valgrind is
indeed the best tool to find what happens.

Now that you converted your application to the posix skin, running
valgrind is easy: simply compile your application without xenomai. Of
course, it will not have a good determinism, but at least it should
allow you to find the bug.

-- 
                                                 Gilles.

Re: [Xenomai-help] SIGSEGV Segmentation fault during malloc within Xenomai POSIX thread

[Philippe Gerum]

On Thu, 2009-04-09 at 10:53 +0100, Steve wrote:
> Hi,
> 
> I am new to Xenomai (and mailling lists), and have been porting a
> working c++ RTAI program so that it runs on Xenomai using POSIX
> threads, sempahores, etc.  I have succeeded in getting the new version
> of the application running and starting to initialise its various
> components, part of which involves creating threads through a common
> function.  However it seg faults part way through the main thread when
> instantiating an object with the 'new' keyword, and the stack trace
> shows that malloc is the last identifiable call before the fault:
> 
> Call Stack:
> 0xb7ac6f3c ??() /lib/tls/i686/cmov/libc.so.6
> 0xb7ac8cad malloc() /lib/tls/i686/cmov/libc.so.6
> 0xb7da3097
> ObjectName::CurrentFunction(this=0x0b70a531c) /path-to-file/file.cpp
> ...
> 
> I can't post the entire code here, but have not been able to replicate
> this with a simpler example.  However, I can reproduce it exactly
> using malloc in the lines before the call.  I have put an example that
> creates threads and schedules them in a way similar to the full
> program at the end of this email for reference.  The actual call that
> faults is:
> Object *Instance = new Object(StringVar.c_str(), Pointer);

> The same fault was replicated on the line before by inserting:
> char *testWord;
> testWord = (char *) malloc(1000 * sizeof(char));
> 
> Or by inserting:
> char *testWord;
> testWord = (char *) malloc(500 * sizeof(char));
> testWord = (char *) malloc(500 * sizeof(char));
> 
> Or by inserting:
> char *testWord;
> testWord = (char *) malloc(500 * sizeof(char));
> free(testWord);
> testWord = (char *) malloc(500 * sizeof(char));
> 
> The last of these confuses me the most, because in theory the memory
> has been freed and so the second call is no worse off than the first,
> but it seg faults.  This is made worse by the fact that I can call
> malloc and free as many times as I like (i.e. more than 10) on memory
> blocks of size 100*(sizeof(char)), and can call malloc on 11 blocks of
> size 100*(sizeof(char)) without freeing it before the 12th call seg
> faults.
> 
> I am out of ideas - I cannot use Valgrind because Xenomai and Valgrind
> don't play well yet - and am positing this issue here because the
> faulty piece of code worked fine before the Xenomai port.  My only
> thought is that it might be caused by some sort of paging error as a
> result of using MLockAll(), but if so I do not know what to do about
> it.
> 

The port may have uncovered some hidden issue in your code as well.
Using glibc's built-in malloc debug features, I would suggest to:

export MALLOC_CHECK_=1
then run your app; you might get more information.

export MALLOC_CHECK_=2
will trigger an abort() when/if any heap corruption is detected.

Calling mcheck() to verify whether the heap is still fine is another
option.

-- 
Philippe.