[Xenomai-help] mlockall questions

[Xenomai-help] mlockall questions

[Robert McCullough]

Hi,

I am converting part of a C++ application to real-time.

I noticed that all the Xenomai examples call
mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before creating
any real-time threads.
Is calling mlockall a requirement when using Xenomai?

I have added a couple of Xenomai tasks to my C++ app.
A) Without calling mlockall my application seems to run fine but I noticed
in /proc/xenomai/stat that a few page faults occurred.
B) When calling mlockall at the beginning of my main() I do not see any page
faults but some of the C++ threads do not start.

Why would mlockall cause some threads not to start?
How would I fix this?

Best regards, 
Robert McCullough

Re: [Xenomai-help] mlockall questions

[Gilles Chanteperdrix]

Robert McCullough wrote:
 > Hi,
 > 
 > I am converting part of a C++ application to real-time.
 > 
 > I noticed that all the Xenomai examples call
 > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before creating
 > any real-time threads.
 > Is calling mlockall a requirement when using Xenomai?

Yes it is.

 > 
 > I have added a couple of Xenomai tasks to my C++ app.
 > A) Without calling mlockall my application seems to run fine but I noticed
 > in /proc/xenomai/stat that a few page faults occurred.

Well normally your application should stop with a message telling you
that you have not call mlockall. Do you happen to install a handler for
SIGXCPU ?

 > B) When calling mlockall at the beginning of my main() I do not see any page
 > faults but some of the C++ threads do not start.
 > 
 > Why would mlockall cause some threads not to start?

Well, you should check rt_task_create or pthread_create return
value. The usual error is that you run out of memory because of the
stack size problem explained in the TROUBLESHOOTING guide.

-- 


					    Gilles.

Re: [Xenomai-help] mlockall questions

[Robert McCullough]

Hi Gilles,

  --> -----Original Message-----
  --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
  --> Sent: Wednesday, April 09, 2008 3:29 PM
  --> To: rob@domain.hid
  --> Cc: xenomai@xenomai.org
  --> Subject: Re: [Xenomai-help] mlockall questions
  --> 
  --> Robert McCullough wrote:
  -->  > Hi,
  -->  >
  -->  > I am converting part of a C++ application to real-time.
  -->  >
  -->  > I noticed that all the Xenomai examples call
  -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
  --> creating
  -->  > any real-time threads.
  -->  > Is calling mlockall a requirement when using Xenomai?
  --> 
  --> Yes it is.
  --> 
[Robert McCullough] 
OK.
  -->  >
  -->  > I have added a couple of Xenomai tasks to my C++ app.
  -->  > A) Without calling mlockall my application seems to run fine but I
  --> noticed
  -->  > in /proc/xenomai/stat that a few page faults occurred.
  --> 
  --> Well normally your application should stop with a message telling you
  --> that you have not call mlockall. Do you happen to install a handler
  --> for
  --> SIGXCPU ?
[Robert McCullough] 
Yes.
  --> 
  -->  > B) When calling mlockall at the beginning of my main() I do not see
  --> any page
  -->  > faults but some of the C++ threads do not start.
  -->  >
  -->  > Why would mlockall cause some threads not to start?
  --> 
  --> Well, you should check rt_task_create or pthread_create return
  --> value. The usual error is that you run out of memory because of the
  --> stack size problem explained in the TROUBLESHOOTING guide.
  --> 
  --> --
  --> 
  --> 
  --> 					    Gilles.
[Robert McCullough] 
Thanks.
Calling "ulimit -s 2048" before my application in my startup script seems to
work fine when I am running over nfs to the Denx ELDK with a bash shell.
But my application needs to run from a CompactFlash card running BusyBox on
a MPC5200 and BusyBox does not seem to support the "ulimit" command.
Is there another way to set this stack limit size?

Best regards, 
Robert McCullough

Re: [Xenomai-help] mlockall questions

[Philippe Gerum]

Robert McCullough wrote:
> Hi Gilles,
> 
>   --> -----Original Message-----
>   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>   --> Sent: Wednesday, April 09, 2008 3:29 PM
>   --> To: rob@domain.hid
>   --> Cc: xenomai@xenomai.org
>   --> Subject: Re: [Xenomai-help] mlockall questions
>   --> 
>   --> Robert McCullough wrote:
>   -->  > Hi,
>   -->  >
>   -->  > I am converting part of a C++ application to real-time.
>   -->  >
>   -->  > I noticed that all the Xenomai examples call
>   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>   --> creating
>   -->  > any real-time threads.
>   -->  > Is calling mlockall a requirement when using Xenomai?
>   --> 
>   --> Yes it is.
>   --> 
> [Robert McCullough] 
> OK.
>   -->  >
>   -->  > I have added a couple of Xenomai tasks to my C++ app.
>   -->  > A) Without calling mlockall my application seems to run fine but I
>   --> noticed
>   -->  > in /proc/xenomai/stat that a few page faults occurred.
>   --> 
>   --> Well normally your application should stop with a message telling you
>   --> that you have not call mlockall. Do you happen to install a handler
>   --> for
>   --> SIGXCPU ?
> [Robert McCullough] 
> Yes.
>   --> 
>   -->  > B) When calling mlockall at the beginning of my main() I do not see
>   --> any page
>   -->  > faults but some of the C++ threads do not start.
>   -->  >
>   -->  > Why would mlockall cause some threads not to start?
>   --> 
>   --> Well, you should check rt_task_create or pthread_create return
>   --> value. The usual error is that you run out of memory because of the
>   --> stack size problem explained in the TROUBLESHOOTING guide.
>   --> 
>   --> --
>   --> 
>   --> 
>   --> 					    Gilles.
> [Robert McCullough] 
> Thanks.
> Calling "ulimit -s 2048" before my application in my startup script seems to
> work fine when I am running over nfs to the Denx ELDK with a bash shell.
> But my application needs to run from a CompactFlash card running BusyBox on
> a MPC5200 and BusyBox does not seem to support the "ulimit" command.
> Is there another way to set this stack limit size?
> 

- Setting the stack size property into the attribute struct for each new pthread
your application may create: pthread_attr_setstacksize(&thattr, stacksize);

- Or passing the right stack size value to any (non-POSIX) Xenomai API task
creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
t_create()...).

- Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
subsequent glibc-originated threads. This is what the ulimit command does, actually.

-- 
Philippe.

Re: [Xenomai-help] mlockall questions

[Gilles Chanteperdrix]

On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
<philippe.gerum@domain.hid> wrote:
>
> Robert McCullough wrote:
>  > Hi Gilles,
>  >
>  >   --> -----Original Message-----
>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
>  >   --> To: rob@domain.hid
>  >   --> Cc: xenomai@xenomai.org
>  >   --> Subject: Re: [Xenomai-help] mlockall questions
>  >   -->
>  >   --> Robert McCullough wrote:
>  >   -->  > Hi,
>  >   -->  >
>  >   -->  > I am converting part of a C++ application to real-time.
>  >   -->  >
>  >   -->  > I noticed that all the Xenomai examples call
>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>  >   --> creating
>  >   -->  > any real-time threads.
>  >   -->  > Is calling mlockall a requirement when using Xenomai?
>  >   -->
>  >   --> Yes it is.
>  >   -->
>  > [Robert McCullough]
>  > OK.
>  >   -->  >
>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
>  >   -->  > A) Without calling mlockall my application seems to run fine but I
>  >   --> noticed
>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
>  >   -->
>  >   --> Well normally your application should stop with a message telling you
>  >   --> that you have not call mlockall. Do you happen to install a handler
>  >   --> for
>  >   --> SIGXCPU ?
>  > [Robert McCullough]
>  > Yes.
>  >   -->
>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
>  >   --> any page
>  >   -->  > faults but some of the C++ threads do not start.
>  >   -->  >
>  >   -->  > Why would mlockall cause some threads not to start?
>  >   -->
>  >   --> Well, you should check rt_task_create or pthread_create return
>  >   --> value. The usual error is that you run out of memory because of the
>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
>  >   -->
>  >   --> --
>  >   -->
>  >   -->
>  >   -->                                             Gilles.
>  > [Robert McCullough]
>  > Thanks.
>  > Calling "ulimit -s 2048" before my application in my startup script seems to
>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
>  > But my application needs to run from a CompactFlash card running BusyBox on
>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
>  > Is there another way to set this stack limit size?
>  >
>
>  - Setting the stack size property into the attribute struct for each new pthread
>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
>
>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
>  t_create()...).
>
>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
>  subsequent glibc-originated threads. This is what the ulimit command does, actually.

The problem of pthread_attr_setstacksize (or of setting stack size in
one of the various thread creation services) is that it does not work
for the main thread stack. So, you have to use ulimit or to call
setrlimit before running your application, in order to set the main
thread stack size.

Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.

-- 
 Gilles

Re: [Xenomai-help] mlockall questions

[Philippe Gerum]

Gilles Chanteperdrix wrote:
> On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
> <philippe.gerum@domain.hid> wrote:
>> Robert McCullough wrote:
>>  > Hi Gilles,
>>  >
>>  >   --> -----Original Message-----
>>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
>>  >   --> To: rob@domain.hid
>>  >   --> Cc: xenomai@xenomai.org
>>  >   --> Subject: Re: [Xenomai-help] mlockall questions
>>  >   -->
>>  >   --> Robert McCullough wrote:
>>  >   -->  > Hi,
>>  >   -->  >
>>  >   -->  > I am converting part of a C++ application to real-time.
>>  >   -->  >
>>  >   -->  > I noticed that all the Xenomai examples call
>>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>>  >   --> creating
>>  >   -->  > any real-time threads.
>>  >   -->  > Is calling mlockall a requirement when using Xenomai?
>>  >   -->
>>  >   --> Yes it is.
>>  >   -->
>>  > [Robert McCullough]
>>  > OK.
>>  >   -->  >
>>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
>>  >   -->  > A) Without calling mlockall my application seems to run fine but I
>>  >   --> noticed
>>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
>>  >   -->
>>  >   --> Well normally your application should stop with a message telling you
>>  >   --> that you have not call mlockall. Do you happen to install a handler
>>  >   --> for
>>  >   --> SIGXCPU ?
>>  > [Robert McCullough]
>>  > Yes.
>>  >   -->
>>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
>>  >   --> any page
>>  >   -->  > faults but some of the C++ threads do not start.
>>  >   -->  >
>>  >   -->  > Why would mlockall cause some threads not to start?
>>  >   -->
>>  >   --> Well, you should check rt_task_create or pthread_create return
>>  >   --> value. The usual error is that you run out of memory because of the
>>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
>>  >   -->
>>  >   --> --
>>  >   -->
>>  >   -->
>>  >   -->                                             Gilles.
>>  > [Robert McCullough]
>>  > Thanks.
>>  > Calling "ulimit -s 2048" before my application in my startup script seems to
>>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
>>  > But my application needs to run from a CompactFlash card running BusyBox on
>>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
>>  > Is there another way to set this stack limit size?
>>  >
>>
>>  - Setting the stack size property into the attribute struct for each new pthread
>>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
>>
>>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
>>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
>>  t_create()...).
>>
>>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
>>  subsequent glibc-originated threads. This is what the ulimit command does, actually.
> 
> The problem of pthread_attr_setstacksize (or of setting stack size in
> one of the various thread creation services) is that it does not work
> for the main thread stack. So, you have to use ulimit or to call
> setrlimit before running your application, in order to set the main
> thread stack size.
> 
> Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.
> 

Even if you don't have any shell supporting ulimit, you just need to create a
small launcher that basically forks/setrlimit+execs your application.

-- 
Philippe.

Re: [Xenomai-help] mlockall questions

[Gilles Chanteperdrix]

On Thu, Apr 10, 2008 at 4:36 PM, Philippe Gerum
<philippe.gerum@domain.hid> wrote:
>
> Gilles Chanteperdrix wrote:
>  > On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
>  > <philippe.gerum@domain.hid> wrote:
>  >> Robert McCullough wrote:
>  >>  > Hi Gilles,
>  >>  >
>  >>  >   --> -----Original Message-----
>  >>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>  >>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
>  >>  >   --> To: rob@domain.hid
>  >>  >   --> Cc: xenomai@xenomai.org
>  >>  >   --> Subject: Re: [Xenomai-help] mlockall questions
>  >>  >   -->
>  >>  >   --> Robert McCullough wrote:
>  >>  >   -->  > Hi,
>  >>  >   -->  >
>  >>  >   -->  > I am converting part of a C++ application to real-time.
>  >>  >   -->  >
>  >>  >   -->  > I noticed that all the Xenomai examples call
>  >>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>  >>  >   --> creating
>  >>  >   -->  > any real-time threads.
>  >>  >   -->  > Is calling mlockall a requirement when using Xenomai?
>  >>  >   -->
>  >>  >   --> Yes it is.
>  >>  >   -->
>  >>  > [Robert McCullough]
>  >>  > OK.
>  >>  >   -->  >
>  >>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
>  >>  >   -->  > A) Without calling mlockall my application seems to run fine but I
>  >>  >   --> noticed
>  >>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
>  >>  >   -->
>  >>  >   --> Well normally your application should stop with a message telling you
>  >>  >   --> that you have not call mlockall. Do you happen to install a handler
>  >>  >   --> for
>  >>  >   --> SIGXCPU ?
>  >>  > [Robert McCullough]
>  >>  > Yes.
>  >>  >   -->
>  >>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
>  >>  >   --> any page
>  >>  >   -->  > faults but some of the C++ threads do not start.
>  >>  >   -->  >
>  >>  >   -->  > Why would mlockall cause some threads not to start?
>  >>  >   -->
>  >>  >   --> Well, you should check rt_task_create or pthread_create return
>  >>  >   --> value. The usual error is that you run out of memory because of the
>  >>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
>  >>  >   -->
>  >>  >   --> --
>  >>  >   -->
>  >>  >   -->
>  >>  >   -->                                             Gilles.
>  >>  > [Robert McCullough]
>  >>  > Thanks.
>  >>  > Calling "ulimit -s 2048" before my application in my startup script seems to
>  >>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
>  >>  > But my application needs to run from a CompactFlash card running BusyBox on
>  >>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
>  >>  > Is there another way to set this stack limit size?
>  >>  >
>  >>
>  >>  - Setting the stack size property into the attribute struct for each new pthread
>  >>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
>  >>
>  >>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
>  >>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
>  >>  t_create()...).
>  >>
>  >>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
>  >>  subsequent glibc-originated threads. This is what the ulimit command does, actually.
>  >
>  > The problem of pthread_attr_setstacksize (or of setting stack size in
>  > one of the various thread creation services) is that it does not work
>  > for the main thread stack. So, you have to use ulimit or to call
>  > setrlimit before running your application, in order to set the main
>  > thread stack size.
>  >
>  > Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.
>  >
>
>  Even if you don't have any shell supporting ulimit, you just need to create a
>  small launcher that basically forks/setrlimit+execs your application.

Errr... but you need to call setrlimit before fork, otherwise it is too late...

-- 
 Gilles

Re: [Xenomai-help] mlockall questions

[Gilles Chanteperdrix]

On Thu, Apr 10, 2008 at 5:43 PM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> On Thu, Apr 10, 2008 at 4:36 PM, Philippe Gerum
>
>
> <philippe.gerum@domain.hid> wrote:
>  >
>  > Gilles Chanteperdrix wrote:
>  >  > On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
>  >  > <philippe.gerum@domain.hid> wrote:
>  >  >> Robert McCullough wrote:
>  >  >>  > Hi Gilles,
>  >  >>  >
>  >  >>  >   --> -----Original Message-----
>  >  >>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>  >  >>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
>  >  >>  >   --> To: rob@domain.hid
>  >  >>  >   --> Cc: xenomai@xenomai.org
>  >  >>  >   --> Subject: Re: [Xenomai-help] mlockall questions
>  >  >>  >   -->
>  >  >>  >   --> Robert McCullough wrote:
>  >  >>  >   -->  > Hi,
>  >  >>  >   -->  >
>  >  >>  >   -->  > I am converting part of a C++ application to real-time.
>  >  >>  >   -->  >
>  >  >>  >   -->  > I noticed that all the Xenomai examples call
>  >  >>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>  >  >>  >   --> creating
>  >  >>  >   -->  > any real-time threads.
>  >  >>  >   -->  > Is calling mlockall a requirement when using Xenomai?
>  >  >>  >   -->
>  >  >>  >   --> Yes it is.
>  >  >>  >   -->
>  >  >>  > [Robert McCullough]
>  >  >>  > OK.
>  >  >>  >   -->  >
>  >  >>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
>  >  >>  >   -->  > A) Without calling mlockall my application seems to run fine but I
>  >  >>  >   --> noticed
>  >  >>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
>  >  >>  >   -->
>  >  >>  >   --> Well normally your application should stop with a message telling you
>  >  >>  >   --> that you have not call mlockall. Do you happen to install a handler
>  >  >>  >   --> for
>  >  >>  >   --> SIGXCPU ?
>  >  >>  > [Robert McCullough]
>  >  >>  > Yes.
>  >  >>  >   -->
>  >  >>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
>  >  >>  >   --> any page
>  >  >>  >   -->  > faults but some of the C++ threads do not start.
>  >  >>  >   -->  >
>  >  >>  >   -->  > Why would mlockall cause some threads not to start?
>  >  >>  >   -->
>  >  >>  >   --> Well, you should check rt_task_create or pthread_create return
>  >  >>  >   --> value. The usual error is that you run out of memory because of the
>  >  >>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
>  >  >>  >   -->
>  >  >>  >   --> --
>  >  >>  >   -->
>  >  >>  >   -->
>  >  >>  >   -->                                             Gilles.
>  >  >>  > [Robert McCullough]
>  >  >>  > Thanks.
>  >  >>  > Calling "ulimit -s 2048" before my application in my startup script seems to
>  >  >>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
>  >  >>  > But my application needs to run from a CompactFlash card running BusyBox on
>  >  >>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
>  >  >>  > Is there another way to set this stack limit size?
>  >  >>  >
>  >  >>
>  >  >>  - Setting the stack size property into the attribute struct for each new pthread
>  >  >>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
>  >  >>
>  >  >>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
>  >  >>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
>  >  >>  t_create()...).
>  >  >>
>  >  >>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
>  >  >>  subsequent glibc-originated threads. This is what the ulimit command does, actually.
>  >  >
>  >  > The problem of pthread_attr_setstacksize (or of setting stack size in
>  >  > one of the various thread creation services) is that it does not work
>  >  > for the main thread stack. So, you have to use ulimit or to call
>  >  > setrlimit before running your application, in order to set the main
>  >  > thread stack size.
>  >  >
>  >  > Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.
>  >  >
>  >
>  >  Even if you don't have any shell supporting ulimit, you just need to create a
>  >  small launcher that basically forks/setrlimit+execs your application.
>
>  Errr... but you need to call setrlimit before fork, otherwise it is too late...

It looks we are wrong. The main thread stack remains allocated
on-demand, even though the process is mlocked. It must be some recent
kernel evolution.

-- 
 Gilles

Re: [Xenomai-help] mlockall questions

[jeff koftinoff]

Gilles Chanteperdrix wrote:
>
> It looks we are wrong. The main thread stack remains allocated
> on-demand, even though the process is mlocked. It must be some recent
> kernel evolution

Hmm.. interesting.  What is the best way to make sure all my stacks are 
not allocated on-demand?

jeffk

Re: [Xenomai-help] mlockall questions

[Gilles Chanteperdrix]

On Thu, Apr 10, 2008 at 7:52 PM, jeff koftinoff <jeffk@domain.hid> wrote:
> Gilles Chanteperdrix wrote:
>
> >
> > It looks we are wrong. The main thread stack remains allocated
> > on-demand, even though the process is mlocked. It must be some recent
> > kernel evolution
> >
>
>  Hmm.. interesting.  What is the best way to make sure all my stacks are not
> allocated on-demand?

Find attached the example I used to read the stack size of an mlocked process.

You can run it on your target to see if you get the same result as me.

Now, I suspect this on-demand allocation only exists for the main
thread stack, I will run further tests to confirm this. So, you may:
- decide not to use the main thread for real-time duties
- memset a large buffer on stack to force the kernel to commit memory.

-- 
 Gilles

Re: [Xenomai-help] mlockall questions

[Gilles Chanteperdrix]

[-- Attachment #1: Type: text/plain, Size: 1242 bytes --]

On Thu, Apr 10, 2008 at 8:03 PM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
>
> On Thu, Apr 10, 2008 at 7:52 PM, jeff koftinoff <jeffk@domain.hid> wrote:
>  > Gilles Chanteperdrix wrote:
>  >
>  > >
>  > > It looks we are wrong. The main thread stack remains allocated
>  > > on-demand, even though the process is mlocked. It must be some recent
>  > > kernel evolution
>  > >
>  >
>  >  Hmm.. interesting.  What is the best way to make sure all my stacks are not
>  > allocated on-demand?
>
>  Find attached the example I used to read the stack size of an mlocked process.
>
>  You can run it on your target to see if you get the same result as me.
>
>  Now, I suspect this on-demand allocation only exists for the main
>  thread stack, I will run further tests to confirm this. So, you may:
>  - decide not to use the main thread for real-time duties
>  - memset a large buffer on stack to force the kernel to commit memory.

Forgot the attachment, but here is a better one, which prints the main
thread stack size as well as a newly created thread stack size. It
shows that, at least in my configuration, only the main thread stack
mapping is grown on-demand.

-- 
                                               Gilles

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: stack_size.c --]
[-- Type: text/x-csrc; name=stack_size.c, Size: 1518 bytes --]

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/mman.h>
#include <string.h>
#include <pthread.h>

long get_stack_size(unsigned long stackptr)
{
	char *line = NULL;
	size_t linesz;
	FILE *file;
	int len;

	file = fopen("/proc/self/maps", "r");
	if (!file) {
		perror("fopen(/proc/self/maps)");
		exit(EXIT_FAILURE);
	}

	while ((len = getline(&line, &linesz, file)) != -1) {
		unsigned long map_start, map_end;

		/* Remove trailing \r\n (if any) */
		while (line[len - 1] == '\n' || line[len - 1] == '\r')
			line[--len] = '\0';

		if (sscanf(line, "%lx-%lx", &map_start, &map_end) < 2) {
			perror("sscanf");
			exit(EXIT_FAILURE);
		}

		if ((unsigned) (stackptr - map_start) >= (map_end - map_start))
			continue;

		free(line);
		fclose(file);
		return map_end - map_start;
	}
	if (line)
		free(line);
	fclose(file);

	fprintf(stderr, "Could not find stack in /proc/self/maps\n");
	exit(EXIT_FAILURE);
}

unsigned long threadstack;
void *thread(void *cookie)
{
	unsigned dummy;
	threadstack = (unsigned long) &dummy;

	pause();

	return cookie;
}

int main(void)
{
	pthread_t tid;
	int err;

	if (mlockall(MCL_CURRENT | MCL_FUTURE)) {
		perror("mlockall");
		exit(EXIT_FAILURE);
	}

	if ((err = pthread_create(&tid, NULL, thread, NULL))) {
		fprintf(stderr, "pthread_create: %s\n", strerror(err));
		exit(EXIT_FAILURE);
	}
	sleep(1);

	printf("stack: %ldK\n", get_stack_size((unsigned long) &tid) / 1024);
	printf("thread stack: %ldK\n", get_stack_size(threadstack) / 1024);

	return 0;
}

Re: [Xenomai-help] mlockall questions

[Philippe Gerum]

Gilles Chanteperdrix wrote:
> On Thu, Apr 10, 2008 at 4:36 PM, Philippe Gerum
> <philippe.gerum@domain.hid> wrote:
>> Gilles Chanteperdrix wrote:
>>  > On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
>>  > <philippe.gerum@domain.hid> wrote:
>>  >> Robert McCullough wrote:
>>  >>  > Hi Gilles,
>>  >>  >
>>  >>  >   --> -----Original Message-----
>>  >>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>>  >>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
>>  >>  >   --> To: rob@domain.hid
>>  >>  >   --> Cc: xenomai@xenomai.org
>>  >>  >   --> Subject: Re: [Xenomai-help] mlockall questions
>>  >>  >   -->
>>  >>  >   --> Robert McCullough wrote:
>>  >>  >   -->  > Hi,
>>  >>  >   -->  >
>>  >>  >   -->  > I am converting part of a C++ application to real-time.
>>  >>  >   -->  >
>>  >>  >   -->  > I noticed that all the Xenomai examples call
>>  >>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>>  >>  >   --> creating
>>  >>  >   -->  > any real-time threads.
>>  >>  >   -->  > Is calling mlockall a requirement when using Xenomai?
>>  >>  >   -->
>>  >>  >   --> Yes it is.
>>  >>  >   -->
>>  >>  > [Robert McCullough]
>>  >>  > OK.
>>  >>  >   -->  >
>>  >>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
>>  >>  >   -->  > A) Without calling mlockall my application seems to run fine but I
>>  >>  >   --> noticed
>>  >>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
>>  >>  >   -->
>>  >>  >   --> Well normally your application should stop with a message telling you
>>  >>  >   --> that you have not call mlockall. Do you happen to install a handler
>>  >>  >   --> for
>>  >>  >   --> SIGXCPU ?
>>  >>  > [Robert McCullough]
>>  >>  > Yes.
>>  >>  >   -->
>>  >>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
>>  >>  >   --> any page
>>  >>  >   -->  > faults but some of the C++ threads do not start.
>>  >>  >   -->  >
>>  >>  >   -->  > Why would mlockall cause some threads not to start?
>>  >>  >   -->
>>  >>  >   --> Well, you should check rt_task_create or pthread_create return
>>  >>  >   --> value. The usual error is that you run out of memory because of the
>>  >>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
>>  >>  >   -->
>>  >>  >   --> --
>>  >>  >   -->
>>  >>  >   -->
>>  >>  >   -->                                             Gilles.
>>  >>  > [Robert McCullough]
>>  >>  > Thanks.
>>  >>  > Calling "ulimit -s 2048" before my application in my startup script seems to
>>  >>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
>>  >>  > But my application needs to run from a CompactFlash card running BusyBox on
>>  >>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
>>  >>  > Is there another way to set this stack limit size?
>>  >>  >
>>  >>
>>  >>  - Setting the stack size property into the attribute struct for each new pthread
>>  >>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
>>  >>
>>  >>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
>>  >>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
>>  >>  t_create()...).
>>  >>
>>  >>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
>>  >>  subsequent glibc-originated threads. This is what the ulimit command does, actually.
>>  >
>>  > The problem of pthread_attr_setstacksize (or of setting stack size in
>>  > one of the various thread creation services) is that it does not work
>>  > for the main thread stack. So, you have to use ulimit or to call
>>  > setrlimit before running your application, in order to set the main
>>  > thread stack size.
>>  >
>>  > Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.
>>  >
>>
>>  Even if you don't have any shell supporting ulimit, you just need to create a
>>  small launcher that basically forks/setrlimit+execs your application.
> 
> Errr... but you need to call setrlimit before fork, otherwise it is too late...
>

#include <sys/resource.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
	if (strcmp(argv[0], "child")) {
		struct rlimit r = { .rlim_cur = 32768, .rlim_max = 32768 };
		setrlimit(RLIMIT_STACK, &r);
		execl(argv[0], "child", NULL);
	} else {
		struct rlimit r;
		if (getrlimit(RLIMIT_STACK, &r) == 0)
			printf("%s: stack size = %d\n", argv[0], r.rlim_cur);
	}

	return 0;
}

-- 
Philippe.

Re: [Xenomai-help] mlockall questions

[Philippe Gerum]

Philippe Gerum wrote:
> Gilles Chanteperdrix wrote:
>> On Thu, Apr 10, 2008 at 4:36 PM, Philippe Gerum
>> <philippe.gerum@domain.hid> wrote:
>>> Gilles Chanteperdrix wrote:
>>>  > On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
>>>  > <philippe.gerum@domain.hid> wrote:
>>>  >> Robert McCullough wrote:
>>>  >>  > Hi Gilles,
>>>  >>  >
>>>  >>  >   --> -----Original Message-----
>>>  >>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>>>  >>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
>>>  >>  >   --> To: rob@domain.hid
>>>  >>  >   --> Cc: xenomai@xenomai.org
>>>  >>  >   --> Subject: Re: [Xenomai-help] mlockall questions
>>>  >>  >   -->
>>>  >>  >   --> Robert McCullough wrote:
>>>  >>  >   -->  > Hi,
>>>  >>  >   -->  >
>>>  >>  >   -->  > I am converting part of a C++ application to real-time.
>>>  >>  >   -->  >
>>>  >>  >   -->  > I noticed that all the Xenomai examples call
>>>  >>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>>>  >>  >   --> creating
>>>  >>  >   -->  > any real-time threads.
>>>  >>  >   -->  > Is calling mlockall a requirement when using Xenomai?
>>>  >>  >   -->
>>>  >>  >   --> Yes it is.
>>>  >>  >   -->
>>>  >>  > [Robert McCullough]
>>>  >>  > OK.
>>>  >>  >   -->  >
>>>  >>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
>>>  >>  >   -->  > A) Without calling mlockall my application seems to run fine but I
>>>  >>  >   --> noticed
>>>  >>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
>>>  >>  >   -->
>>>  >>  >   --> Well normally your application should stop with a message telling you
>>>  >>  >   --> that you have not call mlockall. Do you happen to install a handler
>>>  >>  >   --> for
>>>  >>  >   --> SIGXCPU ?
>>>  >>  > [Robert McCullough]
>>>  >>  > Yes.
>>>  >>  >   -->
>>>  >>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
>>>  >>  >   --> any page
>>>  >>  >   -->  > faults but some of the C++ threads do not start.
>>>  >>  >   -->  >
>>>  >>  >   -->  > Why would mlockall cause some threads not to start?
>>>  >>  >   -->
>>>  >>  >   --> Well, you should check rt_task_create or pthread_create return
>>>  >>  >   --> value. The usual error is that you run out of memory because of the
>>>  >>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
>>>  >>  >   -->
>>>  >>  >   --> --
>>>  >>  >   -->
>>>  >>  >   -->
>>>  >>  >   -->                                             Gilles.
>>>  >>  > [Robert McCullough]
>>>  >>  > Thanks.
>>>  >>  > Calling "ulimit -s 2048" before my application in my startup script seems to
>>>  >>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
>>>  >>  > But my application needs to run from a CompactFlash card running BusyBox on
>>>  >>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
>>>  >>  > Is there another way to set this stack limit size?
>>>  >>  >
>>>  >>
>>>  >>  - Setting the stack size property into the attribute struct for each new pthread
>>>  >>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
>>>  >>
>>>  >>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
>>>  >>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
>>>  >>  t_create()...).
>>>  >>
>>>  >>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
>>>  >>  subsequent glibc-originated threads. This is what the ulimit command does, actually.
>>>  >
>>>  > The problem of pthread_attr_setstacksize (or of setting stack size in
>>>  > one of the various thread creation services) is that it does not work
>>>  > for the main thread stack. So, you have to use ulimit or to call
>>>  > setrlimit before running your application, in order to set the main
>>>  > thread stack size.
>>>  >
>>>  > Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.
>>>  >
>>>
>>>  Even if you don't have any shell supporting ulimit, you just need to create a
>>>  small launcher that basically forks/setrlimit+execs your application.
>> Errr... but you need to call setrlimit before fork, otherwise it is too late...
>>
> 
> #include <sys/resource.h>
> #include <stdio.h>
> #include <unistd.h>
> #include <stdlib.h>
> 
> int main(int argc, char *argv[])
> {
> 	if (strcmp(argv[0], "child")) {
> 		struct rlimit r = { .rlim_cur = 32768, .rlim_max = 32768 };
> 		setrlimit(RLIMIT_STACK, &r);
> 		execl(argv[0], "child", NULL);
> 	} else {
> 		struct rlimit r;
> 		if (getrlimit(RLIMIT_STACK, &r) == 0)
> 			printf("%s: stack size = %d\n", argv[0], r.rlim_cur);
> 	}
> 
> 	return 0;
> }
> 

Just in case you don't think getrlimit() results also apply to the main thread:

#include <sys/resource.h>
#include <stdio.h>
#include <pthread.h>
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
	if (strcmp(argv[0], "child")) {
		struct rlimit r = { .rlim_cur = 32768, .rlim_max = 32768 };
		setrlimit(RLIMIT_STACK, &r);
		execl(argv[0], "child", NULL);
	} else {
		pthread_attr_t attr;
		size_t stacksize;
		void *stackbase;

		if (!pthread_getattr_np(pthread_self(), &attr)) {
			pthread_attr_getstack(&attr, &stackbase, &stacksize);
			printf("stacksize = %lu\n", stacksize);
		}
	}

	return 0;
}

-- 
Philippe.

Re: [Xenomai-help] mlockall questions

[Gilles Chanteperdrix]

Philippe Gerum wrote:
 > Gilles Chanteperdrix wrote:
 > > On Thu, Apr 10, 2008 at 4:36 PM, Philippe Gerum
 > > <philippe.gerum@domain.hid> wrote:
 > >> Gilles Chanteperdrix wrote:
 > >>  > On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
 > >>  > <philippe.gerum@domain.hid> wrote:
 > >>  >> Robert McCullough wrote:
 > >>  >>  > Hi Gilles,
 > >>  >>  >
 > >>  >>  >   --> -----Original Message-----
 > >>  >>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
 > >>  >>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
 > >>  >>  >   --> To: rob@domain.hid
 > >>  >>  >   --> Cc: xenomai@xenomai.org
 > >>  >>  >   --> Subject: Re: [Xenomai-help] mlockall questions
 > >>  >>  >   -->
 > >>  >>  >   --> Robert McCullough wrote:
 > >>  >>  >   -->  > Hi,
 > >>  >>  >   -->  >
 > >>  >>  >   -->  > I am converting part of a C++ application to real-time.
 > >>  >>  >   -->  >
 > >>  >>  >   -->  > I noticed that all the Xenomai examples call
 > >>  >>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
 > >>  >>  >   --> creating
 > >>  >>  >   -->  > any real-time threads.
 > >>  >>  >   -->  > Is calling mlockall a requirement when using Xenomai?
 > >>  >>  >   -->
 > >>  >>  >   --> Yes it is.
 > >>  >>  >   -->
 > >>  >>  > [Robert McCullough]
 > >>  >>  > OK.
 > >>  >>  >   -->  >
 > >>  >>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
 > >>  >>  >   -->  > A) Without calling mlockall my application seems to run fine but I
 > >>  >>  >   --> noticed
 > >>  >>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
 > >>  >>  >   -->
 > >>  >>  >   --> Well normally your application should stop with a message telling you
 > >>  >>  >   --> that you have not call mlockall. Do you happen to install a handler
 > >>  >>  >   --> for
 > >>  >>  >   --> SIGXCPU ?
 > >>  >>  > [Robert McCullough]
 > >>  >>  > Yes.
 > >>  >>  >   -->
 > >>  >>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
 > >>  >>  >   --> any page
 > >>  >>  >   -->  > faults but some of the C++ threads do not start.
 > >>  >>  >   -->  >
 > >>  >>  >   -->  > Why would mlockall cause some threads not to start?
 > >>  >>  >   -->
 > >>  >>  >   --> Well, you should check rt_task_create or pthread_create return
 > >>  >>  >   --> value. The usual error is that you run out of memory because of the
 > >>  >>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
 > >>  >>  >   -->
 > >>  >>  >   --> --
 > >>  >>  >   -->
 > >>  >>  >   -->
 > >>  >>  >   -->                                             Gilles.
 > >>  >>  > [Robert McCullough]
 > >>  >>  > Thanks.
 > >>  >>  > Calling "ulimit -s 2048" before my application in my startup script seems to
 > >>  >>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
 > >>  >>  > But my application needs to run from a CompactFlash card running BusyBox on
 > >>  >>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
 > >>  >>  > Is there another way to set this stack limit size?
 > >>  >>  >
 > >>  >>
 > >>  >>  - Setting the stack size property into the attribute struct for each new pthread
 > >>  >>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
 > >>  >>
 > >>  >>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
 > >>  >>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
 > >>  >>  t_create()...).
 > >>  >>
 > >>  >>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
 > >>  >>  subsequent glibc-originated threads. This is what the ulimit command does, actually.
 > >>  >
 > >>  > The problem of pthread_attr_setstacksize (or of setting stack size in
 > >>  > one of the various thread creation services) is that it does not work
 > >>  > for the main thread stack. So, you have to use ulimit or to call
 > >>  > setrlimit before running your application, in order to set the main
 > >>  > thread stack size.
 > >>  >
 > >>  > Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.
 > >>  >
 > >>
 > >>  Even if you don't have any shell supporting ulimit, you just need to create a
 > >>  small launcher that basically forks/setrlimit+execs your application.
 > > 
 > > Errr... but you need to call setrlimit before fork, otherwise it is too late...
 > >
 > 
 > #include <sys/resource.h>
 > #include <stdio.h>
 > #include <unistd.h>
 > #include <stdlib.h>
 > 
 > int main(int argc, char *argv[])
 > {
 > 	if (strcmp(argv[0], "child")) {
 > 		struct rlimit r = { .rlim_cur = 32768, .rlim_max = 32768 };
 > 		setrlimit(RLIMIT_STACK, &r);
 > 		execl(argv[0], "child", NULL);
 > 	} else {
 > 		struct rlimit r;
 > 		if (getrlimit(RLIMIT_STACK, &r) == 0)
 > 			printf("%s: stack size = %d\n", argv[0], r.rlim_cur);
 > 	}
 > 
 > 	return 0;
 > }

I had a doubt about what I wrote, just after writing it, which is why I
started doing my own testing. But I got distracted by the fact that all
the main thread stack was no longer commited. Now, I wonder, do you
think we should try to work around this behaviour in the I-pipe patch ?

-- 


					    Gilles.

Re: [Xenomai-help] mlockall questions

[Philippe Gerum]

Gilles Chanteperdrix wrote:
> Philippe Gerum wrote:
>  > Gilles Chanteperdrix wrote:
>  > > On Thu, Apr 10, 2008 at 4:36 PM, Philippe Gerum
>  > > <philippe.gerum@domain.hid> wrote:
>  > >> Gilles Chanteperdrix wrote:
>  > >>  > On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
>  > >>  > <philippe.gerum@domain.hid> wrote:
>  > >>  >> Robert McCullough wrote:
>  > >>  >>  > Hi Gilles,
>  > >>  >>  >
>  > >>  >>  >   --> -----Original Message-----
>  > >>  >>  >   --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
>  > >>  >>  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
>  > >>  >>  >   --> To: rob@domain.hid
>  > >>  >>  >   --> Cc: xenomai@xenomai.org
>  > >>  >>  >   --> Subject: Re: [Xenomai-help] mlockall questions
>  > >>  >>  >   -->
>  > >>  >>  >   --> Robert McCullough wrote:
>  > >>  >>  >   -->  > Hi,
>  > >>  >>  >   -->  >
>  > >>  >>  >   -->  > I am converting part of a C++ application to real-time.
>  > >>  >>  >   -->  >
>  > >>  >>  >   -->  > I noticed that all the Xenomai examples call
>  > >>  >>  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of main before
>  > >>  >>  >   --> creating
>  > >>  >>  >   -->  > any real-time threads.
>  > >>  >>  >   -->  > Is calling mlockall a requirement when using Xenomai?
>  > >>  >>  >   -->
>  > >>  >>  >   --> Yes it is.
>  > >>  >>  >   -->
>  > >>  >>  > [Robert McCullough]
>  > >>  >>  > OK.
>  > >>  >>  >   -->  >
>  > >>  >>  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
>  > >>  >>  >   -->  > A) Without calling mlockall my application seems to run fine but I
>  > >>  >>  >   --> noticed
>  > >>  >>  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
>  > >>  >>  >   -->
>  > >>  >>  >   --> Well normally your application should stop with a message telling you
>  > >>  >>  >   --> that you have not call mlockall. Do you happen to install a handler
>  > >>  >>  >   --> for
>  > >>  >>  >   --> SIGXCPU ?
>  > >>  >>  > [Robert McCullough]
>  > >>  >>  > Yes.
>  > >>  >>  >   -->
>  > >>  >>  >   -->  > B) When calling mlockall at the beginning of my main() I do not see
>  > >>  >>  >   --> any page
>  > >>  >>  >   -->  > faults but some of the C++ threads do not start.
>  > >>  >>  >   -->  >
>  > >>  >>  >   -->  > Why would mlockall cause some threads not to start?
>  > >>  >>  >   -->
>  > >>  >>  >   --> Well, you should check rt_task_create or pthread_create return
>  > >>  >>  >   --> value. The usual error is that you run out of memory because of the
>  > >>  >>  >   --> stack size problem explained in the TROUBLESHOOTING guide.
>  > >>  >>  >   -->
>  > >>  >>  >   --> --
>  > >>  >>  >   -->
>  > >>  >>  >   -->
>  > >>  >>  >   -->                                             Gilles.
>  > >>  >>  > [Robert McCullough]
>  > >>  >>  > Thanks.
>  > >>  >>  > Calling "ulimit -s 2048" before my application in my startup script seems to
>  > >>  >>  > work fine when I am running over nfs to the Denx ELDK with a bash shell.
>  > >>  >>  > But my application needs to run from a CompactFlash card running BusyBox on
>  > >>  >>  > a MPC5200 and BusyBox does not seem to support the "ulimit" command.
>  > >>  >>  > Is there another way to set this stack limit size?
>  > >>  >>  >
>  > >>  >>
>  > >>  >>  - Setting the stack size property into the attribute struct for each new pthread
>  > >>  >>  your application may create: pthread_attr_setstacksize(&thattr, stacksize);
>  > >>  >>
>  > >>  >>  - Or passing the right stack size value to any (non-POSIX) Xenomai API task
>  > >>  >>  creation service, we do enforce it (e.g. rt_task_create(), taskInit(),
>  > >>  >>  t_create()...).
>  > >>  >>
>  > >>  >>  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default value for all
>  > >>  >>  subsequent glibc-originated threads. This is what the ulimit command does, actually.
>  > >>  >
>  > >>  > The problem of pthread_attr_setstacksize (or of setting stack size in
>  > >>  > one of the various thread creation services) is that it does not work
>  > >>  > for the main thread stack. So, you have to use ulimit or to call
>  > >>  > setrlimit before running your application, in order to set the main
>  > >>  > thread stack size.
>  > >>  >
>  > >>  > Now about the ulimit builtin in busybox: busybox ash shell supports ulimit.
>  > >>  >
>  > >>
>  > >>  Even if you don't have any shell supporting ulimit, you just need to create a
>  > >>  small launcher that basically forks/setrlimit+execs your application.
>  > > 
>  > > Errr... but you need to call setrlimit before fork, otherwise it is too late...
>  > >
>  > 
>  > #include <sys/resource.h>
>  > #include <stdio.h>
>  > #include <unistd.h>
>  > #include <stdlib.h>
>  > 
>  > int main(int argc, char *argv[])
>  > {
>  > 	if (strcmp(argv[0], "child")) {
>  > 		struct rlimit r = { .rlim_cur = 32768, .rlim_max = 32768 };
>  > 		setrlimit(RLIMIT_STACK, &r);
>  > 		execl(argv[0], "child", NULL);
>  > 	} else {
>  > 		struct rlimit r;
>  > 		if (getrlimit(RLIMIT_STACK, &r) == 0)
>  > 			printf("%s: stack size = %d\n", argv[0], r.rlim_cur);
>  > 	}
>  > 
>  > 	return 0;
>  > }
> 
> I had a doubt about what I wrote, just after writing it, which is why I
> started doing my own testing. But I got distracted by the fact that all
> the main thread stack was no longer commited. Now, I wonder, do you
> think we should try to work around this behaviour in the I-pipe patch ?
> 

I'm not exactly sure that we should care that much about the main thread growing
on demand actually. What we really want is to keep it from having all the stack
space committed at once when mlocking, which is a different issue. Having on
demand main stack would start biting us only if real-time code sits on top of
it, which is not really a common usage pattern, since people tend to start
real-time threads for that purpose.

A simple test here on x86_64 (2.6.22) confirms that main threads are given a
~84Kb stack segment whatever the stack limit setting is, at least /proc/pid/maps
says that.

-- 
Philippe.

Re: [Xenomai-help] mlockall questions

[Robert McCullough]

  --> -----Original Message-----
  --> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
  --> Sent: Thursday, April 10, 2008 5:25 AM
  --> To: philippe.gerum@domain.hid
  --> Cc: rob@domain.hid; xenomai@xenomai.org
  --> Subject: Re: [Xenomai-help] mlockall questions
  --> 
  --> On Thu, Apr 10, 2008 at 2:53 AM, Philippe Gerum
  --> <philippe.gerum@domain.hid> wrote:
  --> >
  --> > Robert McCullough wrote:
  --> >  > Hi Gilles,
  --> >  >
  --> >  >   --> -----Original Message-----
  --> >  >   --> From: Gilles Chanteperdrix
  --> [mailto:gilles.chanteperdrix@xenomai.org]
  --> >  >   --> Sent: Wednesday, April 09, 2008 3:29 PM
  --> >  >   --> To: rob@domain.hid
  --> >  >   --> Cc: xenomai@xenomai.org
  --> >  >   --> Subject: Re: [Xenomai-help] mlockall questions
  --> >  >   -->
  --> >  >   --> Robert McCullough wrote:
  --> >  >   -->  > Hi,
  --> >  >   -->  >
  --> >  >   -->  > I am converting part of a C++ application to real-time.
  --> >  >   -->  >
  --> >  >   -->  > I noticed that all the Xenomai examples call
  --> >  >   -->  > mlockall(MCL_CURRENT|MCL_FUTURE) at the beginning of
  --> main before
  --> >  >   --> creating
  --> >  >   -->  > any real-time threads.
  --> >  >   -->  > Is calling mlockall a requirement when using Xenomai?
  --> >  >   -->
  --> >  >   --> Yes it is.
  --> >  >   -->
  --> >  > [Robert McCullough]
  --> >  > OK.
  --> >  >   -->  >
  --> >  >   -->  > I have added a couple of Xenomai tasks to my C++ app.
  --> >  >   -->  > A) Without calling mlockall my application seems to run
  --> fine but I
  --> >  >   --> noticed
  --> >  >   -->  > in /proc/xenomai/stat that a few page faults occurred.
  --> >  >   -->
  --> >  >   --> Well normally your application should stop with a message
  --> telling you
  --> >  >   --> that you have not call mlockall. Do you happen to install a
  --> handler
  --> >  >   --> for
  --> >  >   --> SIGXCPU ?
  --> >  > [Robert McCullough]
  --> >  > Yes.
  --> >  >   -->
  --> >  >   -->  > B) When calling mlockall at the beginning of my main() I
  --> do not see
  --> >  >   --> any page
  --> >  >   -->  > faults but some of the C++ threads do not start.
  --> >  >   -->  >
  --> >  >   -->  > Why would mlockall cause some threads not to start?
  --> >  >   -->
  --> >  >   --> Well, you should check rt_task_create or pthread_create
  --> return
  --> >  >   --> value. The usual error is that you run out of memory
  --> because of the
  --> >  >   --> stack size problem explained in the TROUBLESHOOTING guide.
  --> >  >   -->
  --> >  >   --> --
  --> >  >   -->
  --> >  >   -->
  --> >  >   -->                                             Gilles.
  --> >  > [Robert McCullough]
  --> >  > Thanks.
  --> >  > Calling "ulimit -s 2048" before my application in my startup
  --> script seems to
  --> >  > work fine when I am running over nfs to the Denx ELDK with a bash
  --> shell.
  --> >  > But my application needs to run from a CompactFlash card running
  --> BusyBox on
  --> >  > a MPC5200 and BusyBox does not seem to support the "ulimit"
  --> command.
  --> >  > Is there another way to set this stack limit size?
  --> >  >
  --> >
  --> >  - Setting the stack size property into the attribute struct for
  --> each new pthread
  --> >  your application may create: pthread_attr_setstacksize(&thattr,
  --> stacksize);
  --> >
  --> >  - Or passing the right stack size value to any (non-POSIX) Xenomai
  --> API task
  --> >  creation service, we do enforce it (e.g. rt_task_create(),
  --> taskInit(),
  --> >  t_create()...).
  --> >
  --> >  - Or calling setrlimit(RLIMIT_STACK, ...) to set a global default
  --> value for all
  --> >  subsequent glibc-originated threads. This is what the ulimit
  --> command does, actually.
  --> 
  --> The problem of pthread_attr_setstacksize (or of setting stack size in
  --> one of the various thread creation services) is that it does not work
  --> for the main thread stack. So, you have to use ulimit or to call
  --> setrlimit before running your application, in order to set the main
  --> thread stack size.
  --> 
  --> Now about the ulimit builtin in busybox: busybox ash shell supports
  --> ulimit.
  --> 
  --> --
  -->  Gilles

[Robert McCullough] 
I was able to update my busybox to the latest version and use ulimit.
Everything is working fine.
Thanks Gilles and Philippe for all the help and addition investigating.

Best regards, 
Robert McCullough