[Xenomai-help] Running out of stack memory in kernel-space

[Xenomai-help] Running out of stack memory in kernel-space

[Andreas Glatz]

Hi,

We have to start 40+ Xenomai tasks in kernel space.
We want to allocate at least 4kbyte of stack memory
for each of them. So we need the stack heap size 
to be more than 160kbyte. 

In the documentation it says that the total 
stack heap size cannot be more than 128kbyte
(the limit of kmalloc).

Now my idea is to modify the Xenomai code to have 
more than one stack heaps. So when I'm running out
of space in one heap I can switch to the next heap...

Is there an easier way to increase the max. stack heap size?

Best regards, 
Andreas 

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Philippe Gerum]

On Thu, 2009-05-28 at 11:59 -0400, Andreas Glatz wrote:
> Hi,
> 
> We have to start 40+ Xenomai tasks in kernel space.
> We want to allocate at least 4kbyte of stack memory
> for each of them. So we need the stack heap size 
> to be more than 160kbyte. 
> 
> In the documentation it says that the total 
> stack heap size cannot be more than 128kbyte
> (the limit of kmalloc).

Could you point me at that doc? TIA,

> 
> Now my idea is to modify the Xenomai code to have 
> more than one stack heaps. So when I'm running out
> of space in one heap I can switch to the next heap...
> 
> Is there an easier way to increase the max. stack heap size?
> 

See CONFIG_XENO_OPT_SYS_STACKPOOLSZ, nucleus configuration menu.

> Best regards, 
> Andreas 
> 
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
-- 
Philippe.

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Philippe Gerum]

On Thu, 2009-05-28 at 11:59 -0400, Andreas Glatz wrote:
> Hi,
> 
> We have to start 40+ Xenomai tasks in kernel space.

Why subjecting yourself to the pain of running a complex application in
kernel space? The difference between kernel and userland latencies on an
mpc836x should not be worth it.

> We want to allocate at least 4kbyte of stack memory
> for each of them. So we need the stack heap size 
> to be more than 160kbyte. 
> 
> In the documentation it says that the total 
> stack heap size cannot be more than 128kbyte
> (the limit of kmalloc).
> 
> Now my idea is to modify the Xenomai code to have 
> more than one stack heaps. So when I'm running out
> of space in one heap I can switch to the next heap...
> 
> Is there an easier way to increase the max. stack heap size?
> 
> Best regards, 
> Andreas 
> 
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
-- 
Philippe.

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Andreas Glatz]

On Thu, 2009-05-28 at 18:52 +0200, Philippe Gerum wrote:
> On Thu, 2009-05-28 at 11:59 -0400, Andreas Glatz wrote:
> > Hi,
> > 
> > We have to start 40+ Xenomai tasks in kernel space.
> 
> Why subjecting yourself to the pain of running a complex application in
> kernel space? The difference between kernel and userland latencies on an
> mpc836x should not be worth it.
> 

We see a 70us difference between running exactly the same
application in kernel-space and user-space. The application
registers the rx and tx interrupt of one UEC and replaces
the buffer descriptor rings allocated by Linux, and 
takes care of the buffer handling. During the
test we send out an Ethernet frame from the MPC8360, the
frame gets processed on the other end of the wire and 
eventually we receive the response from the other device. 
The total round trip time (function call send() to 
return of function call receive() measured with __xn_rdtsc()) 
is 230us in kernel-space and 300us in user-space.

FYI: Another device which used direct SMI instead of Ethernet
to talk to the device on the other end of the wire showed a
round trip time of 40us... 


Andreas

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Andreas Glatz]

On Thu, 2009-05-28 at 18:39 +0200, Philippe Gerum wrote:
> On Thu, 2009-05-28 at 11:59 -0400, Andreas Glatz wrote:
> > Hi,
> > 
> > We have to start 40+ Xenomai tasks in kernel space.
> > We want to allocate at least 4kbyte of stack memory
> > for each of them. So we need the stack heap size 
> > to be more than 160kbyte. 
> > 
> > In the documentation it says that the total 
> > stack heap size cannot be more than 128kbyte
> > (the limit of kmalloc).
> 
> Could you point me at that doc? TIA,
> 
> > 
> > Now my idea is to modify the Xenomai code to have 
> > more than one stack heaps. So when I'm running out
> > of space in one heap I can switch to the next heap...
> > 
> > Is there an easier way to increase the max. stack heap size?
> > 
> 
> See CONFIG_XENO_OPT_SYS_STACKPOOLSZ, nucleus configuration menu.

I found this in kernel/xenomai/nucleus/Kconfig :

config XENO_OPT_SYS_STACKPOOLSZ
        depends on XENO_GENERIC_STACKPOOL
    int "Size of the private stack pool (Kb)"
    default 32
    range 0 128
    help

    On this architecture, kernel-based Xenomai threads get the
    stack space they need from a private memory pool.  If you
    don't start any kernel-based thread (i.e. no RTDM driver
    thread, and no real-time task created from an application
    embodied into a kernel module), you may leave a zero value for
    this option. The size is expressed in Kilobytes.


And if you follow the CONFIG_XENO_OPT_SYS_STACKPOOLSZ macro 
you eventually end up at this comment in kernel/xenomai/nucleus/pod.c :

    /*
     * We have to differentiate the system heap memory from the
     * pool the kernel thread stacks will be obtained from,
     * because on some architectures, vmalloc memory may not be
     * accessed while running in physical addressing mode
     * (e.g. exception trampoline code on powerpc with standard
     * MMU support - CONFIG_PPC_STD_MMU). Meanwhile, since we want
     * to allow the system heap to be larger than 128Kb in
     * contiguous memory, we can't restrict to using kmalloc()
     * memory for it either.  Therefore, we manage a private stack
     * pool for kernel-based threads which will be populated with
     * the kind of memory the underlying arch requires, still
     * allowing the system heap to rely on a vmalloc'ed segment.
     */
    heapaddr = xnarch_alloc_stack_mem(CONFIG_XENO_OPT_SYS_STACKPOOLSZ *
1024);



Andreas

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Philippe Gerum]

On Thu, 2009-05-28 at 14:04 -0400, Andreas Glatz wrote:
> On Thu, 2009-05-28 at 18:52 +0200, Philippe Gerum wrote:
> > On Thu, 2009-05-28 at 11:59 -0400, Andreas Glatz wrote:
> > > Hi,
> > > 
> > > We have to start 40+ Xenomai tasks in kernel space.
> > 
> > Why subjecting yourself to the pain of running a complex application in
> > kernel space? The difference between kernel and userland latencies on an
> > mpc836x should not be worth it.
> > 
> 
> We see a 70us difference between running exactly the same
> application in kernel-space and user-space. The application
> registers the rx and tx interrupt of one UEC and replaces
> the buffer descriptor rings allocated by Linux, and 
> takes care of the buffer handling. During the
> test we send out an Ethernet frame from the MPC8360, the
> frame gets processed on the other end of the wire and 
> eventually we receive the response from the other device. 
> The total round trip time (function call send() to 
> return of function call receive() measured with __xn_rdtsc()) 
> is 230us in kernel-space and 300us in user-space.
> 

A 70 us difference is pathological, this does not follow the common
latency ratio between kernel-based and userland tasking, really. If you
still have your userland test at hand, you may want to
display /proc/xenomai/stat periodically while it runs, checking whether
any of your communication task has a suspicious increase of its MSW
counter.

The other aspect, if I understand correctly, may involve the method you
picked for transferring data back and forth between your MAC layer and
the tasks running in userland. Message queues for instance would
probably not be the best approach in your case, since you would likely
pay extra data copies due to the different address spaces.

> 
> FYI: Another device which used direct SMI instead of Ethernet
> to talk to the device on the other end of the wire showed a
> round trip time of 40us... 
> 

What bothers me is that you seem to determine a latency based on
measurements of a round trip time, obtained on an ethernet network. So
it is a bit difficult for me to give any interpretation about the 70 us
figure; too many variables in that equation.

> 
> Andreas
-- 
Philippe.

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Philippe Gerum]

On Thu, 2009-05-28 at 14:23 -0400, Andreas Glatz wrote:
> On Thu, 2009-05-28 at 18:39 +0200, Philippe Gerum wrote:
> > On Thu, 2009-05-28 at 11:59 -0400, Andreas Glatz wrote:
> > > Hi,
> > > 
> > > We have to start 40+ Xenomai tasks in kernel space.
> > > We want to allocate at least 4kbyte of stack memory
> > > for each of them. So we need the stack heap size 
> > > to be more than 160kbyte. 
> > > 
> > > In the documentation it says that the total 
> > > stack heap size cannot be more than 128kbyte
> > > (the limit of kmalloc).
> > 
> > Could you point me at that doc? TIA,
> > 
> > > 
> > > Now my idea is to modify the Xenomai code to have 
> > > more than one stack heaps. So when I'm running out
> > > of space in one heap I can switch to the next heap...
> > > 
> > > Is there an easier way to increase the max. stack heap size?
> > > 
> > 
> > See CONFIG_XENO_OPT_SYS_STACKPOOLSZ, nucleus configuration menu.
> 
> I found this in kernel/xenomai/nucleus/Kconfig :
> 
> config XENO_OPT_SYS_STACKPOOLSZ
>         depends on XENO_GENERIC_STACKPOOL
>     int "Size of the private stack pool (Kb)"
>     default 32
>     range 0 128
>     help
> 
>     On this architecture, kernel-based Xenomai threads get the
>     stack space they need from a private memory pool.  If you
>     don't start any kernel-based thread (i.e. no RTDM driver
>     thread, and no real-time task created from an application
>     embodied into a kernel module), you may leave a zero value for
>     this option. The size is expressed in Kilobytes.
> 

Ok, you must be running a Xenomai release prior to 2.4.7 where this
useless limitation was lifted. I would really, really, really, really,
indeed, I mean, no kidding, I swear, life will be good after this,
recommend (*) that you upgrade to 2.4.8 when it is available.

For the time being, you could just get rid of the 128k limit with that
patch:

--- ksrc/nucleus/Kconfig~	2008-11-09 18:54:04.000000000 +0100
+++ ksrc/nucleus/Kconfig	2009-05-29 00:10:05.000000000 +0200
@@ -145,7 +145,6 @@
         depends on XENO_GENERIC_STACKPOOL
 	int "Size of the private stack pool (Kb)"
 	default 32
-	range 0 128
 	help
 
 	On this architecture, kernel-based Xenomai threads get the

(*) no, seriously.

> 
> And if you follow the CONFIG_XENO_OPT_SYS_STACKPOOLSZ macro 
> you eventually end up at this comment in kernel/xenomai/nucleus/pod.c :
> 
>     /*
>      * We have to differentiate the system heap memory from the
>      * pool the kernel thread stacks will be obtained from,
>      * because on some architectures, vmalloc memory may not be
>      * accessed while running in physical addressing mode
>      * (e.g. exception trampoline code on powerpc with standard
>      * MMU support - CONFIG_PPC_STD_MMU). Meanwhile, since we want
>      * to allow the system heap to be larger than 128Kb in
>      * contiguous memory, we can't restrict to using kmalloc()
>      * memory for it either.  Therefore, we manage a private stack
>      * pool for kernel-based threads which will be populated with
>      * the kind of memory the underlying arch requires, still
>      * allowing the system heap to rely on a vmalloc'ed segment.
>      */
>     heapaddr = xnarch_alloc_stack_mem(CONFIG_XENO_OPT_SYS_STACKPOOLSZ *
> 1024);

I will fix this leftover in 2.4.8, thanks.

> 
> 
> 
> Andreas
> 
-- 
Philippe.

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Andreas Glatz]

[-- Attachment #1: Type: text/plain, Size: 3374 bytes --]

Hi,


> What bothers me is that you seem to determine a latency based on
> measurements of a round trip time, obtained on an ethernet network. So
> it is a bit difficult for me to give any interpretation about the 70 us
> figure; too many variables in that equation.
> 
> > 


I finally upgraded Xenomai from version 2.4.4 to 2.4.8
as you strongly suggested last time and I can say 
that this fixes all of the bugs we found in the earlier 
version. Thanks a lot for that!!

Furthermore I did some serious measurements with the new
Xenomai version on an MPC8360. In short, I measured
an interrupt latency around 7us when running in kernel-space
and around 35us when running in user-space. You can find
the details of those measurements the end of this Email!

I have two more questions:

1) In the legacy code I am porting, critical sections are
protected by disabling/enabling ALL interrupts (like
sti/cli in kernel-space). This method is much faster than
locking a mutex, which might put a task to sleep. Are there
similar, portable functions available under Xenomai or IPIPE?

2) In the legacy code we also have some low priority
tasks which are performing lengthy operations (around 1s).
Of course, those low-priority tasks prevent Linux tasks
from running. In our case this behaviour is bad. Just in 
the case of those low-priority task, Linux should be able 
to preempt those tasks. From our viewpoint, the best solution
to that problem would be if we could assign those tasks 
the same or a lower priority than Linux tasks. That would
give us the best of both worlds: We could call Xenomai 
API functions from those task without switching back and
forth between primary and secondary mode and Linux tasks 
wouldn't be preempted.


Thanks again to you and the other Xenomai contributors 
for producing such a good piece of Software!! I hope 
that you'll keep the good work up. I'm also looking
forward trying out Version 2.5!


Best regards,
Andreas Glatz









---------------------------------------------------------

The test setup is shown in file "testsetup.pdf". 

We are running a Xenomai application on the MPC8360 
which grabs the UEC 7 IRQ and replaces the RX and TX 
buffers and buffer rings of the Linux UEC driver by its 
own RX and TX buffers and buffer rings. Those buffers are 
allocated from a RT_HEAP object which was created with 
the H_DMA flag. The Xenomai application is highly portable 
between kernel- and user-space. The only differences between
the kernel-space and user-space version is the way they
are linked (shared libraries vs exported functions).

The Xenomai application periodically sends a 64byte 
Ethernet frame over the 100Mbit MII Interface to the 
Ethernet Switch where the Frame is looped back on
Port 1. The frame travels back over the MII interface 
to the MPC8360 and is received by the Xenomai application.

As described in "testsetup.pdf" we measured the following
signals with an Oscilloscope: Rx_DV, Tx_EN, GPIO Pin.

The GPIO Pin is toggled every time the UEC 7 ISR is called.
The UEC 7 Interrupt is triggered after Rx_DV and Tx_EN 
go from high to low. So the time between the neg. slope of
Rx_DV and Tx_EN and a Pin state change is the equal to 
the interrupt delay. See "loopks.pdf" and "loopus.pdf"
for detailed results. 

[-- Attachment #2: testsetup.pdf --]
[-- Type: application/pdf, Size: 30527 bytes --]

[-- Attachment #3: loopks.pdf --]
[-- Type: application/pdf, Size: 77576 bytes --]

[-- Attachment #4: loopus.pdf --]
[-- Type: application/pdf, Size: 76722 bytes --]

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Philippe Gerum]

On Mon, 2009-06-22 at 09:23 -0400, Andreas Glatz wrote:
> Hi,
> 
> 
> > What bothers me is that you seem to determine a latency based on
> > measurements of a round trip time, obtained on an ethernet network. So
> > it is a bit difficult for me to give any interpretation about the 70 us
> > figure; too many variables in that equation.
> > 
> > > 
> 
> 
> I finally upgraded Xenomai from version 2.4.4 to 2.4.8
> as you strongly suggested last time and I can say 
> that this fixes all of the bugs we found in the earlier 
> version. Thanks a lot for that!!
> 
> Furthermore I did some serious measurements with the new
> Xenomai version on an MPC8360. In short, I measured
> an interrupt latency around 7us when running in kernel-space
> and around 35us when running in user-space. You can find
> the details of those measurements the end of this Email!
> 
> I have two more questions:
> 
> 1) In the legacy code I am porting, critical sections are
> protected by disabling/enabling ALL interrupts (like
> sti/cli in kernel-space). This method is much faster than
> locking a mutex, which might put a task to sleep.

Usually yes, but at the expense of wrecking the priority-based model of
the RTOS when misused.

>  Are there
> similar, portable functions available under Xenomai or IPIPE?

You should never rely directly on I-pipe features, using a Xenomai
interface is the way to go.

If writing driver code based on RTDM (the recommended way), see there:
http://www.xenomai.org/documentation/branches/v2.4.x/html/api/group__rtdmsync.html

Otherwise, if you are writing ad hoc application code in kernel space,
which is asking for trouble (yeah, I can't help it, sorry), you will
have to resort to core services from the Xenomai nucleus directly: see
splhigh(), splexit(), xnlock_get[irqsave], xnlock_put[_irqrestore].
Regarding the second form, you may notice a pre-defined lock called
"nklock", which is used by the nucleus internally. You may use it at
your own risk (basically introducing more latency if misused), but you
may also define your own RT spinlock based on the xnlock_t type, taking
care of possible locking order issues with the nklock.
see include/asm-generic/system.h.

Disclaimer: I never gave you any information about how to write sloppy
code abusing design layers, just like I did. This is not me writing
this. This information shall not be known outside of the Internet.
If, despite all due diligence from my side, you happen to receive this
information, this is strictly unwanted.

> 
> 2) In the legacy code we also have some low priority
> tasks which are performing lengthy operations (around 1s).
> Of course, those low-priority tasks prevent Linux tasks
> from running. In our case this behaviour is bad. Just in 
> the case of those low-priority task, Linux should be able 
> to preempt those tasks. From our viewpoint, the best solution
> to that problem would be if we could assign those tasks 
> the same or a lower priority than Linux tasks. That would
> give us the best of both worlds: We could call Xenomai 
> API functions from those task without switching back and
> forth between primary and secondary mode and Linux tasks 
> wouldn't be preempted.

You could either set your scheduling parameters to SCHED_RR to relax the
system once in while, given that this would only approximate what you
want to do, or you could upgrade to 2.5, which provides a sporadic
server scheduling class. In that case, you could fine tune the amount of
runtime devoted to any given task of your system, within a given period
of time.

> 
> 
> Thanks again to you and the other Xenomai contributors 
> for producing such a good piece of Software!! I hope 
> that you'll keep the good work up. I'm also looking
> forward trying out Version 2.5!
> 
> 
> Best regards,
> Andreas Glatz
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ---------------------------------------------------------
> 
> The test setup is shown in file "testsetup.pdf". 
> 
> We are running a Xenomai application on the MPC8360 
> which grabs the UEC 7 IRQ and replaces the RX and TX 
> buffers and buffer rings of the Linux UEC driver by its 
> own RX and TX buffers and buffer rings. Those buffers are 
> allocated from a RT_HEAP object which was created with 
> the H_DMA flag. The Xenomai application is highly portable 
> between kernel- and user-space. The only differences between
> the kernel-space and user-space version is the way they
> are linked (shared libraries vs exported functions).
> 
> The Xenomai application periodically sends a 64byte 
> Ethernet frame over the 100Mbit MII Interface to the 
> Ethernet Switch where the Frame is looped back on
> Port 1. The frame travels back over the MII interface 
> to the MPC8360 and is received by the Xenomai application.
> 
> As described in "testsetup.pdf" we measured the following
> signals with an Oscilloscope: Rx_DV, Tx_EN, GPIO Pin.
> 
> The GPIO Pin is toggled every time the UEC 7 ISR is called.
> The UEC 7 Interrupt is triggered after Rx_DV and Tx_EN 
> go from high to low. So the time between the neg. slope of
> Rx_DV and Tx_EN and a Pin state change is the equal to 
> the interrupt delay. See "loopks.pdf" and "loopus.pdf"
> for detailed results. 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
-- 
Philippe.

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Gilles Chanteperdrix]

Philippe Gerum wrote:
> On Mon, 2009-06-22 at 09:23 -0400, Andreas Glatz wrote:
>> 1) In the legacy code I am porting, critical sections are
>> protected by disabling/enabling ALL interrupts (like
>> sti/cli in kernel-space). This method is much faster than
>> locking a mutex, which might put a task to sleep.
> 
> Usually yes, but at the expense of wrecking the priority-based model of
> the RTOS when misused.
> 
>>  Are there
>> similar, portable functions available under Xenomai or IPIPE?
> 
> You should never rely directly on I-pipe features, using a Xenomai
> interface is the way to go.
> 
> If writing driver code based on RTDM (the recommended way), see there:
> http://www.xenomai.org/documentation/branches/v2.4.x/html/api/group__rtdmsync.html
> 
> Otherwise, if you are writing ad hoc application code in kernel space,
> which is asking for trouble (yeah, I can't help it, sorry), you will
> have to resort to core services from the Xenomai nucleus directly: see
> splhigh(), splexit(), xnlock_get[irqsave], xnlock_put[_irqrestore].
> Regarding the second form, you may notice a pre-defined lock called
> "nklock", which is used by the nucleus internally. You may use it at
> your own risk (basically introducing more latency if misused), but you
> may also define your own RT spinlock based on the xnlock_t type, taking
> care of possible locking order issues with the nklock.
> see include/asm-generic/system.h.

The rule of thumb is: if you need to reschedule while holding an xnlock,
then this xnlock must be the nklock. Otherwise, you will be in trouble.

-- 
                                          Gilles

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Philippe Gerum]

On Tue, 2009-06-23 at 11:37 +0200, Gilles Chanteperdrix wrote:
> Philippe Gerum wrote:
> > On Mon, 2009-06-22 at 09:23 -0400, Andreas Glatz wrote:
> >> 1) In the legacy code I am porting, critical sections are
> >> protected by disabling/enabling ALL interrupts (like
> >> sti/cli in kernel-space). This method is much faster than
> >> locking a mutex, which might put a task to sleep.
> > 
> > Usually yes, but at the expense of wrecking the priority-based model of
> > the RTOS when misused.
> > 
> >>  Are there
> >> similar, portable functions available under Xenomai or IPIPE?
> > 
> > You should never rely directly on I-pipe features, using a Xenomai
> > interface is the way to go.
> > 
> > If writing driver code based on RTDM (the recommended way), see there:
> > http://www.xenomai.org/documentation/branches/v2.4.x/html/api/group__rtdmsync.html
> > 
> > Otherwise, if you are writing ad hoc application code in kernel space,
> > which is asking for trouble (yeah, I can't help it, sorry), you will
> > have to resort to core services from the Xenomai nucleus directly: see
> > splhigh(), splexit(), xnlock_get[irqsave], xnlock_put[_irqrestore].
> > Regarding the second form, you may notice a pre-defined lock called
> > "nklock", which is used by the nucleus internally. You may use it at
> > your own risk (basically introducing more latency if misused), but you
> > may also define your own RT spinlock based on the xnlock_t type, taking
> > care of possible locking order issues with the nklock.
> > see include/asm-generic/system.h.
> 
> The rule of thumb is: if you need to reschedule while holding an xnlock,
> then this xnlock must be the nklock. Otherwise, you will be in trouble.

Actually, the real rule of thumb is: do not call into any Xenomai
services holding this lock, unless you really know what you are doing.

> 
-- 
Philippe.

Re: [Xenomai-help] Running out of stack memory in kernel-space

[Philippe Gerum]

On Tue, 2009-06-23 at 11:30 +0200, Philippe Gerum wrote:
> On Mon, 2009-06-22 at 09:23 -0400, Andreas Glatz wrote:
> > Hi,
> > 
> > 
> > > What bothers me is that you seem to determine a latency based on
> > > measurements of a round trip time, obtained on an ethernet network. So
> > > it is a bit difficult for me to give any interpretation about the 70 us
> > > figure; too many variables in that equation.
> > > 
> > > > 
> > 
> > 
> > I finally upgraded Xenomai from version 2.4.4 to 2.4.8
> > as you strongly suggested last time and I can say 
> > that this fixes all of the bugs we found in the earlier 
> > version. Thanks a lot for that!!
> > 
> > Furthermore I did some serious measurements with the new
> > Xenomai version on an MPC8360. In short, I measured
> > an interrupt latency around 7us when running in kernel-space
> > and around 35us when running in user-space. You can find
> > the details of those measurements the end of this Email!
> > 
> > I have two more questions:
> > 
> > 1) In the legacy code I am porting, critical sections are
> > protected by disabling/enabling ALL interrupts (like
> > sti/cli in kernel-space). This method is much faster than
> > locking a mutex, which might put a task to sleep.
> 
> Usually yes, but at the expense of wrecking the priority-based model of
> the RTOS when misused.
> 
> >  Are there
> > similar, portable functions available under Xenomai or IPIPE?
> 
> You should never rely directly on I-pipe features, using a Xenomai
> interface is the way to go.

I may have misunderstood your question actually: were you asking for a
way to block interrupts for user-space? If so, then forget about it.
There is no way to do this sanely, and safely. You should use mutexes;
2.5 even supports fastsynchs underneath, which could be roughly compared
as Xenomai's counterparts of Linux futexes.

-- 
Philippe.