[Xenomai-help] multicore: reserving core(s) for Xenomai

[Xenomai-help] multicore: reserving core(s) for Xenomai

[Henri Roosen]

We'll start experiments running our Xenomai application on a
multi-core CPU (x86/ARM). I'm sure there are some Xenomai users who
already have experience with it.

What I would like to do is to run the Xenomai realtime threads on one
core of the multi-core CPU. And I would like to reserve this core only
for the Xenomai realtime threads.

I'm using the Native API, so I can pass the CPU affinity flags for the
Xenomai threads during task creation/shadow. But how can I make sure
other threads will not make use of the same processor? I cannot call
taskset for every task that is spawned... right?

Are there any idea's on how this configuration could be made easier?
It would be nice if there was a kernel config option to reserve
core(s) for Xenomai that Linux will not use.

Any ideas and help are welcome!

Thanks,
Henri.

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Kim Mathiassen]

Hi,

You might use the kernel option isolcpus. Have a look at

http://www.linuxtopia.org/online_books/linux_kernel/kernel_configuration/re46.html

Best regards,
Kim Mathiassen

On 09/14/2011 02:18 PM, Henri Roosen wrote:
> We'll start experiments running our Xenomai application on a
> multi-core CPU (x86/ARM). I'm sure there are some Xenomai users who
> already have experience with it.
>
> What I would like to do is to run the Xenomai realtime threads on one
> core of the multi-core CPU. And I would like to reserve this core only
> for the Xenomai realtime threads.
>
> I'm using the Native API, so I can pass the CPU affinity flags for the
> Xenomai threads during task creation/shadow. But how can I make sure
> other threads will not make use of the same processor? I cannot call
> taskset for every task that is spawned... right?
>
> Are there any idea's on how this configuration could be made easier?
> It would be nice if there was a kernel config option to reserve
> core(s) for Xenomai that Linux will not use.
>
> Any ideas and help are welcome!
>
> Thanks,
> Henri.
>
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Gilles Chanteperdrix]

On 09/14/2011 02:18 PM, Henri Roosen wrote:
> We'll start experiments running our Xenomai application on a
> multi-core CPU (x86/ARM). I'm sure there are some Xenomai users who
> already have experience with it.
> 
> What I would like to do is to run the Xenomai realtime threads on one
> core of the multi-core CPU. And I would like to reserve this core only
> for the Xenomai realtime threads.
> 
> I'm using the Native API, so I can pass the CPU affinity flags for the
> Xenomai threads during task creation/shadow. But how can I make sure
> other threads will not make use of the same processor? I cannot call
> taskset for every task that is spawned... right?
> 
> Are there any idea's on how this configuration could be made easier?
> It would be nice if there was a kernel config option to reserve
> core(s) for Xenomai that Linux will not use.
> 
> Any ideas and help are welcome!

The kernel has an "isolcpus" option, which allows to specify which cpus
should not be used by Linux.

See Documentation/kernel-parameters.txt in the linux kernel sources for
an explanation.

-- 
					    Gilles.

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Jan Kiszka]

On 2011-09-14 15:13, Gilles Chanteperdrix wrote:
> On 09/14/2011 02:18 PM, Henri Roosen wrote:
>> We'll start experiments running our Xenomai application on a
>> multi-core CPU (x86/ARM). I'm sure there are some Xenomai users who
>> already have experience with it.
>>
>> What I would like to do is to run the Xenomai realtime threads on one
>> core of the multi-core CPU. And I would like to reserve this core only
>> for the Xenomai realtime threads.
>>
>> I'm using the Native API, so I can pass the CPU affinity flags for the
>> Xenomai threads during task creation/shadow. But how can I make sure
>> other threads will not make use of the same processor? I cannot call
>> taskset for every task that is spawned... right?
>>
>> Are there any idea's on how this configuration could be made easier?
>> It would be nice if there was a kernel config option to reserve
>> core(s) for Xenomai that Linux will not use.
>>
>> Any ideas and help are welcome!
> 
> The kernel has an "isolcpus" option, which allows to specify which cpus
> should not be used by Linux.

...or cgroups. isolcpus is much simpler to set up, but less strict (any
process tuning its affinity is able to circumvent this, but not cgroup
isolations).

> 
> See Documentation/kernel-parameters.txt in the linux kernel sources for
> an explanation.

...but be warned that there is no 100% isolation. Quite a few Linux
kernel threads and IPIs are required to continue hitting any online CPU.
Things are improving to reduce those noise Linux-wise, but there is no
100% in sight yet.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Henry Bausley]

Jan does the lack of 100% isolation imply if a Xenomai kernel thread
running on the isolated CPU used to much duty cycle Linux would lock up
since it couldn't get a chance at the isolated cpu.

On Wed, 2011-09-14 at 15:23 +0200, Jan Kiszka wrote:
> On 2011-09-14 15:13, Gilles Chanteperdrix wrote:
> > On 09/14/2011 02:18 PM, Henri Roosen wrote:
> >> We'll start experiments running our Xenomai application on a
> >> multi-core CPU (x86/ARM). I'm sure there are some Xenomai users who
> >> already have experience with it.
> >>
> >> What I would like to do is to run the Xenomai realtime threads on one
> >> core of the multi-core CPU. And I would like to reserve this core only
> >> for the Xenomai realtime threads.
> >>
> >> I'm using the Native API, so I can pass the CPU affinity flags for the
> >> Xenomai threads during task creation/shadow. But how can I make sure
> >> other threads will not make use of the same processor? I cannot call
> >> taskset for every task that is spawned... right?
> >>
> >> Are there any idea's on how this configuration could be made easier?
> >> It would be nice if there was a kernel config option to reserve
> >> core(s) for Xenomai that Linux will not use.
> >>
> >> Any ideas and help are welcome!
> > 
> > The kernel has an "isolcpus" option, which allows to specify which cpus
> > should not be used by Linux.
> 
> ...or cgroups. isolcpus is much simpler to set up, but less strict (any
> process tuning its affinity is able to circumvent this, but not cgroup
> isolations).
> 
> > 
> > See Documentation/kernel-parameters.txt in the linux kernel sources for
> > an explanation.
> 
> ...but be warned that there is no 100% isolation. Quite a few Linux
> kernel threads and IPIs are required to continue hitting any online CPU.
> Things are improving to reduce those noise Linux-wise, but there is no
> 100% in sight yet.
> 
> Jan
> 





Outbound scan for Spam or Virus by Barracuda at Delta Tau

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Jan Kiszka]

On 2011-09-14 16:39, Henry Bausley wrote:
> Jan does the lack of 100% isolation imply if a Xenomai kernel thread
> running on the isolated CPU used to much duty cycle Linux would lock up
> since it couldn't get a chance at the isolated cpu.

Yep - just like some plain Linux RT thread running as CPU hog on a
vanilla kernel with highest prio and rt-group throttling disabled.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Eric Noulard]

2011/9/14 Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>:
> On 09/14/2011 02:18 PM, Henri Roosen wrote:
>> We'll start experiments running our Xenomai application on a
>> multi-core CPU (x86/ARM). I'm sure there are some Xenomai users who
>> already have experience with it.
>>
>> What I would like to do is to run the Xenomai realtime threads on one
>> core of the multi-core CPU. And I would like to reserve this core only
>> for the Xenomai realtime threads.
>>
>> I'm using the Native API, so I can pass the CPU affinity flags for the
>> Xenomai threads during task creation/shadow. But how can I make sure
>> other threads will not make use of the same processor? I cannot call
>> taskset for every task that is spawned... right?
>>
>> Are there any idea's on how this configuration could be made easier?
>> It would be nice if there was a kernel config option to reserve
>> core(s) for Xenomai that Linux will not use.
>>
>> Any ideas and help are welcome!
>
> The kernel has an "isolcpus" option, which allows to specify which cpus
> should not be used by Linux.
>
> See Documentation/kernel-parameters.txt in the linux kernel sources for
> an explanation.

Here is a direct link
http://lxr.linux.no/#linux+v2.6.39/Documentation/kernel-parameters.txt#L1063

Would cpuset work with Xenomai as well?
http://lxr.linux.no/#linux+v2.6.39/Documentation/cgroups/cpusets.txt

Just curious,I did not tried.


-- 
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Henri Roosen]

On Wed, Sep 14, 2011 at 3:24 PM, Eric Noulard <eric.noulard@domain.hid> wrote:
> 2011/9/14 Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>:
>> On 09/14/2011 02:18 PM, Henri Roosen wrote:
>>> We'll start experiments running our Xenomai application on a
>>> multi-core CPU (x86/ARM). I'm sure there are some Xenomai users who
>>> already have experience with it.
>>>
>>> What I would like to do is to run the Xenomai realtime threads on one
>>> core of the multi-core CPU. And I would like to reserve this core only
>>> for the Xenomai realtime threads.
>>>
>>> I'm using the Native API, so I can pass the CPU affinity flags for the
>>> Xenomai threads during task creation/shadow. But how can I make sure
>>> other threads will not make use of the same processor? I cannot call
>>> taskset for every task that is spawned... right?
>>>
>>> Are there any idea's on how this configuration could be made easier?
>>> It would be nice if there was a kernel config option to reserve
>>> core(s) for Xenomai that Linux will not use.
>>>
>>> Any ideas and help are welcome!
>>
>> The kernel has an "isolcpus" option, which allows to specify which cpus
>> should not be used by Linux.
>>
>> See Documentation/kernel-parameters.txt in the linux kernel sources for
>> an explanation.
>
> Here is a direct link
> http://lxr.linux.no/#linux+v2.6.39/Documentation/kernel-parameters.txt#L1063
>
> Would cpuset work with Xenomai as well?
> http://lxr.linux.no/#linux+v2.6.39/Documentation/cgroups/cpusets.txt
>
> Just curious,I did not tried.
>
>
> --
> Erk
> Membre de l'April - « promouvoir et défendre le logiciel libre » -
> http://www.april.org
>

Thanks everyone for the quick and clear answers!
Exactly what I was looking for.

Henri.

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Jeff Weber]

[-- Attachment #1: Type: text/plain, Size: 1506 bytes --]

On Wed, Sep 14, 2011 at 8:13 AM, Gilles Chanteperdrix <
gilles.chanteperdrix@xenomai.org> wrote:

> On 09/14/2011 02:18 PM, Henri Roosen wrote:
> [snip]
> >
> > What I would like to do is to run the Xenomai realtime threads on one
> > core of the multi-core CPU. And I would like to reserve this core only
> > for the Xenomai realtime threads.
> >
> [snip]
> The kernel has an "isolcpus" option, which allows to specify which cpus
> should not be used by Linux.
>
> See Documentation/kernel-parameters.txt in the linux kernel sources for
> an explanation.
>
>
I tried to do this a year ago with the kernel isolcpus option, but could not
get this to work:

>Gilles Chanteperdrix to Jeff, xenomai-help
>Jeff  wrote:
>> What is the best method for segregating Xenomai and Linux by CPU?  I was
>> unable to do this cleanly using /proc/xenomai/affinity for Xenomai,
>> together with the Linux isolcpus boot option.  Details below.
>
>I have no definite answer, I have not checked. What I suspect, however,
>is that when you try and force an affinity for a Xenomai thread, it
>tries and apply this affinity in secondary mode, i.e. as a plain Linux
>thread. So, will fail if the target thread is not in the "isolcpus"
>mask. In other words, a Xenomai thread can not run on a CPU where Linux
>threads can not run.

at the time my config was:
x86 32-bit arch
linux-2.6.34.5
Xenomai-2.5.4
adeos-ipipe-2.6.34.5-x86-2.7-03.patch

Has the Xenomai cpu affinity or Linux isolcpus implementation changed since
then?

Jeff

[-- Attachment #2: Type: text/html, Size: 4181 bytes --]

Re: [Xenomai-help] multicore: reserving core(s) for Xenomai

[Gilles Chanteperdrix]

On 09/14/2011 05:00 PM, Jeff Weber wrote:
> On Wed, Sep 14, 2011 at 8:13 AM, Gilles Chanteperdrix <
> gilles.chanteperdrix@xenomai.org> wrote:
> 
>> On 09/14/2011 02:18 PM, Henri Roosen wrote:
>> [snip]
>>>
>>> What I would like to do is to run the Xenomai realtime threads on one
>>> core of the multi-core CPU. And I would like to reserve this core only
>>> for the Xenomai realtime threads.
>>>
>> [snip]
>> The kernel has an "isolcpus" option, which allows to specify which cpus
>> should not be used by Linux.
>>
>> See Documentation/kernel-parameters.txt in the linux kernel sources for
>> an explanation.
>>
>>
> I tried to do this a year ago with the kernel isolcpus option, but could not
> get this to work:
> 
>> Gilles Chanteperdrix to Jeff, xenomai-help
>> Jeff  wrote:
>>> What is the best method for segregating Xenomai and Linux by CPU?  I was
>>> unable to do this cleanly using /proc/xenomai/affinity for Xenomai,
>>> together with the Linux isolcpus boot option.  Details below.
>>
>> I have no definite answer, I have not checked. What I suspect, however,
>> is that when you try and force an affinity for a Xenomai thread, it
>> tries and apply this affinity in secondary mode, i.e. as a plain Linux
>> thread. So, will fail if the target thread is not in the "isolcpus"
>> mask. In other words, a Xenomai thread can not run on a CPU where Linux
>> threads can not run.

This explanation sounds like crap: looking at the documentation, it
seems that when using the "isolcpus" option, threads can run on an
isolated cpu if they force their affinity.

-- 
                                                                Gilles.