[Xenomai-help] isolate realtime and Linux by CPU on SMP system

[Xenomai-help] isolate realtime and Linux by CPU on SMP system

[Jeff Weber]

[-- Attachment #1: Type: text/plain, Size: 1700 bytes --]

I have a SMP system with 2 CPUs, and wish to assign all Linux functionality
(processes, threads, interrupts, etc)  to one CPU, and all Xenomai, RTnet
functionality to the other CPU.  Global solutions for isolating each domain
would be preferrable to walking lists of processes, threads, etc.

Initially, writing to /proc/xenomai/affinity appears to isolate all Xenomai
threads to the desired CPU.  (I haven't had an opportunity to test Xenomai
interrupts yet.)

Then I encountered difficulties when starting to isolate Linux processes,
threads.  Using the kernel boot option isolcpus=0 resulted in 49 out of 126
Linux processes running on CPU0.

Switching instead to boot option isolcpus=1 placed 11 out of 126 Linux
processes on CPU1, the remainder on CPU0  The processes running on CPU1 were
migration/1, ksoftirqd/1, watchdog/1, etc, so this may be the bare minimum
possible on CPU1.  I considered this "good enough".

Then I attempted to set Xenomai global affinity to CPU1:
echo 00000002 > /proc/xenomai/affinity
But running my Xenomai Posix skin application failed:
Xenomai Posix skin init: pthread_setschedparam: Invalid argument

Interestingly, switching Xenomai global affinity to CPU0 then succeeded:
echo 00000001 > /proc/xenomai/affinity
But my Xenomai threads then ran on CPU0 alongside the bulk of the Linux
processes.

I was able to assign all Linux interrupts to either CPU, without any
difficulties, by manipulating the IRQBALANCE_BANNED_CPUS environment for the
irqbalance daemon.

What is the best way to assign Linux to use one CPU, and Xenomai, RTnet to
use the other CPU?

my config:
x86 32-bit arch
linux-2.6.34.5
Xenomai-2.5.4
adeos-ipipe-2.6.34.5-x86-2.7-03.patch

TIA,
Jeff

[-- Attachment #2: Type: text/html, Size: 1822 bytes --]

[Xenomai-help] Fwd: isolate realtime and Linux by CPU on SMP system

[Jeff Weber]

[-- Attachment #1: Type: text/plain, Size: 2110 bytes --]

What is the best method for segregating Xenomai and Linux by CPU?  I was
unable to do this cleanly using /proc/xenomai/affinity for Xenomai, together
with the Linux isolcpus boot option.  Details below.

TIA,
Jeff

---------- Forwarded message ----------
From: Jeff Weber <jwamsc@domain.hid>
Date: Wed, Sep 15, 2010 at 4:40 PM
Subject: isolate realtime and Linux by CPU on SMP system
To: xenomai@xenomai.org


I have a SMP system with 2 CPUs, and wish to assign all Linux functionality
(processes, threads, interrupts, etc)  to one CPU, and all Xenomai, RTnet
functionality to the other CPU.  Global solutions for isolating each domain
would be preferrable to walking lists of processes, threads, etc.

Initially, writing to /proc/xenomai/affinity appears to isolate all Xenomai
threads to the desired CPU.  (I haven't had an opportunity to test Xenomai
interrupts yet.)

Then I encountered difficulties when starting to isolate Linux processes,
threads.  Using the kernel boot option isolcpus=0 resulted in 49 out of 126
Linux processes running on CPU0.

Switching instead to boot option isolcpus=1 placed 11 out of 126 Linux
processes on CPU1, the remainder on CPU0  The processes running on CPU1 were
migration/1, ksoftirqd/1, watchdog/1, etc, so this may be the bare minimum
possible on CPU1.  I considered this "good enough".

Then I attempted to set Xenomai global affinity to CPU1:
echo 00000002 > /proc/xenomai/affinity
But running my Xenomai Posix skin application failed:
Xenomai Posix skin init: pthread_setschedparam: Invalid argument

Interestingly, switching Xenomai global affinity to CPU0 then succeeded:
echo 00000001 > /proc/xenomai/affinity
But my Xenomai threads then ran on CPU0 alongside the bulk of the Linux
processes.

I was able to assign all Linux interrupts to either CPU, without any
difficulties, by manipulating the IRQBALANCE_BANNED_CPUS environment for the
irqbalance daemon.

What is the best way to assign Linux to use one CPU, and Xenomai, RTnet to
use the other CPU?

my config:
x86 32-bit arch
linux-2.6.34.5
Xenomai-2.5.4
adeos-ipipe-2.6.34.5-x86-2.7-03.patch

TIA,
Jeff

[-- Attachment #2: Type: text/html, Size: 2491 bytes --]

Re: [Xenomai-help] Fwd: isolate realtime and Linux by CPU on SMP system

[Gilles Chanteperdrix]

Jeff Weber wrote:
> What is the best method for segregating Xenomai and Linux by CPU?  I was
> unable to do this cleanly using /proc/xenomai/affinity for Xenomai,
> together with the Linux isolcpus boot option.  Details below.

I have no definite answer, I have not checked. What I suspect, however,
is that when you try and force an affinity for a Xenomai thread, it
tries and apply this affinity in secondary mode, i.e. as a plain Linux
thread. So, will fail if the target thread is not in the "isolcpus"
mask. In other words, a Xenomai thread can not run on a CPU where Linux
threads can not run.

-- 
					    Gilles.

Re: [Xenomai-help] Fwd: isolate realtime and Linux by CPU on SMP system

[Jeff Weber]

[-- Attachment #1: Type: text/plain, Size: 1020 bytes --]

On Tue, Sep 21, 2010 at 9:44 AM, Gilles Chanteperdrix <
gilles.chanteperdrix@xenomai.org> wrote:

> Jeff Weber wrote:
> > What is the best method for segregating Xenomai and Linux by CPU?  I was
> > unable to do this cleanly using /proc/xenomai/affinity for Xenomai,
> > together with the Linux isolcpus boot option.  Details below.
>
> I have no definite answer, I have not checked. What I suspect, however,
> is that when you try and force an affinity for a Xenomai thread, it
> tries and apply this affinity in secondary mode, i.e. as a plain Linux
> thread. So, will fail if the target thread is not in the "isolcpus"
> mask. In other words, a Xenomai thread can not run on a CPU where Linux
> threads can not run.
>
> Is there a means to startup the Linux init process, with the constraint of
only running on a given CPU, and have this property inherited by all
children Linux processes?

If true, then I am hopeful I can place all Xenomai primary mode threads on
the other CPU using /proc/xenomai/affinity .

Jeff

[-- Attachment #2: Type: text/html, Size: 1383 bytes --]