[Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Arnout Vandecappelle]

   Hoi,

   I noticed that when using the native skin, starting an application with
"taskset 2 <my_app>" does not put the application on processor 2.  Diving into
the code, it turns out that calling rt_task_shadow() throws away the thread's
cpuset, and also rt_task_create() looks only at the mode parameter and ignores
the parent thread's affinity.

   Is this intentional?  If yes, that means that using the native skin, there
is no way to set a task's affinity except explicitly specifying T_CPU when
the task is started, right?

   Note, by the way, that calling sched_setaffinity _after_ the task has started
_will_ change the affinity (cfr. switchtest), but any calls _before_ the task
has started are ignored.

   If it is not intentional, I think the attached patch solves the problem.

   Regards,
   Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xenomai-affinity.patch
Type: text/x-patch
Size: 706 bytes
Desc: not available
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20121017/45d52a61/attachment.bin>

Re: [Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Gilles Chanteperdrix]

On 10/17/2012 01:28 PM, Arnout Vandecappelle wrote:
>    Hoi,
> 
>    I noticed that when using the native skin, starting an application with
> "taskset 2 <my_app>" does not put the application on processor 2.  Diving into
> the code, it turns out that calling rt_task_shadow() throws away the thread's
> cpuset,

That is an issue.

> and also rt_task_create() looks only at the mode parameter and ignores
> the parent thread's affinity.

That does not look like an issue. Why should a task inherit its parent's
affinity.

> 
>    Is this intentional?  If yes, that means that using the native skin, there
> is no way to set a task's affinity except explicitly specifying T_CPU when
> the task is started, right?

No, this looks like a different issue.

> 
>    Note, by the way, that calling sched_setaffinity _after_ the task has started
> _will_ change the affinity (cfr. switchtest), but any calls _before_ the task
> has started are ignored.

Well, if the thread is not created, how sched_setaffinity change its
affinity?

-- 
					    Gilles.

Re: [Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Arnout Vandecappelle]

On 17/10/12 14:32, Gilles Chanteperdrix wrote:
> On 10/17/2012 01:28 PM, Arnout Vandecappelle wrote:
>>     Hoi,
>>
>>     I noticed that when using the native skin, starting an application with
>> "taskset 2<my_app>" does not put the application on processor 2.  Diving into
>> the code, it turns out that calling rt_task_shadow() throws away the thread's
>> cpuset,
>
> That is an issue.

  With cpuset, I mean the Linux cpuset, i.e. current->cpus_allowed.  There
may have been some misunderstanding about this.


>> and also rt_task_create() looks only at the mode parameter and ignores
>> the parent thread's affinity.
>
> That does not look like an issue. Why should a task inherit its parent's
> affinity.

  I don't know if it should - it's not specified in the documentation.  But
it is how posix threads behave.

  Also, if the affinity is not inherited, there isn't much point of having
more than one processor in it, because nucleus will just fix the task to
the first cpu of the set (if I understand correctly).


>>     Is this intentional?  If yes, that means that using the native skin, there
>> is no way to set a task's affinity except explicitly specifying T_CPU when
>> the task is started, right?
>
> No, this looks like a different issue.
>
>>
>>     Note, by the way, that calling sched_setaffinity _after_ the task has started
>> _will_ change the affinity (cfr. switchtest), but any calls _before_ the task
>> has started are ignored.
>
> Well, if the thread is not created, how sched_setaffinity change its
> affinity?

  The (shadow) thread is created, but with a different cpuset than the
Linux thread.  Or rather, the cpuset is changed by shadowing the thread.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

Re: [Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Gilles Chanteperdrix]

On 10/17/2012 03:02 PM, Arnout Vandecappelle wrote:
> On 17/10/12 14:32, Gilles Chanteperdrix wrote:
>> On 10/17/2012 01:28 PM, Arnout Vandecappelle wrote:
>>>     Hoi,
>>>
>>>     I noticed that when using the native skin, starting an application with
>>> "taskset 2<my_app>" does not put the application on processor 2.  Diving into
>>> the code, it turns out that calling rt_task_shadow() throws away the thread's
>>> cpuset,
>>
>> That is an issue.
> 
>   With cpuset, I mean the Linux cpuset, i.e. current->cpus_allowed.  There
> may have been some misunderstanding about this.
> 
> 
>>> and also rt_task_create() looks only at the mode parameter and ignores
>>> the parent thread's affinity.
>>
>> That does not look like an issue. Why should a task inherit its parent's
>> affinity.
> 
>   I don't know if it should - it's not specified in the documentation.  But
> it is how posix threads behave.

Ok. That is an argument. But if an affinity is specified by the mode
argument, it should override the parent's affinity.

> 
>   Also, if the affinity is not inherited, there isn't much point of having
> more than one processor in it, because nucleus will just fix the task to
> the first cpu of the set (if I understand correctly).

You understand correctly, Xenomai does not do any automatic migration.
So, there is no point in having more than one cpu in the affinity mask
for a xenomai task.

> 
> 
>>>     Is this intentional?  If yes, that means that using the native skin, there
>>> is no way to set a task's affinity except explicitly specifying T_CPU when
>>> the task is started, right?
>>
>> No, this looks like a different issue.
>>
>>>
>>>     Note, by the way, that calling sched_setaffinity _after_ the task has started
>>> _will_ change the affinity (cfr. switchtest), but any calls _before_ the task
>>> has started are ignored.
>>
>> Well, if the thread is not created, how sched_setaffinity change its
>> affinity?
> 
>   The (shadow) thread is created, but with a different cpuset than the
> Linux thread.  Or rather, the cpuset is changed by shadowing the thread.

Ok, misunderstanding again, for me, when you say "before the task has
started" clearly means that you are talking about a task created with
rt_task_create.

-- 
					    Gilles.

Re: [Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Arnout Vandecappelle]

On 17/10/12 15:10, Gilles Chanteperdrix wrote:
> On 10/17/2012 03:02 PM, Arnout Vandecappelle wrote:
>> On 17/10/12 14:32, Gilles Chanteperdrix wrote:
>>> On 10/17/2012 01:28 PM, Arnout Vandecappelle wrote:
[snip]
>>>> and also rt_task_create() looks only at the mode parameter and ignores
>>>> the parent thread's affinity.
>>>
>>> That does not look like an issue. Why should a task inherit its parent's
>>> affinity.
>>
>>    I don't know if it should - it's not specified in the documentation.  But
>> it is how posix threads behave.
>
> Ok. That is an argument. But if an affinity is specified by the mode
> argument, it should override the parent's affinity.

  OK.  Would you like me to refresh my patch?


[snip]
>>>>      Note, by the way, that calling sched_setaffinity _after_ the task has started
>>>> _will_ change the affinity (cfr. switchtest), but any calls _before_ the task
>>>> has started are ignored.
>>>
>>> Well, if the thread is not created, how sched_setaffinity change its
>>> affinity?
>>
>>    The (shadow) thread is created, but with a different cpuset than the
>> Linux thread.  Or rather, the cpuset is changed by shadowing the thread.
>
> Ok, misunderstanding again, for me, when you say "before the task has
> started" clearly means that you are talking about a task created with
> rt_task_create.

  Yes, that's what I mean.

When you do:

CPU_SET(1, &cpuset);
sched_setaffinity(0, sizeof(cpuset), &cpuset);
rt_task_shadow(&task, "test", 99, 0);

then rt_task_shadow will migrate the thread from cpu 1 to cpu 0,
so it migrates twice overall (assuming Linux had started it on cpu 0).

When you do:

rt_task_shadow(&task, "test", 99, 0);
CPU_SET(1, &cpuset);
sched_setaffinity(0, sizeof(cpuset), &cpuset);

then sched_setaffinity will migrate the thread from cpu 0 to cpu 1.

  My point was that it seems inconsistent that it makes a difference
in which order you shadow the task and you specify which cpu it should
run on.

  Alternatively, you could document in rt_task_shadow that it will
implicitly migrate to the lowest CPU (even if that's not the one we're
currently executing on).


  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

Re: [Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Gilles Chanteperdrix]

On 10/17/2012 06:33 PM, Arnout Vandecappelle wrote:

> On 17/10/12 15:10, Gilles Chanteperdrix wrote:
>> On 10/17/2012 03:02 PM, Arnout Vandecappelle wrote:
>>> On 17/10/12 14:32, Gilles Chanteperdrix wrote:
>>>> On 10/17/2012 01:28 PM, Arnout Vandecappelle wrote:
> [snip]
>>>>> and also rt_task_create() looks only at the mode parameter and ignores
>>>>> the parent thread's affinity.
>>>>
>>>> That does not look like an issue. Why should a task inherit its parent's
>>>> affinity.
>>>
>>>    I don't know if it should - it's not specified in the documentation.  But
>>> it is how posix threads behave.
>>
>> Ok. That is an argument. But if an affinity is specified by the mode
>> argument, it should override the parent's affinity.
> 
>   OK.  Would you like me to refresh my patch?


Yes, please.

> 
> 
> [snip]
>>>>>      Note, by the way, that calling sched_setaffinity _after_ the task has started
>>>>> _will_ change the affinity (cfr. switchtest), but any calls _before_ the task
>>>>> has started are ignored.
>>>>
>>>> Well, if the thread is not created, how sched_setaffinity change its
>>>> affinity?
>>>
>>>    The (shadow) thread is created, but with a different cpuset than the
>>> Linux thread.  Or rather, the cpuset is changed by shadowing the thread.
>>
>> Ok, misunderstanding again, for me, when you say "before the task has
>> started" clearly means that you are talking about a task created with
>> rt_task_create.
> 
>   Yes, that's what I mean.
> 
> When you do:
> 
> CPU_SET(1, &cpuset);
> sched_setaffinity(0, sizeof(cpuset), &cpuset);
> rt_task_shadow(&task, "test", 99, 0);
> 
> then rt_task_shadow will migrate the thread from cpu 1 to cpu 0,
> so it migrates twice overall (assuming Linux had started it on cpu 0).
> 
> When you do:
> 
> rt_task_shadow(&task, "test", 99, 0);
> CPU_SET(1, &cpuset);
> sched_setaffinity(0, sizeof(cpuset), &cpuset);
> 
> then sched_setaffinity will migrate the thread from cpu 0 to cpu 1.
> 
>   My point was that it seems inconsistent that it makes a difference
> in which order you shadow the task and you specify which cpu it should
> run on.


Yes, I got it, we need to fix that too.

Regards.

-- 
                                                                Gilles.

Re: [Xenomai] Native skin rt_task_create/rt_task_shadow doesn't inherit affinity

[Arnout Vandecappelle]

On 17/10/12 21:59, Gilles Chanteperdrix wrote:
>> >
>> >  When you do:
>> >
>> >  CPU_SET(1,&cpuset);
>> >  sched_setaffinity(0, sizeof(cpuset),&cpuset);
>> >  rt_task_shadow(&task, "test", 99, 0);
>> >
>> >  then rt_task_shadow will migrate the thread from cpu 1 to cpu 0,
>> >  so it migrates twice overall (assuming Linux had started it on cpu 0).
>> >
>> >  When you do:
>> >
>> >  rt_task_shadow(&task, "test", 99, 0);
>> >  CPU_SET(1,&cpuset);
>> >  sched_setaffinity(0, sizeof(cpuset),&cpuset);
>> >
>> >  then sched_setaffinity will migrate the thread from cpu 0 to cpu 1.
>> >
>> >     My point was that it seems inconsistent that it makes a difference
>> >  in which order you shadow the task and you specify which cpu it should
>> >  run on.
>
> Yes, I got it, we need to fix that too.

  On second thought, it actually makes more sense if the xenomai nucleus would
keep the thread on the CPU it is currently assigned to (unless the affinity
specifies something else).  In fact, I thought this was already the case...
And I can see code in xnpod_start_thread which does exactly that.  So why does
the first example migrate the task to a different CPU?


  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F