[Xenomai-help] New scheduler class

[Xenomai-help] New scheduler class

[Andreas Glatz]

Hi Philippe,

At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).

You suggested that in 2.5 you would introduce scheduling classes and by writing a special scheduling class we would be able to do that.

I took a look at the 2.5 source code and found 4 different scheduling classes but not a lot of documentation :)

Is there any documentation available?
What does sched_tp.c implement? Partition scheduling?

Best regards,

Andreas 

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 573 bytes --]

Andreas Glatz wrote:
> Hi Philippe,
> 
> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
> 

What prevents using a borderline thread (if you need to interact with
blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Andreas Glatz wrote:
>> Hi Philippe,
>>
>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>
> 
> What prevents using a borderline thread (if you need to interact with
> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?

Well, this does not really guarantee that the thread will run only when
linux is idle. The thread will eat some cpu time, the nice level is not
a strict priority, as you know. But in fact, I wonder why Andreas wants
a new scheduling policy for xenomai, what is needed, is simply a
SCHED_IDLE (maybe it exists ?!) policy for Linux.

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

We have the same requirement for which we also have no solution. One
of our threads has a configurable priority of which at the lowest
priority the thread should behave like an idle task that is only
scheduled when all xenomai threads are idle and also the Linux domain
is idle. The thread might use rtdm-drivers and xenomai mutexes and
therefore has to be moved to the Xenomai domain. But then it always
has higher priority than any threads scheduled in linux.

Actually, when seen from the first domain, this requirement would be
solved when we could set a first domain priority for the second
domain... Gilles, is that what is meant by the SCHED_IDLE policy for
Linux?


On Fri, Apr 2, 2010 at 2:11 AM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> Jan Kiszka wrote:
>> Andreas Glatz wrote:
>>> Hi Philippe,
>>>
>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>
>>
>> What prevents using a borderline thread (if you need to interact with
>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>
> Well, this does not really guarantee that the thread will run only when
> linux is idle. The thread will eat some cpu time, the nice level is not
> a strict priority, as you know. But in fact, I wonder why Andreas wants
> a new scheduling policy for xenomai, what is needed, is simply a
> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>
> --
>                                            Gilles.
>
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
>

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Henri Roosen wrote:
> We have the same requirement for which we also have no solution. One
> of our threads has a configurable priority of which at the lowest
> priority the thread should behave like an idle task that is only
> scheduled when all xenomai threads are idle and also the Linux domain
> is idle. The thread might use rtdm-drivers and xenomai mutexes and
> therefore has to be moved to the Xenomai domain. But then it always
> has higher priority than any threads scheduled in linux.
> 
> Actually, when seen from the first domain, this requirement would be
> solved when we could set a first domain priority for the second
> domain... Gilles, is that what is meant by the SCHED_IDLE policy for
> Linux?

What I meant is that this problem really looks like a problem with Linux
scheduler, not with Xenomai scheduler. You want your task scheduled when
Linux has not any other task to run, the fact that it will be run when
Xenomai has no other task to run simply follows from the fact that Linux
tasks are already run when Xenomai has not any Xenomai task to run.

And in fact SCHED_IDLE already exists and does what you expect. However,
I do not know if a task running with SCHED_IDLE may have a shadow (I
guess not), that is our real problem now.

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Henri Roosen wrote:
>> We have the same requirement for which we also have no solution. One
>> of our threads has a configurable priority of which at the lowest
>> priority the thread should behave like an idle task that is only
>> scheduled when all xenomai threads are idle and also the Linux domain
>> is idle. The thread might use rtdm-drivers and xenomai mutexes and
>> therefore has to be moved to the Xenomai domain. But then it always
>> has higher priority than any threads scheduled in linux.

Ok. Got it. So what we want is that a thread with SCHED_IDLE priority is
less priority than a thread running with SCHED_OTHER scheduling policy.
Now, giving it a lower priority than other Linux thread probably
requires defining a new scheduling policy for Xenomai, yes.

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

Henri Roosen wrote:
> We have the same requirement for which we also have no solution. One
> of our threads has a configurable priority of which at the lowest
> priority the thread should behave like an idle task that is only
> scheduled when all xenomai threads are idle and also the Linux domain
> is idle. The thread might use rtdm-drivers and xenomai mutexes and
> therefore has to be moved to the Xenomai domain. But then it always
> has higher priority than any threads scheduled in linux.

That's not true, Xenomai threads can run in non-RT scheduling classes as
well. They may just gain RT priority while holding some lock that is
requested by a RT thread as well.

> 
> Actually, when seen from the first domain, this requirement would be
> solved when we could set a first domain priority for the second
> domain... Gilles, is that what is meant by the SCHED_IDLE policy for
> Linux?

"nice +20", thus a very small CPU share if other task are runnable.

Note that "true" SCHED_IDLE, ie. no CPU share if some other task is
runnable, could only work if we forbid access to Linux syscalls for such
tasks - not feasible.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

Jan Kiszka wrote:
> Henri Roosen wrote:
>> We have the same requirement for which we also have no solution. One
>> of our threads has a configurable priority of which at the lowest
>> priority the thread should behave like an idle task that is only
>> scheduled when all xenomai threads are idle and also the Linux domain
>> is idle. The thread might use rtdm-drivers and xenomai mutexes and
>> therefore has to be moved to the Xenomai domain. But then it always
>> has higher priority than any threads scheduled in linux.
> 
> That's not true, Xenomai threads can run in non-RT scheduling classes as
> well. They may just gain RT priority while holding some lock that is
> requested by a RT thread as well.
> 
>> Actually, when seen from the first domain, this requirement would be
>> solved when we could set a first domain priority for the second
>> domain... Gilles, is that what is meant by the SCHED_IDLE policy for
>> Linux?
> 
> "nice +20", thus a very small CPU share if other task are runnable.
> 
> Note that "true" SCHED_IDLE, ie. no CPU share if some other task is
> runnable, could only work if we forbid access to Linux syscalls for such
> tasks - not feasible.

Actually, "nothing is impossible": That task could gain normal Linux
priority while running the syscall. But it would have to loose
immediately again when leaving Linux. So no lazy mode switching like we
do for other Xenomai tasks, otherwise these tasks would loose there idle
property after the first syscall.

The point is that things get fairly special and maybe invasive to
Xenomai, and I'm still not seeing the practical benefit compared to nice
19 or 20.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Jan Kiszka wrote:
>> Henri Roosen wrote:
>>> We have the same requirement for which we also have no solution. One
>>> of our threads has a configurable priority of which at the lowest
>>> priority the thread should behave like an idle task that is only
>>> scheduled when all xenomai threads are idle and also the Linux domain
>>> is idle. The thread might use rtdm-drivers and xenomai mutexes and
>>> therefore has to be moved to the Xenomai domain. But then it always
>>> has higher priority than any threads scheduled in linux.
>> That's not true, Xenomai threads can run in non-RT scheduling classes as
>> well. They may just gain RT priority while holding some lock that is
>> requested by a RT thread as well.
>>
>>> Actually, when seen from the first domain, this requirement would be
>>> solved when we could set a first domain priority for the second
>>> domain... Gilles, is that what is meant by the SCHED_IDLE policy for
>>> Linux?
>> "nice +20", thus a very small CPU share if other task are runnable.
>>
>> Note that "true" SCHED_IDLE, ie. no CPU share if some other task is
>> runnable, could only work if we forbid access to Linux syscalls for such
>> tasks - not feasible.
> 
> Actually, "nothing is impossible": That task could gain normal Linux
> priority while running the syscall. But it would have to loose
> immediately again when leaving Linux. So no lazy mode switching like we
> do for other Xenomai tasks, otherwise these tasks would loose there idle
> property after the first syscall.
> 
> The point is that things get fairly special and maybe invasive to
> Xenomai, and I'm still not seeing the practical benefit compared to nice
> 19 or 20.

I think the problem is not to have a "true" SCHED_IDLE when running
under Linux scheduler, but rather when running under Xenomai scheduler.
And that can probably be arranged by implementing the SCHED_IDLE policy
for Xenomai.


-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

Ok, I might give Jan's proposal a try and could be sufficient for us:a
Xenomai thread with non-RT scheduling class (didn't know that was
possible).

We just need the thread in the Xenomai domain for being able to access
Xenomai resources, but want it to be scheduled together with the Linux
nice levels. A near IDLE nice level should be ok. We just don't want
that our Xenomai 'idle' thread with a while(1) would prevent Linux to
run.

On Fri, Apr 2, 2010 at 11:27 AM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> Jan Kiszka wrote:
>> Jan Kiszka wrote:
>>> Henri Roosen wrote:
>>>> We have the same requirement for which we also have no solution. One
>>>> of our threads has a configurable priority of which at the lowest
>>>> priority the thread should behave like an idle task that is only
>>>> scheduled when all xenomai threads are idle and also the Linux domain
>>>> is idle. The thread might use rtdm-drivers and xenomai mutexes and
>>>> therefore has to be moved to the Xenomai domain. But then it always
>>>> has higher priority than any threads scheduled in linux.
>>> That's not true, Xenomai threads can run in non-RT scheduling classes as
>>> well. They may just gain RT priority while holding some lock that is
>>> requested by a RT thread as well.
>>>
>>>> Actually, when seen from the first domain, this requirement would be
>>>> solved when we could set a first domain priority for the second
>>>> domain... Gilles, is that what is meant by the SCHED_IDLE policy for
>>>> Linux?
>>> "nice +20", thus a very small CPU share if other task are runnable.
>>>
>>> Note that "true" SCHED_IDLE, ie. no CPU share if some other task is
>>> runnable, could only work if we forbid access to Linux syscalls for such
>>> tasks - not feasible.
>>
>> Actually, "nothing is impossible": That task could gain normal Linux
>> priority while running the syscall. But it would have to loose
>> immediately again when leaving Linux. So no lazy mode switching like we
>> do for other Xenomai tasks, otherwise these tasks would loose there idle
>> property after the first syscall.
>>
>> The point is that things get fairly special and maybe invasive to
>> Xenomai, and I'm still not seeing the practical benefit compared to nice
>> 19 or 20.
>
> I think the problem is not to have a "true" SCHED_IDLE when running
> under Linux scheduler, but rather when running under Xenomai scheduler.
> And that can probably be arranged by implementing the SCHED_IDLE policy
> for Xenomai.
>
>
> --
>                                            Gilles.
>

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Henri Roosen wrote:
> Ok, I might give Jan's proposal a try and could be sufficient for us:a
> Xenomai thread with non-RT scheduling class (didn't know that was
> possible).
> 
> We just need the thread in the Xenomai domain for being able to access
> Xenomai resources, but want it to be scheduled together with the Linux
> nice levels. A near IDLE nice level should be ok. We just don't want
> that our Xenomai 'idle' thread with a while(1) would prevent Linux to
> run.

That will happen with SCHED_OTHER if while(1) occurs while running in
primary mode.

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

Actually that is what I thought in the first place, however Jan's
comment "That's not true, Xenomai threads can run in non-RT scheduling
classes as well. They may just gain RT priority while holding some
lock that is requested by a RT thread as well." made me think I was
wrong...

So we would really need a SCHED_IDLE for Xenomai then to solve this problem?

On Fri, Apr 2, 2010 at 11:34 AM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> Henri Roosen wrote:
>> Ok, I might give Jan's proposal a try and could be sufficient for us:a
>> Xenomai thread with non-RT scheduling class (didn't know that was
>> possible).
>>
>> We just need the thread in the Xenomai domain for being able to access
>> Xenomai resources, but want it to be scheduled together with the Linux
>> nice levels. A near IDLE nice level should be ok. We just don't want
>> that our Xenomai 'idle' thread with a while(1) would prevent Linux to
>> run.
>
> That will happen with SCHED_OTHER if while(1) occurs while running in
> primary mode.
>
> --
>                                            Gilles.
>

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 802 bytes --]

Henri Roosen wrote:
> Actually that is what I thought in the first place, however Jan's
> comment "That's not true, Xenomai threads can run in non-RT scheduling
> classes as well. They may just gain RT priority while holding some
> lock that is requested by a RT thread as well." made me think I was
> wrong...
> 
> So we would really need a SCHED_IDLE for Xenomai then to solve this problem?

I don't think so. But we do need to solve the issue that a non-RT thread
stays too long in primary mode and is thus scheduled by Xenomai with the
wrong priority /wrt other Linux task at its level.

For the time being, you can work around this by issuing a Linux syscall
before entering long processing loops - unless your task doesn't do this
anyway, e.g. to perform some Linux I/O.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Andreas Glatz]

> > Actually that is what I thought in the first place, however Jan's
> > comment "That's not true, Xenomai threads can run in non-RT scheduling
> > classes as well. They may just gain RT priority while holding some
> > lock that is requested by a RT thread as well." made me think I was
> > wrong...
> >
> > So we would really need a SCHED_IDLE for Xenomai then to solve this problem?
>
> I don't think so. But we do need to solve the issue that a non-RT thread
> stays too long in primary mode and is thus scheduled by Xenomai with the
> wrong priority /wrt other Linux task at its level.
> 
> For the time being, you can work around this by issuing a Linux syscall
> before entering long processing loops - unless your task doesn't do this
> anyway, e.g. to perform some Linux I/O.
>

I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)

Andreas

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1508 bytes --]

Andreas Glatz wrote:
>>> Actually that is what I thought in the first place, however Jan's
>>> comment "That's not true, Xenomai threads can run in non-RT scheduling
>>> classes as well. They may just gain RT priority while holding some
>>> lock that is requested by a RT thread as well." made me think I was
>>> wrong...
>>>
>>> So we would really need a SCHED_IDLE for Xenomai then to solve this problem?
>> I don't think so. But we do need to solve the issue that a non-RT thread
>> stays too long in primary mode and is thus scheduled by Xenomai with the
>> wrong priority /wrt other Linux task at its level.
>>
>> For the time being, you can work around this by issuing a Linux syscall
>> before entering long processing loops - unless your task doesn't do this
>> anyway, e.g. to perform some Linux I/O.
>>
> 
> I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)

The proper way would be to not having to change the application code.
But this workaround (Linux syscall or *_set_mode()) is required until we
improve the nucleus.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Andreas Glatz]

Hi,

On Tue, Apr 06, 2010 at 02:45:20AM -0400, Jan Kiszka wrote:
> Andreas Glatz wrote:
> >>> Actually that is what I thought in the first place, however Jan's
> >>> comment "That's not true, Xenomai threads can run in non-RT scheduling
> >>> classes as well. They may just gain RT priority while holding some
> >>> lock that is requested by a RT thread as well." made me think I was
> >>> wrong...
> >>>
> >>> So we would really need a SCHED_IDLE for Xenomai then to solve this problem?
> >> I don't think so. But we do need to solve the issue that a non-RT thread
> >> stays too long in primary mode and is thus scheduled by Xenomai with the
> >> wrong priority /wrt other Linux task at its level.
> >>
> >> For the time being, you can work around this by issuing a Linux syscall
> >> before entering long processing loops - unless your task doesn't do this
> >> anyway, e.g. to perform some Linux I/O.
> >>
> > 
> > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
> 
> The proper way would be to not having to change the application code.
> But this workaround (Linux syscall or *_set_mode()) is required until we
> improve the nucleus.

I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having 
to review and insert a Linux syscall or *_set_mode() in the application code I just call 
rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which 
should mostly run in secondary mode under SCHED_IDLE (see example further down). The task 
marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately 
switch back to secondary mode once the Xenomai skincall is done. 

We identified just one case where this task has to stay in primary mode. This is between 
rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost. 
If the task stayed in secondary mode during that time it either would potentally delay the 
execution of a high priority task or would kill the system.

The patch seems to work for us. Our statistics task which blocked the system for a long 
time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is 
heavily loaded now the statistics will get out of sync but the UI will still be responsive.

One thing I've noticed though, and this is not related to the patch (I verified it on a 
vanilla Xenomai system): Consider the example I included. It prints average cycle times 
and the cycle time variance of the high priority task ("T2"). I noticed a big difference 
in the cycle time variance when switching the first task ("T1") to secondary mode with 
rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or 
SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :) 
Can someone give me a short explanation or point me somewhere to get an explanation for
this behaviour? I didn't expect such a difference in variance:

SCHED_FIFO:
task=2 count=1000 average=10505us variance=851(us)^2
task=2 count=1000 average=10504us variance=176(us)^2
task=2 count=1000 average=10505us variance=716(us)^2
task=2 count=1000 average=10504us variance=148(us)^2
task=2 count=1000 average=10504us variance=143(us)^2
task=2 count=1000 average=10504us variance=141(us)^2
task=2 count=1000 average=10504us variance=138(us)^2

SCHED_NORMAL:
task=2 count=1000 average=10501us variance=2115(us)^2
task=2 count=1000 average=10504us variance=3121(us)^2
task=2 count=1000 average=10500us variance=161(us)^2
task=2 count=1000 average=10501us variance=1136(us)^2
task=2 count=1000 average=10500us variance=194(us)^2
task=2 count=1000 average=10501us variance=1971(us)^2
task=2 count=1000 average=10500us variance=132(us)^2
task=2 count=1000 average=10501us variance=1173(us)^2

SCHED_IDLE:
task=2 count=1000 average=10504us variance=3413(us)^2
task=2 count=1000 average=10503us variance=3567(us)^2
task=2 count=1000 average=10504us variance=3409(us)^2
task=2 count=1000 average=10504us variance=1743(us)^2
task=2 count=1000 average=10504us variance=2710(us)^2
task=2 count=1000 average=10504us variance=2548(us)^2
task=2 count=1000 average=10504us variance=2364(us)^2
task=2 count=1000 average=10504us variance=2867(us)^2
task=2 count=1000 average=10504us variance=2755(us)^2


Regards, Andreas

(Xenomai 2.4.10.1, Linux 2.6.32, Ipipe 2.8)


EXAMPLE APPLICATION:

#include <stdio.h>
#include <unistd.h>
#include <native/task.h>
#include <native/sem.h>
#include <native/mutex.h>
#include <sys/mman.h>
#include <rtdk.h>
#include <sched.h>
#include <linux/sched.h>

typedef struct test {
    RTIME timestamp;
    RTIME sum;
    RTIME sumsq;
    int count;
} test_t;

static test_t test1 = {0, 0, 0, 0}, test2 = {0, 0, 0, 0};
static RT_TASK task1, task2;
static RT_MUTEX mutex1, mutex2;

static void task_body( void* cookie )
{
    RTIME timestamp, delta;
    test_t* ptr = (test_t*)cookie;
    int num = ( ptr == &test1) ? 1 : 2;

    if( num == 1 ) rt_task_set_mode(0, 0x00080000, NULL);

    while(1)
    {
        rt_mutex_acquire(&mutex1, TM_INFINITE);

        timestamp = __xn_rdtsc();
        delta = (timestamp - ptr->timestamp) >> 6 /* in us */;
        ptr->sum += delta;
        ptr->sumsq += delta*delta;
        ptr->timestamp = timestamp;
        if( ++ptr->count >= 1000 ) {
            RTIME avg, var;
            avg = ptr->sum/ptr->count;
            var = (ptr->sumsq - (ptr->sum*ptr->sum)/ptr->count)/(ptr->count-1);

            if( num == 2 )
                rt_printf("task=%d count=%d average=%lluus variance=%llu(us)^2\n",
                          num, ptr->count, avg, var);

            ptr->sum = 0;
            ptr->sumsq = 0;
            ptr->count = 0;
        }

		// If commented T1 basically runs in a while(1) {}
		// loop without any sleeps. UI should be responsive
		// since T1 runs with SCHED_IDLE.
		// If uncommented the sleep causes T2 to boost the
		// priority of T1. 
        if( num == 1 ) rt_task_sleep(10000000);

		// T1 automatically switches to secondary mode after
		// this call.
        rt_mutex_release(&mutex1);

        // Give T1 time to run
        if( num == 2 ) rt_task_sleep(10000000);
    }
}

int main(int argc, char* argv[])
{
    int err;

    mlockall(MCL_CURRENT|MCL_FUTURE);

    rt_print_auto_init(1);

    test1.timestamp = test2.timestamp = __xn_rdtsc();
    err  = rt_mutex_create(&mutex1, "M1");
    err += rt_mutex_create(&mutex2, "M2");
    err += rt_task_spawn(&task1, "T1", 0, 33, 0, task_body, (void*)&test1);
    err += rt_task_spawn(&task2, "T2", 0, 66, 0, task_body, (void*)&test2);
    if( !err )
    {
        pause();
    }

    return err;
}

IDLE PATCH:

diff -ruN linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h
--- linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h	2010-04-13 20:02:21.000000000 -0400
+++ linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h	2010-04-14 10:38:09.000000000 -0400
@@ -89,6 +89,8 @@
 #define __xn_exec_adaptive   0x40
 /* Do not restart syscall upon signal receipt. */
 #define __xn_exec_norestart  0x80
+/* Do not switch to secondary mode after syscall if thread has XNIDLE flag set (see #XNIDLE) */
+#define __xn_exec_norelax    0x100
 /* Context-agnostic syscall. Will actually run in Xenomai domain. */
 #define __xn_exec_any        0x0
 /* Short-hand for shadow init syscall. */
diff -ruN linux-2.6.32-5RR9/include/xenomai/native/task.h linux-2.6.32-5RR9-new/include/xenomai/native/task.h
--- linux-2.6.32-5RR9/include/xenomai/native/task.h	2010-04-13 20:02:21.000000000 -0400
+++ linux-2.6.32-5RR9-new/include/xenomai/native/task.h	2010-04-14 10:38:09.000000000 -0400
@@ -52,6 +52,7 @@
 #define T_SHIELD   XNSHIELD   /**< See #XNSHIELD  */ 
 #define T_WARNSW   XNTRAPSW   /**< See #XNTRAPSW  */ 
 #define T_RPIOFF   XNRPIOFF   /**< See #XNRPIOFF  */ 
+#define T_IDLE     XNIDLE     /**< See #XNIDLE    */
 #define T_PRIMARY  0x00000200	/* Recycle internal bits status which */
 #define T_JOINABLE 0x00000400	/* won't be passed to the nucleus.  */
 /*! @} */ /* Ends doxygen-group native_task_status */
diff -ruN linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h
--- linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h	2010-04-13 20:02:21.000000000 -0400
+++ linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h	2010-04-14 10:38:09.000000000 -0400
@@ -55,6 +55,7 @@
 #define XNSHIELD  0x00010000 /**< IRQ shield is enabled (shadow only) */
 #define XNTRAPSW  0x00020000 /**< Trap execution mode switches */
 #define XNRPIOFF  0x00040000 /**< Stop priority coupling (shadow only) */
+#define XNIDLE    0x00080000 /**< Switches to secondary mode after syscalls if not holding mutexes */
 
 #define XNFPU     0x00100000 /**< Thread uses FPU */
 #define XNSHADOW  0x00200000 /**< Shadow thread */
@@ -90,7 +91,7 @@
 }
 
 #define XNTHREAD_BLOCK_BITS   (XNSUSP|XNPEND|XNDELAY|XNDORMANT|XNRELAX|XNHELD)
-#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF)
+#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF|XNIDLE)
 
 /* These state flags are available to the real-time interfaces */
 #define XNTHREAD_STATE_SPARE0  0x10000000
@@ -186,6 +187,8 @@
 
     xnpqueue_t claimq;		/* Owned resources claimed by others (PIP) */
 
+    int lockcnt;			/* Mutexes which are currently locked by this thread */
+
     struct xnsynch *wchan;	/* Resource the thread pends on */
 
     struct xnsynch *wwake;	/* Wait channel the thread was resumed from */
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c
--- linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c	2010-04-14 18:04:20.000000000 -0400
@@ -1187,7 +1187,7 @@
 void xnshadow_relax(int notify)
 {
 	xnthread_t *thread = xnpod_current_thread();
-	int prio;
+	int prio, policy;
 	spl_t s;
 
 	XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
@@ -1217,9 +1217,9 @@
 		xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
 			    thread->name, xnthread_user_pid(thread));
 
-	prio = normalize_priority(xnthread_current_priority(thread));
-	rthal_reenter_root(get_switch_lock_owner(),
-			   prio ? SCHED_FIFO : SCHED_NORMAL, prio);
+	prio = xnthread_test_state(thread, XNIDLE) ? 0 : normalize_priority(xnthread_current_priority(thread));
+	policy = xnthread_test_state(thread, XNIDLE) ? SCHED_IDLE : (prio ? SCHED_FIFO : SCHED_NORMAL);
+	rthal_reenter_root(get_switch_lock_owner(), policy, prio);
 
 	xnstat_counter_inc(&thread->stat.ssw);	/* Account for secondary mode switch. */
 
@@ -2001,8 +2001,13 @@
 
 	if (xnpod_shadow_p() && signal_pending(p))
 		request_syscall_restart(thread, regs, sysflags);
-	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
-		xnshadow_harden();	/* -EPERM will be trapped later if needed. */
+	else if ((sysflags & __xn_exec_switchback) != 0 && switched) {
+		if (!xnthread_test_state(thread, XNIDLE) || 
+		    (xnthread_test_state(thread, XNIDLE) && (sysflags & __xn_exec_norelax) != 0))
+		xnshadow_harden();  /* -EPERM will be trapped later if needed. */
+	} else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() && 
+			   xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
+		xnshadow_relax(0);
 
 	return RTHAL_EVENT_STOP;
 
@@ -2137,6 +2142,9 @@
 		request_syscall_restart(xnshadow_thread(current), regs, sysflags);
 	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
 		xnshadow_relax(0);
+	else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() && 
+			 xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
+		xnshadow_relax(0);
 
 	return RTHAL_EVENT_STOP;
 }
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c
--- linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c	2010-04-14 10:38:09.000000000 -0400
@@ -124,6 +124,7 @@
 	thread->rpi = NULL;
 #endif /* CONFIG_XENO_OPT_PRIOCPL */
 	initpq(&thread->claimq);
+	thread->lockcnt = 0;
 
 	xnarch_init_display_context(thread);
 
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c
--- linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c	2010-04-14 10:38:09.000000000 -0400
@@ -396,6 +396,8 @@
 		/* xnsynch_sleep_on() might have stolen the resource,
 		   so we need to put our internal data in sync. */
 		mutex->lockcnt = 1;
+		
+		thread->lockcnt++;
 	}
 
       unlock_and_exit:
@@ -462,6 +464,8 @@
 	if (--mutex->lockcnt > 0)
 		goto unlock_and_exit;
 
+	xnpod_current_thread()->lockcnt--;
+
 	if (xnsynch_wakeup_one_sleeper(&mutex->synch_base)) {
 		mutex->lockcnt = 1;
 		xnpod_schedule();
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c
--- linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c	2010-04-14 10:38:09.000000000 -0400
@@ -3932,7 +3932,7 @@
 	[__native_mutex_create] = {&__rt_mutex_create, __xn_exec_any},
 	[__native_mutex_bind] = {&__rt_mutex_bind, __xn_exec_conforming},
 	[__native_mutex_delete] = {&__rt_mutex_delete, __xn_exec_any},
-	[__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary},
+	[__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary|__xn_exec_norelax},
 	[__native_mutex_release] = {&__rt_mutex_release, __xn_exec_primary},
 	[__native_mutex_inquire] = {&__rt_mutex_inquire, __xn_exec_any},
 	[__native_cond_create] = {&__rt_cond_create, __xn_exec_any},
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c
--- linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c	2010-04-14 10:38:09.000000000 -0400
@@ -1498,7 +1498,7 @@
 	}
 
 	if (((clrmask | setmask) &
-	     ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW)) != 0)
+	     ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW | T_IDLE)) != 0)
 		return -EINVAL;
 
 	if (!xnpod_primary_p())

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

Hi,

On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:
> Hi,
> 
> On Tue, Apr 06, 2010 at 02:45:20AM -0400, Jan Kiszka wrote:
> > Andreas Glatz wrote:
> > >>> Actually that is what I thought in the first place, however Jan's
> > >>> comment "That's not true, Xenomai threads can run in non-RT scheduling
> > >>> classes as well. They may just gain RT priority while holding some
> > >>> lock that is requested by a RT thread as well." made me think I was
> > >>> wrong...
> > >>>
> > >>> So we would really need a SCHED_IDLE for Xenomai then to solve this problem?
> > >> I don't think so. But we do need to solve the issue that a non-RT thread
> > >> stays too long in primary mode and is thus scheduled by Xenomai with the
> > >> wrong priority /wrt other Linux task at its level.
> > >>
> > >> For the time being, you can work around this by issuing a Linux syscall
> > >> before entering long processing loops - unless your task doesn't do this
> > >> anyway, e.g. to perform some Linux I/O.
> > >>
> > > 
> > > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
> > 
> > The proper way would be to not having to change the application code.
> > But this workaround (Linux syscall or *_set_mode()) is required until we
> > improve the nucleus.
> 
> I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having 
> to review and insert a Linux syscall or *_set_mode() in the application code I just call 
> rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which 
> should mostly run in secondary mode under SCHED_IDLE (see example further down). The task 
> marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately 
> switch back to secondary mode once the Xenomai skincall is done. 
> 
> We identified just one case where this task has to stay in primary mode. This is between 
> rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost. 
> If the task stayed in secondary mode during that time it either would potentally delay the 
> execution of a high priority task or would kill the system.
> 
> The patch seems to work for us. Our statistics task which blocked the system for a long 
> time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is 
> heavily loaded now the statistics will get out of sync but the UI will still be responsive.
> 

The logic of this patch looks ok for the native skin, given that 2.4.x
does not provide a centralized implementation for dealing with exclusive
resources, like 2.5.x with xnsynch_acquire/release, and always emits a
syscall to manage those resources.

This said, you could spare the T_IDLE tag by assuming that any non-RT
shadow thread has to switch back to secondary mode after a syscall,
unless the owned resource count is non-zero. This is where we are
heading to in 2.5.x, since the preferred mode of operation for such
thread has to be fundamentally "relaxed" (otherwise, one would have
created a RT thread, right).

I'm also unsure you should force SCHED_IDLE, instead of picking
SCHED_OTHER for a more general approach to this issue. You can't assume
that userland does want to be reniced that way, at least not from the
nucleus. But I guess this fits your problem though.

To sum up, since we can't really provide a true SCHED_IDLE policy on
linux (i.e. not a nice-level hack), and implementing a sched class in
Xenomai having a lower priority than the existing xnsched_class_idle (in
2.5.x) is not feasible (we could not run any userland task in it
anyway), we'd better stick with SCHED_OTHER.

> One thing I've noticed though, and this is not related to the patch (I verified it on a 
> vanilla Xenomai system): Consider the example I included. It prints average cycle times 
> and the cycle time variance of the high priority task ("T2"). I noticed a big difference 
> in the cycle time variance when switching the first task ("T1") to secondary mode with 
> rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or 
> SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :) 
> Can someone give me a short explanation or point me somewhere to get an explanation for
> this behaviour? I didn't expect such a difference in variance:
> 
> SCHED_FIFO:
> task=2 count=1000 average=10505us variance=851(us)^2
> task=2 count=1000 average=10504us variance=176(us)^2
> task=2 count=1000 average=10505us variance=716(us)^2
> task=2 count=1000 average=10504us variance=148(us)^2
> task=2 count=1000 average=10504us variance=143(us)^2
> task=2 count=1000 average=10504us variance=141(us)^2
> task=2 count=1000 average=10504us variance=138(us)^2
> 
> SCHED_NORMAL:
> task=2 count=1000 average=10501us variance=2115(us)^2
> task=2 count=1000 average=10504us variance=3121(us)^2
> task=2 count=1000 average=10500us variance=161(us)^2
> task=2 count=1000 average=10501us variance=1136(us)^2
> task=2 count=1000 average=10500us variance=194(us)^2
> task=2 count=1000 average=10501us variance=1971(us)^2
> task=2 count=1000 average=10500us variance=132(us)^2
> task=2 count=1000 average=10501us variance=1173(us)^2
> 
> SCHED_IDLE:
> task=2 count=1000 average=10504us variance=3413(us)^2
> task=2 count=1000 average=10503us variance=3567(us)^2
> task=2 count=1000 average=10504us variance=3409(us)^2
> task=2 count=1000 average=10504us variance=1743(us)^2
> task=2 count=1000 average=10504us variance=2710(us)^2
> task=2 count=1000 average=10504us variance=2548(us)^2
> task=2 count=1000 average=10504us variance=2364(us)^2
> task=2 count=1000 average=10504us variance=2867(us)^2
> task=2 count=1000 average=10504us variance=2755(us)^2
> 
> 
> Regards, Andreas
> 
> (Xenomai 2.4.10.1, Linux 2.6.32, Ipipe 2.8)
> 
> 
> EXAMPLE APPLICATION:
> 
> #include <stdio.h>
> #include <unistd.h>
> #include <native/task.h>
> #include <native/sem.h>
> #include <native/mutex.h>
> #include <sys/mman.h>
> #include <rtdk.h>
> #include <sched.h>
> #include <linux/sched.h>
> 
> typedef struct test {
>     RTIME timestamp;
>     RTIME sum;
>     RTIME sumsq;
>     int count;
> } test_t;
> 
> static test_t test1 = {0, 0, 0, 0}, test2 = {0, 0, 0, 0};
> static RT_TASK task1, task2;
> static RT_MUTEX mutex1, mutex2;
> 
> static void task_body( void* cookie )
> {
>     RTIME timestamp, delta;
>     test_t* ptr = (test_t*)cookie;
>     int num = ( ptr == &test1) ? 1 : 2;
> 
>     if( num == 1 ) rt_task_set_mode(0, 0x00080000, NULL);
> 
>     while(1)
>     {
>         rt_mutex_acquire(&mutex1, TM_INFINITE);
> 
>         timestamp = __xn_rdtsc();
>         delta = (timestamp - ptr->timestamp) >> 6 /* in us */;
>         ptr->sum += delta;
>         ptr->sumsq += delta*delta;
>         ptr->timestamp = timestamp;
>         if( ++ptr->count >= 1000 ) {
>             RTIME avg, var;
>             avg = ptr->sum/ptr->count;
>             var = (ptr->sumsq - (ptr->sum*ptr->sum)/ptr->count)/(ptr->count-1);
> 
>             if( num == 2 )
>                 rt_printf("task=%d count=%d average=%lluus variance=%llu(us)^2\n",
>                           num, ptr->count, avg, var);
> 
>             ptr->sum = 0;
>             ptr->sumsq = 0;
>             ptr->count = 0;
>         }
> 
> 		// If commented T1 basically runs in a while(1) {}
> 		// loop without any sleeps. UI should be responsive
> 		// since T1 runs with SCHED_IDLE.
> 		// If uncommented the sleep causes T2 to boost the
> 		// priority of T1. 
>         if( num == 1 ) rt_task_sleep(10000000);
> 
> 		// T1 automatically switches to secondary mode after
> 		// this call.
>         rt_mutex_release(&mutex1);
> 
>         // Give T1 time to run
>         if( num == 2 ) rt_task_sleep(10000000);
>     }
> }
> 
> int main(int argc, char* argv[])
> {
>     int err;
> 
>     mlockall(MCL_CURRENT|MCL_FUTURE);
> 
>     rt_print_auto_init(1);
> 
>     test1.timestamp = test2.timestamp = __xn_rdtsc();
>     err  = rt_mutex_create(&mutex1, "M1");
>     err += rt_mutex_create(&mutex2, "M2");
>     err += rt_task_spawn(&task1, "T1", 0, 33, 0, task_body, (void*)&test1);
>     err += rt_task_spawn(&task2, "T2", 0, 66, 0, task_body, (void*)&test2);
>     if( !err )
>     {
>         pause();
>     }
> 
>     return err;
> }
> 
> IDLE PATCH:
> 
> diff -ruN linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h
> --- linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h	2010-04-13 20:02:21.000000000 -0400
> +++ linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h	2010-04-14 10:38:09.000000000 -0400
> @@ -89,6 +89,8 @@
>  #define __xn_exec_adaptive   0x40
>  /* Do not restart syscall upon signal receipt. */
>  #define __xn_exec_norestart  0x80
> +/* Do not switch to secondary mode after syscall if thread has XNIDLE flag set (see #XNIDLE) */
> +#define __xn_exec_norelax    0x100
>  /* Context-agnostic syscall. Will actually run in Xenomai domain. */
>  #define __xn_exec_any        0x0
>  /* Short-hand for shadow init syscall. */
> diff -ruN linux-2.6.32-5RR9/include/xenomai/native/task.h linux-2.6.32-5RR9-new/include/xenomai/native/task.h
> --- linux-2.6.32-5RR9/include/xenomai/native/task.h	2010-04-13 20:02:21.000000000 -0400
> +++ linux-2.6.32-5RR9-new/include/xenomai/native/task.h	2010-04-14 10:38:09.000000000 -0400
> @@ -52,6 +52,7 @@
>  #define T_SHIELD   XNSHIELD   /**< See #XNSHIELD  */ 
>  #define T_WARNSW   XNTRAPSW   /**< See #XNTRAPSW  */ 
>  #define T_RPIOFF   XNRPIOFF   /**< See #XNRPIOFF  */ 
> +#define T_IDLE     XNIDLE     /**< See #XNIDLE    */
>  #define T_PRIMARY  0x00000200	/* Recycle internal bits status which */
>  #define T_JOINABLE 0x00000400	/* won't be passed to the nucleus.  */
>  /*! @} */ /* Ends doxygen-group native_task_status */
> diff -ruN linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h
> --- linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h	2010-04-13 20:02:21.000000000 -0400
> +++ linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h	2010-04-14 10:38:09.000000000 -0400
> @@ -55,6 +55,7 @@
>  #define XNSHIELD  0x00010000 /**< IRQ shield is enabled (shadow only) */
>  #define XNTRAPSW  0x00020000 /**< Trap execution mode switches */
>  #define XNRPIOFF  0x00040000 /**< Stop priority coupling (shadow only) */
> +#define XNIDLE    0x00080000 /**< Switches to secondary mode after syscalls if not holding mutexes */
>  
>  #define XNFPU     0x00100000 /**< Thread uses FPU */
>  #define XNSHADOW  0x00200000 /**< Shadow thread */
> @@ -90,7 +91,7 @@
>  }
>  
>  #define XNTHREAD_BLOCK_BITS   (XNSUSP|XNPEND|XNDELAY|XNDORMANT|XNRELAX|XNHELD)
> -#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF)
> +#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF|XNIDLE)
>  
>  /* These state flags are available to the real-time interfaces */
>  #define XNTHREAD_STATE_SPARE0  0x10000000
> @@ -186,6 +187,8 @@
>  
>      xnpqueue_t claimq;		/* Owned resources claimed by others (PIP) */
>  
> +    int lockcnt;			/* Mutexes which are currently locked by this thread */
> +
>      struct xnsynch *wchan;	/* Resource the thread pends on */
>  
>      struct xnsynch *wwake;	/* Wait channel the thread was resumed from */
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c
> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c	2010-04-14 18:04:20.000000000 -0400
> @@ -1187,7 +1187,7 @@
>  void xnshadow_relax(int notify)
>  {
>  	xnthread_t *thread = xnpod_current_thread();
> -	int prio;
> +	int prio, policy;
>  	spl_t s;
>  
>  	XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
> @@ -1217,9 +1217,9 @@
>  		xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
>  			    thread->name, xnthread_user_pid(thread));
>  
> -	prio = normalize_priority(xnthread_current_priority(thread));
> -	rthal_reenter_root(get_switch_lock_owner(),
> -			   prio ? SCHED_FIFO : SCHED_NORMAL, prio);
> +	prio = xnthread_test_state(thread, XNIDLE) ? 0 : normalize_priority(xnthread_current_priority(thread));
> +	policy = xnthread_test_state(thread, XNIDLE) ? SCHED_IDLE : (prio ? SCHED_FIFO : SCHED_NORMAL);
> +	rthal_reenter_root(get_switch_lock_owner(), policy, prio);
>  
>  	xnstat_counter_inc(&thread->stat.ssw);	/* Account for secondary mode switch. */
>  
> @@ -2001,8 +2001,13 @@
>  
>  	if (xnpod_shadow_p() && signal_pending(p))
>  		request_syscall_restart(thread, regs, sysflags);
> -	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
> -		xnshadow_harden();	/* -EPERM will be trapped later if needed. */
> +	else if ((sysflags & __xn_exec_switchback) != 0 && switched) {
> +		if (!xnthread_test_state(thread, XNIDLE) || 
> +		    (xnthread_test_state(thread, XNIDLE) && (sysflags & __xn_exec_norelax) != 0))
> +		xnshadow_harden();  /* -EPERM will be trapped later if needed. */
> +	} else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() && 
> +			   xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
> +		xnshadow_relax(0);
>  
>  	return RTHAL_EVENT_STOP;
>  
> @@ -2137,6 +2142,9 @@
>  		request_syscall_restart(xnshadow_thread(current), regs, sysflags);
>  	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
>  		xnshadow_relax(0);
> +	else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() && 
> +			 xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
> +		xnshadow_relax(0);
>  
>  	return RTHAL_EVENT_STOP;
>  }
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c
> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c	2010-04-14 10:38:09.000000000 -0400
> @@ -124,6 +124,7 @@
>  	thread->rpi = NULL;
>  #endif /* CONFIG_XENO_OPT_PRIOCPL */
>  	initpq(&thread->claimq);
> +	thread->lockcnt = 0;
>  
>  	xnarch_init_display_context(thread);
>  
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c
> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c	2010-04-14 10:38:09.000000000 -0400
> @@ -396,6 +396,8 @@
>  		/* xnsynch_sleep_on() might have stolen the resource,
>  		   so we need to put our internal data in sync. */
>  		mutex->lockcnt = 1;
> +		
> +		thread->lockcnt++;
>  	}
>  
>        unlock_and_exit:
> @@ -462,6 +464,8 @@
>  	if (--mutex->lockcnt > 0)
>  		goto unlock_and_exit;
>  
> +	xnpod_current_thread()->lockcnt--;
> +
>  	if (xnsynch_wakeup_one_sleeper(&mutex->synch_base)) {
>  		mutex->lockcnt = 1;
>  		xnpod_schedule();
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c
> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c	2010-04-14 10:38:09.000000000 -0400
> @@ -3932,7 +3932,7 @@
>  	[__native_mutex_create] = {&__rt_mutex_create, __xn_exec_any},
>  	[__native_mutex_bind] = {&__rt_mutex_bind, __xn_exec_conforming},
>  	[__native_mutex_delete] = {&__rt_mutex_delete, __xn_exec_any},
> -	[__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary},
> +	[__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary|__xn_exec_norelax},
>  	[__native_mutex_release] = {&__rt_mutex_release, __xn_exec_primary},
>  	[__native_mutex_inquire] = {&__rt_mutex_inquire, __xn_exec_any},
>  	[__native_cond_create] = {&__rt_cond_create, __xn_exec_any},
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c
> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c	2010-04-14 10:38:09.000000000 -0400
> @@ -1498,7 +1498,7 @@
>  	}
>  
>  	if (((clrmask | setmask) &
> -	     ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW)) != 0)
> +	     ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW | T_IDLE)) != 0)
>  		return -EINVAL;
>  
>  	if (!xnpod_primary_p())
> 
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help


-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Andreas Glatz]

> > > >> For the time being, you can work around this by issuing a Linux syscall
> > > >> before entering long processing loops - unless your task doesn't do this
> > > >> anyway, e.g. to perform some Linux I/O.
> > > >>
> > > >
> > > > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
> > >
> > > The proper way would be to not having to change the application code.
> > > But this workaround (Linux syscall or *_set_mode()) is required until we
> > > improve the nucleus.
> >
> > I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having
> > to review and insert a Linux syscall or *_set_mode() in the application code I just call
> > rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which
> > should mostly run in secondary mode under SCHED_IDLE (see example further down). The task
> > marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately
> > switch back to secondary mode once the Xenomai skincall is done.
> >
> > We identified just one case where this task has to stay in primary mode. This is between
> > rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost.
> > If the task stayed in secondary mode during that time it either would potentally delay the
> > execution of a high priority task or would kill the system.
> >
> > The patch seems to work for us. Our statistics task which blocked the system for a long
> > time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is
> > heavily loaded now the statistics will get out of sync but the UI will still be responsive.
> >
> 
> The logic of this patch looks ok for the native skin, given that 2.4.x
> does not provide a centralized implementation for dealing with exclusive
> resources, like 2.5.x with xnsynch_acquire/release, and always emits a
> syscall to manage those resources.
> 
> This said, you could spare the T_IDLE tag by assuming that any non-RT
> shadow thread has to switch back to secondary mode after a syscall,
> unless the owned resource count is non-zero. This is where we are
> heading to in 2.5.x, since the preferred mode of operation for such
> thread has to be fundamentally "relaxed" (otherwise, one would have
> created a RT thread, right).
> 
> I'm also unsure you should force SCHED_IDLE, instead of picking
> SCHED_OTHER for a more general approach to this issue. You can't assume
> that userland does want to be reniced that way, at least not from the
> nucleus. But I guess this fits your problem though.
> 
> To sum up, since we can't really provide a true SCHED_IDLE policy on
> linux (i.e. not a nice-level hack), and implementing a sched class in
> Xenomai having a lower priority than the existing xnsched_class_idle (in
> 2.5.x) is not feasible (we could not run any userland task in it
> anyway), we'd better stick with SCHED_OTHER.
> 

Thanks a lot for the feedback. Your suggestions simplified the patch. I
also changed SCHED_IDLE to SCHED_OTHER since it might be more beneficial
for the broader audience. Any other suggestions?

After applying this patch, a thread with priority 0 will automatically
switch back to secondary mode after every (native) skincall unless the
task holds a mutex (simple and nested).

The benefit is, that the task with priority 0 (which I called a linux 
domain rt thread) can issue (native) skincalls and share resources with
high prioritiy task. But it doesn't hold up Linux tasks unless it holds
a mutex since it mostly runs in secondary mode and just switches to 
primary mode when needed.

Just one more questions: Philippe said that you have something similar in
2.5. How do you enable it there? By setting the correct sheduling policy?

Andreas 

PATCH:

diff -ruN linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h
--- linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h	2010-04-13 20:02:21.000000000 -0400
+++ linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h	2010-04-19 09:35:44.000000000 -0400
@@ -186,6 +186,8 @@
 
     xnpqueue_t claimq;		/* Owned resources claimed by others (PIP) */
 
+    int lockcnt;			/* Mutexes which are currently locked by this thread */
+
     struct xnsynch *wchan;	/* Resource the thread pends on */
 
     struct xnsynch *wwake;	/* Wait channel the thread was resumed from */
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c
--- linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c	2010-04-19 18:06:39.000000000 -0400
@@ -976,6 +976,16 @@
 	return prio < MAX_RT_PRIO ? prio : MAX_RT_PRIO - 1;
 }
 
+static inline int relax_thread(xnthread_t *thread)
+{
+	/* A thread with bprio == 0 is called a Linux Domain RT thread.
+	   It has to switch to secondary mode after every skin call
+	   if it doesn't hold any mutexes. */
+	return (xnthread_base_priority(thread) == 0 &&
+			thread->lockcnt == 0)
+			? 1 : 0;
+}
+
 static int gatekeeper_thread(void *data)
 {
 	struct __gatekeeper *gk = (struct __gatekeeper *)data;
@@ -1187,7 +1197,7 @@
 void xnshadow_relax(int notify)
 {
 	xnthread_t *thread = xnpod_current_thread();
-	int prio;
+	int prio, policy;
 	spl_t s;
 
 	XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
@@ -1217,9 +1227,11 @@
 		xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
 			    thread->name, xnthread_user_pid(thread));
 
+	/* If thread is a Linux Domain RT thread and it should be relaxed the
+	   base priority should be equal to the current priority equal to 0. */
+	policy = relax_thread(thread) ? SCHED_NORMAL : SCHED_FIFO;
 	prio = normalize_priority(xnthread_current_priority(thread));
-	rthal_reenter_root(get_switch_lock_owner(),
-			   prio ? SCHED_FIFO : SCHED_NORMAL, prio);
+	rthal_reenter_root(get_switch_lock_owner(), policy, prio);
 
 	xnstat_counter_inc(&thread->stat.ssw);	/* Account for secondary mode switch. */
 
@@ -2001,8 +2013,14 @@
 
 	if (xnpod_shadow_p() && signal_pending(p))
 		request_syscall_restart(thread, regs, sysflags);
-	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
-		xnshadow_harden();	/* -EPERM will be trapped later if needed. */
+	else {
+		int relax = xnpod_shadow_p() && relax_thread(thread);
+		
+		if ((sysflags & __xn_exec_switchback) != 0 && switched && !relax)
+			xnshadow_harden();  /* -EPERM will be trapped later if needed. */
+		else if (relax)
+			xnshadow_relax(0);
+	}
 
 	return RTHAL_EVENT_STOP;
 
@@ -2137,6 +2155,9 @@
 		request_syscall_restart(xnshadow_thread(current), regs, sysflags);
 	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
 		xnshadow_relax(0);
+	else if (xnpod_active_p() && xnpod_shadow_p() && 
+			 relax_thread(xnshadow_thread(current)))
+		xnshadow_relax(0);
 
 	return RTHAL_EVENT_STOP;
 }
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c
--- linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c	2010-04-19 19:00:39.000000000 -0400
@@ -124,6 +124,7 @@
 	thread->rpi = NULL;
 #endif /* CONFIG_XENO_OPT_PRIOCPL */
 	initpq(&thread->claimq);
+	thread->lockcnt = 0;
 
 	xnarch_init_display_context(thread);
 
diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c
--- linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c	2010-04-13 20:02:22.000000000 -0400
+++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c	2010-04-19 09:35:44.000000000 -0400
@@ -396,6 +396,8 @@
 		/* xnsynch_sleep_on() might have stolen the resource,
 		   so we need to put our internal data in sync. */
 		mutex->lockcnt = 1;
+		
+		thread->lockcnt++;
 	}
 
       unlock_and_exit:
@@ -462,6 +464,8 @@
 	if (--mutex->lockcnt > 0)
 		goto unlock_and_exit;
 
+	xnpod_current_thread()->lockcnt--;
+
 	if (xnsynch_wakeup_one_sleeper(&mutex->synch_base)) {
 		mutex->lockcnt = 1;
 		xnpod_schedule();
 

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Tue, 2010-04-20 at 10:12 -0400, Andreas Glatz wrote:
> > > > >> For the time being, you can work around this by issuing a Linux syscall
> > > > >> before entering long processing loops - unless your task doesn't do this
> > > > >> anyway, e.g. to perform some Linux I/O.
> > > > >>
> > > > >
> > > > > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
> > > >
> > > > The proper way would be to not having to change the application code.
> > > > But this workaround (Linux syscall or *_set_mode()) is required until we
> > > > improve the nucleus.
> > >
> > > I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having
> > > to review and insert a Linux syscall or *_set_mode() in the application code I just call
> > > rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which
> > > should mostly run in secondary mode under SCHED_IDLE (see example further down). The task
> > > marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately
> > > switch back to secondary mode once the Xenomai skincall is done.
> > >
> > > We identified just one case where this task has to stay in primary mode. This is between
> > > rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost.
> > > If the task stayed in secondary mode during that time it either would potentally delay the
> > > execution of a high priority task or would kill the system.
> > >
> > > The patch seems to work for us. Our statistics task which blocked the system for a long
> > > time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is
> > > heavily loaded now the statistics will get out of sync but the UI will still be responsive.
> > >
> > 
> > The logic of this patch looks ok for the native skin, given that 2.4.x
> > does not provide a centralized implementation for dealing with exclusive
> > resources, like 2.5.x with xnsynch_acquire/release, and always emits a
> > syscall to manage those resources.
> > 
> > This said, you could spare the T_IDLE tag by assuming that any non-RT
> > shadow thread has to switch back to secondary mode after a syscall,
> > unless the owned resource count is non-zero. This is where we are
> > heading to in 2.5.x, since the preferred mode of operation for such
> > thread has to be fundamentally "relaxed" (otherwise, one would have
> > created a RT thread, right).
> > 
> > I'm also unsure you should force SCHED_IDLE, instead of picking
> > SCHED_OTHER for a more general approach to this issue. You can't assume
> > that userland does want to be reniced that way, at least not from the
> > nucleus. But I guess this fits your problem though.
> > 
> > To sum up, since we can't really provide a true SCHED_IDLE policy on
> > linux (i.e. not a nice-level hack), and implementing a sched class in
> > Xenomai having a lower priority than the existing xnsched_class_idle (in
> > 2.5.x) is not feasible (we could not run any userland task in it
> > anyway), we'd better stick with SCHED_OTHER.
> > 
> 
> Thanks a lot for the feedback. Your suggestions simplified the patch. I
> also changed SCHED_IDLE to SCHED_OTHER since it might be more beneficial
> for the broader audience. Any other suggestions?

For an even broader audience, the POSIX skin mutexes should be tracked
as well.

> 
> After applying this patch, a thread with priority 0 will automatically
> switch back to secondary mode after every (native) skincall unless the
> task holds a mutex (simple and nested).
> 
> The benefit is, that the task with priority 0 (which I called a linux 
> domain rt thread)

Actually, no. This is not a rt thread at all, in the sense that you have
zero guarantee wrt latency in that case. Such a thread is actually a non
real-time Xenomai shadow thread, meaning that it may invoke Xenomai
services that require the caller to be a Xenomai thread, without
real-time support though.

>  can issue (native) skincalls and share resources with
> high prioritiy task. But it doesn't hold up Linux tasks unless it holds
> a mutex since it mostly runs in secondary mode and just switches to 
> primary mode when needed.
> 
> Just one more questions: Philippe said that you have something similar in
> 2.5. How do you enable it there? By setting the correct sheduling policy?
> 

There are plans to have it. That behavior would be enabled whenever the
linux policy is SCHED_OTHER, and the base priority is 0 Xenomai-wise.
The latter would be enough for now, but it seems more future-proof not
to assume that only SCHED_OTHER tasks could be assigned Xenomai priority
0.

> Andreas 
> 
> PATCH:
> 
> diff -ruN linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h
> --- linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h	2010-04-13 20:02:21.000000000 -0400
> +++ linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h	2010-04-19 09:35:44.000000000 -0400
> @@ -186,6 +186,8 @@
>  
>      xnpqueue_t claimq;		/* Owned resources claimed by others (PIP) */
>  
> +    int lockcnt;			/* Mutexes which are currently locked by this thread */
> +
>      struct xnsynch *wchan;	/* Resource the thread pends on */
>  
>      struct xnsynch *wwake;	/* Wait channel the thread was resumed from */
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c
> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c	2010-04-19 18:06:39.000000000 -0400
> @@ -976,6 +976,16 @@
>  	return prio < MAX_RT_PRIO ? prio : MAX_RT_PRIO - 1;
>  }
>  
> +static inline int relax_thread(xnthread_t *thread)
> +{
> +	/* A thread with bprio == 0 is called a Linux Domain RT thread.
> +	   It has to switch to secondary mode after every skin call
> +	   if it doesn't hold any mutexes. */
> +	return (xnthread_base_priority(thread) == 0 &&
> +			thread->lockcnt == 0)
> +			? 1 : 0;
> +}
> +
>  static int gatekeeper_thread(void *data)
>  {
>  	struct __gatekeeper *gk = (struct __gatekeeper *)data;
> @@ -1187,7 +1197,7 @@
>  void xnshadow_relax(int notify)
>  {
>  	xnthread_t *thread = xnpod_current_thread();
> -	int prio;
> +	int prio, policy;
>  	spl_t s;
>  
>  	XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
> @@ -1217,9 +1227,11 @@
>  		xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
>  			    thread->name, xnthread_user_pid(thread));
>  
> +	/* If thread is a Linux Domain RT thread and it should be relaxed the
> +	   base priority should be equal to the current priority equal to 0. */
> +	policy = relax_thread(thread) ? SCHED_NORMAL : SCHED_FIFO;
>  	prio = normalize_priority(xnthread_current_priority(thread));
> -	rthal_reenter_root(get_switch_lock_owner(),
> -			   prio ? SCHED_FIFO : SCHED_NORMAL, prio);
> +	rthal_reenter_root(get_switch_lock_owner(), policy, prio);
>  
>  	xnstat_counter_inc(&thread->stat.ssw);	/* Account for secondary mode switch. */
>  
> @@ -2001,8 +2013,14 @@
>  
>  	if (xnpod_shadow_p() && signal_pending(p))
>  		request_syscall_restart(thread, regs, sysflags);
> -	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
> -		xnshadow_harden();	/* -EPERM will be trapped later if needed. */
> +	else {
> +		int relax = xnpod_shadow_p() && relax_thread(thread);
> +		
> +		if ((sysflags & __xn_exec_switchback) != 0 && switched && !relax)
> +			xnshadow_harden();  /* -EPERM will be trapped later if needed. */
> +		else if (relax)
> +			xnshadow_relax(0);
> +	}
>  
>  	return RTHAL_EVENT_STOP;
>  
> @@ -2137,6 +2155,9 @@
>  		request_syscall_restart(xnshadow_thread(current), regs, sysflags);
>  	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
>  		xnshadow_relax(0);
> +	else if (xnpod_active_p() && xnpod_shadow_p() && 
> +			 relax_thread(xnshadow_thread(current)))
> +		xnshadow_relax(0);
>  
>  	return RTHAL_EVENT_STOP;
>  }
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c
> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c	2010-04-19 19:00:39.000000000 -0400
> @@ -124,6 +124,7 @@
>  	thread->rpi = NULL;
>  #endif /* CONFIG_XENO_OPT_PRIOCPL */
>  	initpq(&thread->claimq);
> +	thread->lockcnt = 0;
>  
>  	xnarch_init_display_context(thread);
>  
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c
> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c	2010-04-19 09:35:44.000000000 -0400
> @@ -396,6 +396,8 @@
>  		/* xnsynch_sleep_on() might have stolen the resource,
>  		   so we need to put our internal data in sync. */
>  		mutex->lockcnt = 1;
> +		
> +		thread->lockcnt++;
>  	}
>  
>        unlock_and_exit:
> @@ -462,6 +464,8 @@
>  	if (--mutex->lockcnt > 0)
>  		goto unlock_and_exit;
>  
> +	xnpod_current_thread()->lockcnt--;
> +
>  	if (xnsynch_wakeup_one_sleeper(&mutex->synch_base)) {
>  		mutex->lockcnt = 1;
>  		xnpod_schedule();
>  


-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

On Fri, Apr 23, 2010 at 12:54 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> On Tue, 2010-04-20 at 10:12 -0400, Andreas Glatz wrote:
>> > > > >> For the time being, you can work around this by issuing a Linux syscall
>> > > > >> before entering long processing loops - unless your task doesn't do this
>> > > > >> anyway, e.g. to perform some Linux I/O.
>> > > > >>
>> > > > >
>> > > > > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
>> > > >
>> > > > The proper way would be to not having to change the application code.
>> > > > But this workaround (Linux syscall or *_set_mode()) is required until we
>> > > > improve the nucleus.
>> > >
>> > > I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having
>> > > to review and insert a Linux syscall or *_set_mode() in the application code I just call
>> > > rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which
>> > > should mostly run in secondary mode under SCHED_IDLE (see example further down). The task
>> > > marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately
>> > > switch back to secondary mode once the Xenomai skincall is done.
>> > >
>> > > We identified just one case where this task has to stay in primary mode. This is between
>> > > rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost.
>> > > If the task stayed in secondary mode during that time it either would potentally delay the
>> > > execution of a high priority task or would kill the system.
>> > >
>> > > The patch seems to work for us. Our statistics task which blocked the system for a long
>> > > time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is
>> > > heavily loaded now the statistics will get out of sync but the UI will still be responsive.
>> > >
>> >
>> > The logic of this patch looks ok for the native skin, given that 2.4.x
>> > does not provide a centralized implementation for dealing with exclusive
>> > resources, like 2.5.x with xnsynch_acquire/release, and always emits a
>> > syscall to manage those resources.
>> >
>> > This said, you could spare the T_IDLE tag by assuming that any non-RT
>> > shadow thread has to switch back to secondary mode after a syscall,
>> > unless the owned resource count is non-zero. This is where we are
>> > heading to in 2.5.x, since the preferred mode of operation for such
>> > thread has to be fundamentally "relaxed" (otherwise, one would have
>> > created a RT thread, right).
>> >
>> > I'm also unsure you should force SCHED_IDLE, instead of picking
>> > SCHED_OTHER for a more general approach to this issue. You can't assume
>> > that userland does want to be reniced that way, at least not from the
>> > nucleus. But I guess this fits your problem though.
>> >
>> > To sum up, since we can't really provide a true SCHED_IDLE policy on
>> > linux (i.e. not a nice-level hack), and implementing a sched class in
>> > Xenomai having a lower priority than the existing xnsched_class_idle (in
>> > 2.5.x) is not feasible (we could not run any userland task in it
>> > anyway), we'd better stick with SCHED_OTHER.
>> >
>>
>> Thanks a lot for the feedback. Your suggestions simplified the patch. I
>> also changed SCHED_IDLE to SCHED_OTHER since it might be more beneficial
>> for the broader audience. Any other suggestions?
>
> For an even broader audience, the POSIX skin mutexes should be tracked
> as well.
>
>>
>> After applying this patch, a thread with priority 0 will automatically
>> switch back to secondary mode after every (native) skincall unless the
>> task holds a mutex (simple and nested).
>>
>> The benefit is, that the task with priority 0 (which I called a linux
>> domain rt thread)
>
> Actually, no. This is not a rt thread at all, in the sense that you have
> zero guarantee wrt latency in that case. Such a thread is actually a non
> real-time Xenomai shadow thread, meaning that it may invoke Xenomai
> services that require the caller to be a Xenomai thread, without
> real-time support though.
>
>>  can issue (native) skincalls and share resources with
>> high prioritiy task. But it doesn't hold up Linux tasks unless it holds
>> a mutex since it mostly runs in secondary mode and just switches to
>> primary mode when needed.
>>
>> Just one more questions: Philippe said that you have something similar in
>> 2.5. How do you enable it there? By setting the correct sheduling policy?
>>
>
> There are plans to have it. That behavior would be enabled whenever the
> linux policy is SCHED_OTHER, and the base priority is 0 Xenomai-wise.
> The latter would be enough for now, but it seems more future-proof not
> to assume that only SCHED_OTHER tasks could be assigned Xenomai priority
> 0.
>

Is there any update on the plans to have Xenomai base priority 0
threads to be operating by default in relaxed mode? We are using 2.5.4
and currently apply the patch below as a workaround. The patch is
similar to the 2.4 patch of Andreas in that it actively switches back
base prio 0 shadowed threads into relaxed mode, but doesn't take any
held locks into account.

Please comment on this patch. If there is any preliminary code for
2.5.x available regarding this functionality I'll be happy to test it
;-)

--- xenomai-2.5.4-orig/ksrc/nucleus/shadow.c	2010-07-03 19:58:26.000000000 +0200
+++ xenomai-2.5.4/ksrc/nucleus/shadow.c	2010-09-30 11:35:08.000000000 +0200
@@ -2518,7 +2518,16 @@ static inline int do_losyscall_event(uns
 		sigs = 1;
 		handle_rt_signals(thread, regs, sysflags);
 	}
-	if (!sigs && (sysflags & __xn_exec_switchback) != 0 && switched)
+	/*
+	 * Switch back to relaxed state when the syscall requires it
+	 * or when the base priority of the thread is 0. This way
+	 * base-prio 0 threads will by default be in the Linux domain
+	 * and can compete with the Linux threads for the processor.
+	 */
+	if (switched && !sigs &&
+	    ((sysflags & __xn_exec_switchback) ||
+	     (xnpod_active_p() && xnpod_shadow_p() &&
+	      !xnthread_base_priority(xnshadow_thread(current)))))
 		xnshadow_relax(0, 0);

       ret_handled:


>> Andreas
>>
>> PATCH:
>>
>> diff -ruN linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h
>> --- linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h        2010-04-13 20:02:21.000000000 -0400
>> +++ linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h    2010-04-19 09:35:44.000000000 -0400
>> @@ -186,6 +186,8 @@
>>
>>      xnpqueue_t claimq;               /* Owned resources claimed by others (PIP) */
>>
>> +    int lockcnt;                     /* Mutexes which are currently locked by this thread */
>> +
>>      struct xnsynch *wchan;   /* Resource the thread pends on */
>>
>>      struct xnsynch *wwake;   /* Wait channel the thread was resumed from */
>> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c
>> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c 2010-04-13 20:02:22.000000000 -0400
>> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c     2010-04-19 18:06:39.000000000 -0400
>> @@ -976,6 +976,16 @@
>>       return prio < MAX_RT_PRIO ? prio : MAX_RT_PRIO - 1;
>>  }
>>
>> +static inline int relax_thread(xnthread_t *thread)
>> +{
>> +     /* A thread with bprio == 0 is called a Linux Domain RT thread.
>> +        It has to switch to secondary mode after every skin call
>> +        if it doesn't hold any mutexes. */
>> +     return (xnthread_base_priority(thread) == 0 &&
>> +                     thread->lockcnt == 0)
>> +                     ? 1 : 0;
>> +}
>> +
>>  static int gatekeeper_thread(void *data)
>>  {
>>       struct __gatekeeper *gk = (struct __gatekeeper *)data;
>> @@ -1187,7 +1197,7 @@
>>  void xnshadow_relax(int notify)
>>  {
>>       xnthread_t *thread = xnpod_current_thread();
>> -     int prio;
>> +     int prio, policy;
>>       spl_t s;
>>
>>       XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
>> @@ -1217,9 +1227,11 @@
>>               xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
>>                           thread->name, xnthread_user_pid(thread));
>>
>> +     /* If thread is a Linux Domain RT thread and it should be relaxed the
>> +        base priority should be equal to the current priority equal to 0. */
>> +     policy = relax_thread(thread) ? SCHED_NORMAL : SCHED_FIFO;
>>       prio = normalize_priority(xnthread_current_priority(thread));
>> -     rthal_reenter_root(get_switch_lock_owner(),
>> -                        prio ? SCHED_FIFO : SCHED_NORMAL, prio);
>> +     rthal_reenter_root(get_switch_lock_owner(), policy, prio);
>>
>>       xnstat_counter_inc(&thread->stat.ssw);  /* Account for secondary mode switch. */
>>
>> @@ -2001,8 +2013,14 @@
>>
>>       if (xnpod_shadow_p() && signal_pending(p))
>>               request_syscall_restart(thread, regs, sysflags);
>> -     else if ((sysflags & __xn_exec_switchback) != 0 && switched)
>> -             xnshadow_harden();      /* -EPERM will be trapped later if needed. */
>> +     else {
>> +             int relax = xnpod_shadow_p() && relax_thread(thread);
>> +
>> +             if ((sysflags & __xn_exec_switchback) != 0 && switched && !relax)
>> +                     xnshadow_harden();  /* -EPERM will be trapped later if needed. */
>> +             else if (relax)
>> +                     xnshadow_relax(0);
>> +     }
>>
>>       return RTHAL_EVENT_STOP;
>>
>> @@ -2137,6 +2155,9 @@
>>               request_syscall_restart(xnshadow_thread(current), regs, sysflags);
>>       else if ((sysflags & __xn_exec_switchback) != 0 && switched)
>>               xnshadow_relax(0);
>> +     else if (xnpod_active_p() && xnpod_shadow_p() &&
>> +                      relax_thread(xnshadow_thread(current)))
>> +             xnshadow_relax(0);
>>
>>       return RTHAL_EVENT_STOP;
>>  }
>> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c
>> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c 2010-04-13 20:02:22.000000000 -0400
>> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c     2010-04-19 19:00:39.000000000 -0400
>> @@ -124,6 +124,7 @@
>>       thread->rpi = NULL;
>>  #endif /* CONFIG_XENO_OPT_PRIOCPL */
>>       initpq(&thread->claimq);
>> +     thread->lockcnt = 0;
>>
>>       xnarch_init_display_context(thread);
>>
>> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c
>> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c     2010-04-13 20:02:22.000000000 -0400
>> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c 2010-04-19 09:35:44.000000000 -0400
>> @@ -396,6 +396,8 @@
>>               /* xnsynch_sleep_on() might have stolen the resource,
>>                  so we need to put our internal data in sync. */
>>               mutex->lockcnt = 1;
>> +
>> +             thread->lockcnt++;
>>       }
>>
>>        unlock_and_exit:
>> @@ -462,6 +464,8 @@
>>       if (--mutex->lockcnt > 0)
>>               goto unlock_and_exit;
>>
>> +     xnpod_current_thread()->lockcnt--;
>> +
>>       if (xnsynch_wakeup_one_sleeper(&mutex->synch_base)) {
>>               mutex->lockcnt = 1;
>>               xnpod_schedule();
>>
>
>
> --
> Philippe.
>
>
>
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
>

Thanks,
Henri.

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Henri Roosen wrote:
> On Fri, Apr 23, 2010 at 12:54 PM, Philippe Gerum <rpm@xenomai.org> wrote:
>> On Tue, 2010-04-20 at 10:12 -0400, Andreas Glatz wrote:
>>> Just one more questions: Philippe said that you have something similar in
>>> 2.5. How do you enable it there? By setting the correct sheduling policy?
>>>
>> There are plans to have it. That behavior would be enabled whenever the
>> linux policy is SCHED_OTHER, and the base priority is 0 Xenomai-wise.
>> The latter would be enough for now, but it seems more future-proof not
>> to assume that only SCHED_OTHER tasks could be assigned Xenomai priority
>> 0.
>>
> 
> Is there any update on the plans to have Xenomai base priority 0
> threads to be operating by default in relaxed mode? We are using 2.5.4
> and currently apply the patch below as a workaround. The patch is
> similar to the 2.4 patch of Andreas in that it actively switches back
> base prio 0 shadowed threads into relaxed mode, but doesn't take any
> held locks into account.
> 
> Please comment on this patch. If there is any preliminary code for
> 2.5.x available regarding this functionality I'll be happy to test it
> ;-)

The implementation for this feature needs breaking the ABI, so it will
never be done in 2.5 branch. Fortunately, the 2.6 branch development is
almost started, to fix this issue, as well as a few other ones.

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Thu, 2010-09-30 at 14:52 +0200, Henri Roosen wrote:
> On Fri, Apr 23, 2010 at 12:54 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> > On Tue, 2010-04-20 at 10:12 -0400, Andreas Glatz wrote:
> >> > > > >> For the time being, you can work around this by issuing a Linux syscall
> >> > > > >> before entering long processing loops - unless your task doesn't do this
> >> > > > >> anyway, e.g. to perform some Linux I/O.
> >> > > > >>
> >> > > > >
> >> > > > > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
> >> > > >
> >> > > > The proper way would be to not having to change the application code.
> >> > > > But this workaround (Linux syscall or *_set_mode()) is required until we
> >> > > > improve the nucleus.
> >> > >
> >> > > I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having
> >> > > to review and insert a Linux syscall or *_set_mode() in the application code I just call
> >> > > rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which
> >> > > should mostly run in secondary mode under SCHED_IDLE (see example further down). The task
> >> > > marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately
> >> > > switch back to secondary mode once the Xenomai skincall is done.
> >> > >
> >> > > We identified just one case where this task has to stay in primary mode. This is between
> >> > > rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost.
> >> > > If the task stayed in secondary mode during that time it either would potentally delay the
> >> > > execution of a high priority task or would kill the system.
> >> > >
> >> > > The patch seems to work for us. Our statistics task which blocked the system for a long
> >> > > time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is
> >> > > heavily loaded now the statistics will get out of sync but the UI will still be responsive.
> >> > >
> >> >
> >> > The logic of this patch looks ok for the native skin, given that 2.4.x
> >> > does not provide a centralized implementation for dealing with exclusive
> >> > resources, like 2.5.x with xnsynch_acquire/release, and always emits a
> >> > syscall to manage those resources.
> >> >
> >> > This said, you could spare the T_IDLE tag by assuming that any non-RT
> >> > shadow thread has to switch back to secondary mode after a syscall,
> >> > unless the owned resource count is non-zero. This is where we are
> >> > heading to in 2.5.x, since the preferred mode of operation for such
> >> > thread has to be fundamentally "relaxed" (otherwise, one would have
> >> > created a RT thread, right).
> >> >
> >> > I'm also unsure you should force SCHED_IDLE, instead of picking
> >> > SCHED_OTHER for a more general approach to this issue. You can't assume
> >> > that userland does want to be reniced that way, at least not from the
> >> > nucleus. But I guess this fits your problem though.
> >> >
> >> > To sum up, since we can't really provide a true SCHED_IDLE policy on
> >> > linux (i.e. not a nice-level hack), and implementing a sched class in
> >> > Xenomai having a lower priority than the existing xnsched_class_idle (in
> >> > 2.5.x) is not feasible (we could not run any userland task in it
> >> > anyway), we'd better stick with SCHED_OTHER.
> >> >
> >>
> >> Thanks a lot for the feedback. Your suggestions simplified the patch. I
> >> also changed SCHED_IDLE to SCHED_OTHER since it might be more beneficial
> >> for the broader audience. Any other suggestions?
> >
> > For an even broader audience, the POSIX skin mutexes should be tracked
> > as well.
> >
> >>
> >> After applying this patch, a thread with priority 0 will automatically
> >> switch back to secondary mode after every (native) skincall unless the
> >> task holds a mutex (simple and nested).
> >>
> >> The benefit is, that the task with priority 0 (which I called a linux
> >> domain rt thread)
> >
> > Actually, no. This is not a rt thread at all, in the sense that you have
> > zero guarantee wrt latency in that case. Such a thread is actually a non
> > real-time Xenomai shadow thread, meaning that it may invoke Xenomai
> > services that require the caller to be a Xenomai thread, without
> > real-time support though.
> >
> >>  can issue (native) skincalls and share resources with
> >> high prioritiy task. But it doesn't hold up Linux tasks unless it holds
> >> a mutex since it mostly runs in secondary mode and just switches to
> >> primary mode when needed.
> >>
> >> Just one more questions: Philippe said that you have something similar in
> >> 2.5. How do you enable it there? By setting the correct sheduling policy?
> >>
> >
> > There are plans to have it. That behavior would be enabled whenever the
> > linux policy is SCHED_OTHER, and the base priority is 0 Xenomai-wise.
> > The latter would be enough for now, but it seems more future-proof not
> > to assume that only SCHED_OTHER tasks could be assigned Xenomai priority
> > 0.
> >
> 
> Is there any update on the plans to have Xenomai base priority 0
> threads to be operating by default in relaxed mode? We are using 2.5.4
> and currently apply the patch below as a workaround. The patch is
> similar to the 2.4 patch of Andreas in that it actively switches back
> base prio 0 shadowed threads into relaxed mode, but doesn't take any
> held locks into account.
> 
> Please comment on this patch. If there is any preliminary code for
> 2.5.x available regarding this functionality I'll be happy to test it
> ;-)
> 

I have blackfin festivity to handle right now, but next in queue is this
issue.

-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

Thanks Philippe, Gilles,

Looking forward to this feature!

On Mon, Oct 4, 2010 at 1:02 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> On Thu, 2010-09-30 at 14:52 +0200, Henri Roosen wrote:
>> On Fri, Apr 23, 2010 at 12:54 PM, Philippe Gerum <rpm@xenomai.org> wrote:
>> > On Tue, 2010-04-20 at 10:12 -0400, Andreas Glatz wrote:
>> >> > > > >> For the time being, you can work around this by issuing a Linux syscall
>> >> > > > >> before entering long processing loops - unless your task doesn't do this
>> >> > > > >> anyway, e.g. to perform some Linux I/O.
>> >> > > > >>
>> >> > > > >
>> >> > > > > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
>> >> > > >
>> >> > > > The proper way would be to not having to change the application code.
>> >> > > > But this workaround (Linux syscall or *_set_mode()) is required until we
>> >> > > > improve the nucleus.
>> >> > >
>> >> > > I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having
>> >> > > to review and insert a Linux syscall or *_set_mode() in the application code I just call
>> >> > > rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which
>> >> > > should mostly run in secondary mode under SCHED_IDLE (see example further down). The task
>> >> > > marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately
>> >> > > switch back to secondary mode once the Xenomai skincall is done.
>> >> > >
>> >> > > We identified just one case where this task has to stay in primary mode. This is between
>> >> > > rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost.
>> >> > > If the task stayed in secondary mode during that time it either would potentally delay the
>> >> > > execution of a high priority task or would kill the system.
>> >> > >
>> >> > > The patch seems to work for us. Our statistics task which blocked the system for a long
>> >> > > time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is
>> >> > > heavily loaded now the statistics will get out of sync but the UI will still be responsive.
>> >> > >
>> >> >
>> >> > The logic of this patch looks ok for the native skin, given that 2.4.x
>> >> > does not provide a centralized implementation for dealing with exclusive
>> >> > resources, like 2.5.x with xnsynch_acquire/release, and always emits a
>> >> > syscall to manage those resources.
>> >> >
>> >> > This said, you could spare the T_IDLE tag by assuming that any non-RT
>> >> > shadow thread has to switch back to secondary mode after a syscall,
>> >> > unless the owned resource count is non-zero. This is where we are
>> >> > heading to in 2.5.x, since the preferred mode of operation for such
>> >> > thread has to be fundamentally "relaxed" (otherwise, one would have
>> >> > created a RT thread, right).
>> >> >
>> >> > I'm also unsure you should force SCHED_IDLE, instead of picking
>> >> > SCHED_OTHER for a more general approach to this issue. You can't assume
>> >> > that userland does want to be reniced that way, at least not from the
>> >> > nucleus. But I guess this fits your problem though.
>> >> >
>> >> > To sum up, since we can't really provide a true SCHED_IDLE policy on
>> >> > linux (i.e. not a nice-level hack), and implementing a sched class in
>> >> > Xenomai having a lower priority than the existing xnsched_class_idle (in
>> >> > 2.5.x) is not feasible (we could not run any userland task in it
>> >> > anyway), we'd better stick with SCHED_OTHER.
>> >> >
>> >>
>> >> Thanks a lot for the feedback. Your suggestions simplified the patch. I
>> >> also changed SCHED_IDLE to SCHED_OTHER since it might be more beneficial
>> >> for the broader audience. Any other suggestions?
>> >
>> > For an even broader audience, the POSIX skin mutexes should be tracked
>> > as well.
>> >
>> >>
>> >> After applying this patch, a thread with priority 0 will automatically
>> >> switch back to secondary mode after every (native) skincall unless the
>> >> task holds a mutex (simple and nested).
>> >>
>> >> The benefit is, that the task with priority 0 (which I called a linux
>> >> domain rt thread)
>> >
>> > Actually, no. This is not a rt thread at all, in the sense that you have
>> > zero guarantee wrt latency in that case. Such a thread is actually a non
>> > real-time Xenomai shadow thread, meaning that it may invoke Xenomai
>> > services that require the caller to be a Xenomai thread, without
>> > real-time support though.
>> >
>> >>  can issue (native) skincalls and share resources with
>> >> high prioritiy task. But it doesn't hold up Linux tasks unless it holds
>> >> a mutex since it mostly runs in secondary mode and just switches to
>> >> primary mode when needed.
>> >>
>> >> Just one more questions: Philippe said that you have something similar in
>> >> 2.5. How do you enable it there? By setting the correct sheduling policy?
>> >>
>> >
>> > There are plans to have it. That behavior would be enabled whenever the
>> > linux policy is SCHED_OTHER, and the base priority is 0 Xenomai-wise.
>> > The latter would be enough for now, but it seems more future-proof not
>> > to assume that only SCHED_OTHER tasks could be assigned Xenomai priority
>> > 0.
>> >
>>
>> Is there any update on the plans to have Xenomai base priority 0
>> threads to be operating by default in relaxed mode? We are using 2.5.4
>> and currently apply the patch below as a workaround. The patch is
>> similar to the 2.4 patch of Andreas in that it actively switches back
>> base prio 0 shadowed threads into relaxed mode, but doesn't take any
>> held locks into account.
>>
>> Please comment on this patch. If there is any preliminary code for
>> 2.5.x available regarding this functionality I'll be happy to test it
>> ;-)
>>
>
> I have blackfin festivity to handle right now, but next in queue is this
> issue.
>
> --
> Philippe.
>
>
>

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Thu, 2010-09-30 at 14:52 +0200, Henri Roosen wrote:
> On Fri, Apr 23, 2010 at 12:54 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> > On Tue, 2010-04-20 at 10:12 -0400, Andreas Glatz wrote:
> >> > > > >> For the time being, you can work around this by issuing a Linux syscall
> >> > > > >> before entering long processing loops - unless your task doesn't do this
> >> > > > >> anyway, e.g. to perform some Linux I/O.
> >> > > > >>
> >> > > > >
> >> > > > > I think that's need. Currently the statistics task takes a mutex and waits on a message queue for messages. That's the only time it should potentially run in primary mode. After it returns the Mutex it should continue running with a policy similar to SCHED_IDLE to give other tasks a chance to run. I see how switching back to secondary mode could be achieved by issuing a Linux syscall. Is there another way which doesn't involve changing the source code of our application? (The proper way?)
> >> > > >
> >> > > > The proper way would be to not having to change the application code.
> >> > > > But this workaround (Linux syscall or *_set_mode()) is required until we
> >> > > > improve the nucleus.
> >> > >
> >> > > I generated a patch against 2.4.10.1 to get this behaviour (see further down). Instead of having
> >> > > to review and insert a Linux syscall or *_set_mode() in the application code I just call
> >> > > rt_task_set_mode(0, T_IDLE, NULL) at the beginning of the task body of the task which
> >> > > should mostly run in secondary mode under SCHED_IDLE (see example further down). The task
> >> > > marked with T_IDLE will switch to primary mode at every Xenomai skincall and immediately
> >> > > switch back to secondary mode once the Xenomai skincall is done.
> >> > >
> >> > > We identified just one case where this task has to stay in primary mode. This is between
> >> > > rt_mutex_aquire() and rt_mutex_release() since it may undergo a priority inversion boost.
> >> > > If the task stayed in secondary mode during that time it either would potentally delay the
> >> > > execution of a high priority task or would kill the system.
> >> > >
> >> > > The patch seems to work for us. Our statistics task which blocked the system for a long
> >> > > time (and made the UI running under Linux unresponsive) is running with T_IDLE. If Linux is
> >> > > heavily loaded now the statistics will get out of sync but the UI will still be responsive.
> >> > >
> >> >
> >> > The logic of this patch looks ok for the native skin, given that 2.4.x
> >> > does not provide a centralized implementation for dealing with exclusive
> >> > resources, like 2.5.x with xnsynch_acquire/release, and always emits a
> >> > syscall to manage those resources.
> >> >
> >> > This said, you could spare the T_IDLE tag by assuming that any non-RT
> >> > shadow thread has to switch back to secondary mode after a syscall,
> >> > unless the owned resource count is non-zero. This is where we are
> >> > heading to in 2.5.x, since the preferred mode of operation for such
> >> > thread has to be fundamentally "relaxed" (otherwise, one would have
> >> > created a RT thread, right).
> >> >
> >> > I'm also unsure you should force SCHED_IDLE, instead of picking
> >> > SCHED_OTHER for a more general approach to this issue. You can't assume
> >> > that userland does want to be reniced that way, at least not from the
> >> > nucleus. But I guess this fits your problem though.
> >> >
> >> > To sum up, since we can't really provide a true SCHED_IDLE policy on
> >> > linux (i.e. not a nice-level hack), and implementing a sched class in
> >> > Xenomai having a lower priority than the existing xnsched_class_idle (in
> >> > 2.5.x) is not feasible (we could not run any userland task in it
> >> > anyway), we'd better stick with SCHED_OTHER.
> >> >
> >>
> >> Thanks a lot for the feedback. Your suggestions simplified the patch. I
> >> also changed SCHED_IDLE to SCHED_OTHER since it might be more beneficial
> >> for the broader audience. Any other suggestions?
> >
> > For an even broader audience, the POSIX skin mutexes should be tracked
> > as well.
> >
> >>
> >> After applying this patch, a thread with priority 0 will automatically
> >> switch back to secondary mode after every (native) skincall unless the
> >> task holds a mutex (simple and nested).
> >>
> >> The benefit is, that the task with priority 0 (which I called a linux
> >> domain rt thread)
> >
> > Actually, no. This is not a rt thread at all, in the sense that you have
> > zero guarantee wrt latency in that case. Such a thread is actually a non
> > real-time Xenomai shadow thread, meaning that it may invoke Xenomai
> > services that require the caller to be a Xenomai thread, without
> > real-time support though.
> >
> >>  can issue (native) skincalls and share resources with
> >> high prioritiy task. But it doesn't hold up Linux tasks unless it holds
> >> a mutex since it mostly runs in secondary mode and just switches to
> >> primary mode when needed.
> >>
> >> Just one more questions: Philippe said that you have something similar in
> >> 2.5. How do you enable it there? By setting the correct sheduling policy?
> >>
> >
> > There are plans to have it. That behavior would be enabled whenever the
> > linux policy is SCHED_OTHER, and the base priority is 0 Xenomai-wise.
> > The latter would be enough for now, but it seems more future-proof not
> > to assume that only SCHED_OTHER tasks could be assigned Xenomai priority
> > 0.
> >
> 
> Is there any update on the plans to have Xenomai base priority 0
> threads to be operating by default in relaxed mode? We are using 2.5.4
> and currently apply the patch below as a workaround. The patch is
> similar to the 2.4 patch of Andreas in that it actively switches back
> base prio 0 shadowed threads into relaxed mode, but doesn't take any
> held locks into account.
> 
> Please comment on this patch. If there is any preliminary code for
> 2.5.x available regarding this functionality I'll be happy to test it
> ;-)

The auto-relax feature is now available from -head (upcoming 2.6.x). You
will need all commits from
http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
to
http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da

-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:

> One thing I've noticed though, and this is not related to the patch (I verified it on a 
> vanilla Xenomai system): Consider the example I included. It prints average cycle times 
> and the cycle time variance of the high priority task ("T2"). I noticed a big difference 
> in the cycle time variance when switching the first task ("T1") to secondary mode with 
> rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or 
> SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :) 
> Can someone give me a short explanation or point me somewhere to get an explanation for
> this behaviour? I didn't expect such a difference in variance:
> 

Moving back to secondary mode is nothing more than triggering the
rescheduling procedure linux-wise, so the time credit for
SCHED_OTHER/IDLE tasks decay as usual during your work loop, preemption
by high priority task is more likely and so on. This variance is just
the sign that those tasks cannot ask for more than what their scheduling
policy grants.

> SCHED_FIFO:
> task=2 count=1000 average=10505us variance=851(us)^2
> task=2 count=1000 average=10504us variance=176(us)^2
> task=2 count=1000 average=10505us variance=716(us)^2
> task=2 count=1000 average=10504us variance=148(us)^2
> task=2 count=1000 average=10504us variance=143(us)^2
> task=2 count=1000 average=10504us variance=141(us)^2
> task=2 count=1000 average=10504us variance=138(us)^2
> 
> SCHED_NORMAL:
> task=2 count=1000 average=10501us variance=2115(us)^2
> task=2 count=1000 average=10504us variance=3121(us)^2
> task=2 count=1000 average=10500us variance=161(us)^2
> task=2 count=1000 average=10501us variance=1136(us)^2
> task=2 count=1000 average=10500us variance=194(us)^2
> task=2 count=1000 average=10501us variance=1971(us)^2
> task=2 count=1000 average=10500us variance=132(us)^2
> task=2 count=1000 average=10501us variance=1173(us)^2
> 
> SCHED_IDLE:
> task=2 count=1000 average=10504us variance=3413(us)^2
> task=2 count=1000 average=10503us variance=3567(us)^2
> task=2 count=1000 average=10504us variance=3409(us)^2
> task=2 count=1000 average=10504us variance=1743(us)^2
> task=2 count=1000 average=10504us variance=2710(us)^2
> task=2 count=1000 average=10504us variance=2548(us)^2
> task=2 count=1000 average=10504us variance=2364(us)^2
> task=2 count=1000 average=10504us variance=2867(us)^2
> task=2 count=1000 average=10504us variance=2755(us)^2
> 
> 
> Regards, Andreas
> 
> (Xenomai 2.4.10.1, Linux 2.6.32, Ipipe 2.8)
> 
> 
> EXAMPLE APPLICATION:
> 
> #include <stdio.h>
> #include <unistd.h>
> #include <native/task.h>
> #include <native/sem.h>
> #include <native/mutex.h>
> #include <sys/mman.h>
> #include <rtdk.h>
> #include <sched.h>
> #include <linux/sched.h>
> 
> typedef struct test {
>     RTIME timestamp;
>     RTIME sum;
>     RTIME sumsq;
>     int count;
> } test_t;
> 
> static test_t test1 = {0, 0, 0, 0}, test2 = {0, 0, 0, 0};
> static RT_TASK task1, task2;
> static RT_MUTEX mutex1, mutex2;
> 
> static void task_body( void* cookie )
> {
>     RTIME timestamp, delta;
>     test_t* ptr = (test_t*)cookie;
>     int num = ( ptr == &test1) ? 1 : 2;
> 
>     if( num == 1 ) rt_task_set_mode(0, 0x00080000, NULL);
> 
>     while(1)
>     {
>         rt_mutex_acquire(&mutex1, TM_INFINITE);
> 
>         timestamp = __xn_rdtsc();
>         delta = (timestamp - ptr->timestamp) >> 6 /* in us */;
>         ptr->sum += delta;
>         ptr->sumsq += delta*delta;
>         ptr->timestamp = timestamp;
>         if( ++ptr->count >= 1000 ) {
>             RTIME avg, var;
>             avg = ptr->sum/ptr->count;
>             var = (ptr->sumsq - (ptr->sum*ptr->sum)/ptr->count)/(ptr->count-1);
> 
>             if( num == 2 )
>                 rt_printf("task=%d count=%d average=%lluus variance=%llu(us)^2\n",
>                           num, ptr->count, avg, var);
> 
>             ptr->sum = 0;
>             ptr->sumsq = 0;
>             ptr->count = 0;
>         }
> 
> 		// If commented T1 basically runs in a while(1) {}
> 		// loop without any sleeps. UI should be responsive
> 		// since T1 runs with SCHED_IDLE.
> 		// If uncommented the sleep causes T2 to boost the
> 		// priority of T1. 
>         if( num == 1 ) rt_task_sleep(10000000);
> 
> 		// T1 automatically switches to secondary mode after
> 		// this call.
>         rt_mutex_release(&mutex1);
> 
>         // Give T1 time to run
>         if( num == 2 ) rt_task_sleep(10000000);
>     }
> }
> 
> int main(int argc, char* argv[])
> {
>     int err;
> 
>     mlockall(MCL_CURRENT|MCL_FUTURE);
> 
>     rt_print_auto_init(1);
> 
>     test1.timestamp = test2.timestamp = __xn_rdtsc();
>     err  = rt_mutex_create(&mutex1, "M1");
>     err += rt_mutex_create(&mutex2, "M2");
>     err += rt_task_spawn(&task1, "T1", 0, 33, 0, task_body, (void*)&test1);
>     err += rt_task_spawn(&task2, "T2", 0, 66, 0, task_body, (void*)&test2);
>     if( !err )
>     {
>         pause();
>     }
> 
>     return err;
> }
> 
> IDLE PATCH:
> 
> diff -ruN linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h
> --- linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h	2010-04-13 20:02:21.000000000 -0400
> +++ linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h	2010-04-14 10:38:09.000000000 -0400
> @@ -89,6 +89,8 @@
>  #define __xn_exec_adaptive   0x40
>  /* Do not restart syscall upon signal receipt. */
>  #define __xn_exec_norestart  0x80
> +/* Do not switch to secondary mode after syscall if thread has XNIDLE flag set (see #XNIDLE) */
> +#define __xn_exec_norelax    0x100
>  /* Context-agnostic syscall. Will actually run in Xenomai domain. */
>  #define __xn_exec_any        0x0
>  /* Short-hand for shadow init syscall. */
> diff -ruN linux-2.6.32-5RR9/include/xenomai/native/task.h linux-2.6.32-5RR9-new/include/xenomai/native/task.h
> --- linux-2.6.32-5RR9/include/xenomai/native/task.h	2010-04-13 20:02:21.000000000 -0400
> +++ linux-2.6.32-5RR9-new/include/xenomai/native/task.h	2010-04-14 10:38:09.000000000 -0400
> @@ -52,6 +52,7 @@
>  #define T_SHIELD   XNSHIELD   /**< See #XNSHIELD  */ 
>  #define T_WARNSW   XNTRAPSW   /**< See #XNTRAPSW  */ 
>  #define T_RPIOFF   XNRPIOFF   /**< See #XNRPIOFF  */ 
> +#define T_IDLE     XNIDLE     /**< See #XNIDLE    */
>  #define T_PRIMARY  0x00000200	/* Recycle internal bits status which */
>  #define T_JOINABLE 0x00000400	/* won't be passed to the nucleus.  */
>  /*! @} */ /* Ends doxygen-group native_task_status */
> diff -ruN linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h
> --- linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h	2010-04-13 20:02:21.000000000 -0400
> +++ linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h	2010-04-14 10:38:09.000000000 -0400
> @@ -55,6 +55,7 @@
>  #define XNSHIELD  0x00010000 /**< IRQ shield is enabled (shadow only) */
>  #define XNTRAPSW  0x00020000 /**< Trap execution mode switches */
>  #define XNRPIOFF  0x00040000 /**< Stop priority coupling (shadow only) */
> +#define XNIDLE    0x00080000 /**< Switches to secondary mode after syscalls if not holding mutexes */
>  
>  #define XNFPU     0x00100000 /**< Thread uses FPU */
>  #define XNSHADOW  0x00200000 /**< Shadow thread */
> @@ -90,7 +91,7 @@
>  }
>  
>  #define XNTHREAD_BLOCK_BITS   (XNSUSP|XNPEND|XNDELAY|XNDORMANT|XNRELAX|XNHELD)
> -#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF)
> +#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF|XNIDLE)
>  
>  /* These state flags are available to the real-time interfaces */
>  #define XNTHREAD_STATE_SPARE0  0x10000000
> @@ -186,6 +187,8 @@
>  
>      xnpqueue_t claimq;		/* Owned resources claimed by others (PIP) */
>  
> +    int lockcnt;			/* Mutexes which are currently locked by this thread */
> +
>      struct xnsynch *wchan;	/* Resource the thread pends on */
>  
>      struct xnsynch *wwake;	/* Wait channel the thread was resumed from */
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c
> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c	2010-04-14 18:04:20.000000000 -0400
> @@ -1187,7 +1187,7 @@
>  void xnshadow_relax(int notify)
>  {
>  	xnthread_t *thread = xnpod_current_thread();
> -	int prio;
> +	int prio, policy;
>  	spl_t s;
>  
>  	XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
> @@ -1217,9 +1217,9 @@
>  		xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
>  			    thread->name, xnthread_user_pid(thread));
>  
> -	prio = normalize_priority(xnthread_current_priority(thread));
> -	rthal_reenter_root(get_switch_lock_owner(),
> -			   prio ? SCHED_FIFO : SCHED_NORMAL, prio);
> +	prio = xnthread_test_state(thread, XNIDLE) ? 0 : normalize_priority(xnthread_current_priority(thread));
> +	policy = xnthread_test_state(thread, XNIDLE) ? SCHED_IDLE : (prio ? SCHED_FIFO : SCHED_NORMAL);
> +	rthal_reenter_root(get_switch_lock_owner(), policy, prio);
>  
>  	xnstat_counter_inc(&thread->stat.ssw);	/* Account for secondary mode switch. */
>  
> @@ -2001,8 +2001,13 @@
>  
>  	if (xnpod_shadow_p() && signal_pending(p))
>  		request_syscall_restart(thread, regs, sysflags);
> -	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
> -		xnshadow_harden();	/* -EPERM will be trapped later if needed. */
> +	else if ((sysflags & __xn_exec_switchback) != 0 && switched) {
> +		if (!xnthread_test_state(thread, XNIDLE) || 
> +		    (xnthread_test_state(thread, XNIDLE) && (sysflags & __xn_exec_norelax) != 0))
> +		xnshadow_harden();  /* -EPERM will be trapped later if needed. */
> +	} else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() && 
> +			   xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
> +		xnshadow_relax(0);
>  
>  	return RTHAL_EVENT_STOP;
>  
> @@ -2137,6 +2142,9 @@
>  		request_syscall_restart(xnshadow_thread(current), regs, sysflags);
>  	else if ((sysflags & __xn_exec_switchback) != 0 && switched)
>  		xnshadow_relax(0);
> +	else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() && 
> +			 xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
> +		xnshadow_relax(0);
>  
>  	return RTHAL_EVENT_STOP;
>  }
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c
> --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c	2010-04-14 10:38:09.000000000 -0400
> @@ -124,6 +124,7 @@
>  	thread->rpi = NULL;
>  #endif /* CONFIG_XENO_OPT_PRIOCPL */
>  	initpq(&thread->claimq);
> +	thread->lockcnt = 0;
>  
>  	xnarch_init_display_context(thread);
>  
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c
> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c	2010-04-14 10:38:09.000000000 -0400
> @@ -396,6 +396,8 @@
>  		/* xnsynch_sleep_on() might have stolen the resource,
>  		   so we need to put our internal data in sync. */
>  		mutex->lockcnt = 1;
> +		
> +		thread->lockcnt++;
>  	}
>  
>        unlock_and_exit:
> @@ -462,6 +464,8 @@
>  	if (--mutex->lockcnt > 0)
>  		goto unlock_and_exit;
>  
> +	xnpod_current_thread()->lockcnt--;
> +
>  	if (xnsynch_wakeup_one_sleeper(&mutex->synch_base)) {
>  		mutex->lockcnt = 1;
>  		xnpod_schedule();
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c
> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c	2010-04-14 10:38:09.000000000 -0400
> @@ -3932,7 +3932,7 @@
>  	[__native_mutex_create] = {&__rt_mutex_create, __xn_exec_any},
>  	[__native_mutex_bind] = {&__rt_mutex_bind, __xn_exec_conforming},
>  	[__native_mutex_delete] = {&__rt_mutex_delete, __xn_exec_any},
> -	[__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary},
> +	[__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary|__xn_exec_norelax},
>  	[__native_mutex_release] = {&__rt_mutex_release, __xn_exec_primary},
>  	[__native_mutex_inquire] = {&__rt_mutex_inquire, __xn_exec_any},
>  	[__native_cond_create] = {&__rt_cond_create, __xn_exec_any},
> diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c
> --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c	2010-04-13 20:02:22.000000000 -0400
> +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c	2010-04-14 10:38:09.000000000 -0400
> @@ -1498,7 +1498,7 @@
>  	}
>  
>  	if (((clrmask | setmask) &
> -	     ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW)) != 0)
> +	     ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW | T_IDLE)) != 0)
>  		return -EINVAL;
>  
>  	if (!xnpod_primary_p())
> 
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help


-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Andreas Glatz]

Hi,

On Thu, Apr 15, 2010 at 06:46:58AM -0400, Philippe Gerum wrote:
> On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:
> 
> > One thing I've noticed though, and this is not related to the patch (I verified it on a
> > vanilla Xenomai system): Consider the example I included. It prints average cycle times
> > and the cycle time variance of the high priority task ("T2"). I noticed a big difference
> > in the cycle time variance when switching the first task ("T1") to secondary mode with
> > rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or
> > SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :)
> > Can someone give me a short explanation or point me somewhere to get an explanation for
> > this behaviour? I didn't expect such a difference in variance:
> >
> 
> Moving back to secondary mode is nothing more than triggering the
> rescheduling procedure linux-wise, so the time credit for
> SCHED_OTHER/IDLE tasks decay as usual during your work loop, preemption
> by high priority task is more likely and so on. This variance is just
> the sign that those tasks cannot ask for more than what their scheduling
> policy grants.
> 

I think, I didn't express myself clearly enough. I was more puzzled about 
the variance of the pure RT task (task w/o any secondary mode switches).
So it seems that changing the scheduling policy of the "relaxed" task has
an influence on the variance of the pure RT task. So the RT task seems
to wait for the "relaxed" task. But where exacly does it wait for it? Mmmmm...

Andreas


> > SCHED_FIFO:
> > task=2 count=1000 average=10505us variance=851(us)^2
> > task=2 count=1000 average=10504us variance=176(us)^2
> > task=2 count=1000 average=10505us variance=716(us)^2
> > task=2 count=1000 average=10504us variance=148(us)^2
> > task=2 count=1000 average=10504us variance=143(us)^2
> > task=2 count=1000 average=10504us variance=141(us)^2
> > task=2 count=1000 average=10504us variance=138(us)^2
> >
> > SCHED_NORMAL:
> > task=2 count=1000 average=10501us variance=2115(us)^2
> > task=2 count=1000 average=10504us variance=3121(us)^2
> > task=2 count=1000 average=10500us variance=161(us)^2
> > task=2 count=1000 average=10501us variance=1136(us)^2
> > task=2 count=1000 average=10500us variance=194(us)^2
> > task=2 count=1000 average=10501us variance=1971(us)^2
> > task=2 count=1000 average=10500us variance=132(us)^2
> > task=2 count=1000 average=10501us variance=1173(us)^2
> >
> > SCHED_IDLE:
> > task=2 count=1000 average=10504us variance=3413(us)^2
> > task=2 count=1000 average=10503us variance=3567(us)^2
> > task=2 count=1000 average=10504us variance=3409(us)^2
> > task=2 count=1000 average=10504us variance=1743(us)^2
> > task=2 count=1000 average=10504us variance=2710(us)^2
> > task=2 count=1000 average=10504us variance=2548(us)^2
> > task=2 count=1000 average=10504us variance=2364(us)^2
> > task=2 count=1000 average=10504us variance=2867(us)^2
> > task=2 count=1000 average=10504us variance=2755(us)^2
> >
> >
> > Regards, Andreas
> >
> > (Xenomai 2.4.10.1, Linux 2.6.32, Ipipe 2.8)
> >
> >
> > EXAMPLE APPLICATION:
> >
> > #include <stdio.h>
> > #include <unistd.h>
> > #include <native/task.h>
> > #include <native/sem.h>
> > #include <native/mutex.h>
> > #include <sys/mman.h>
> > #include <rtdk.h>
> > #include <sched.h>
> > #include <linux/sched.h>
> >
> > typedef struct test {
> >     RTIME timestamp;
> >     RTIME sum;
> >     RTIME sumsq;
> >     int count;
> > } test_t;
> >
> > static test_t test1 = {0, 0, 0, 0}, test2 = {0, 0, 0, 0};
> > static RT_TASK task1, task2;
> > static RT_MUTEX mutex1, mutex2;
> >
> > static void task_body( void* cookie )
> > {
> >     RTIME timestamp, delta;
> >     test_t* ptr = (test_t*)cookie;
> >     int num = ( ptr == &test1) ? 1 : 2;
> >
> >     if( num == 1 ) rt_task_set_mode(0, 0x00080000, NULL);
> >
> >     while(1)
> >     {
> >         rt_mutex_acquire(&mutex1, TM_INFINITE);
> >
> >         timestamp = __xn_rdtsc();
> >         delta = (timestamp - ptr->timestamp) >> 6 /* in us */;
> >         ptr->sum += delta;
> >         ptr->sumsq += delta*delta;
> >         ptr->timestamp = timestamp;
> >         if( ++ptr->count >= 1000 ) {
> >             RTIME avg, var;
> >             avg = ptr->sum/ptr->count;
> >             var = (ptr->sumsq - (ptr->sum*ptr->sum)/ptr->count)/(ptr->count-1);
> >
> >             if( num == 2 )
> >                 rt_printf("task=%d count=%d average=%lluus variance=%llu(us)^2\n",
> >                           num, ptr->count, avg, var);
> >
> >             ptr->sum = 0;
> >             ptr->sumsq = 0;
> >             ptr->count = 0;
> >         }
> >
> >               // If commented T1 basically runs in a while(1) {}
> >               // loop without any sleeps. UI should be responsive
> >               // since T1 runs with SCHED_IDLE.
> >               // If uncommented the sleep causes T2 to boost the
> >               // priority of T1.
> >         if( num == 1 ) rt_task_sleep(10000000);
> >
> >               // T1 automatically switches to secondary mode after
> >               // this call.
> >         rt_mutex_release(&mutex1);
> >
> >         // Give T1 time to run
> >         if( num == 2 ) rt_task_sleep(10000000);
> >     }
> > }
> >
> > int main(int argc, char* argv[])
> > {
> >     int err;
> >
> >     mlockall(MCL_CURRENT|MCL_FUTURE);
> >
> >     rt_print_auto_init(1);
> >
> >     test1.timestamp = test2.timestamp = __xn_rdtsc();
> >     err  = rt_mutex_create(&mutex1, "M1");
> >     err += rt_mutex_create(&mutex2, "M2");
> >     err += rt_task_spawn(&task1, "T1", 0, 33, 0, task_body, (void*)&test1);
> >     err += rt_task_spawn(&task2, "T2", 0, 66, 0, task_body, (void*)&test2);
> >     if( !err )
> >     {
> >         pause();
> >     }
> >
> >     return err;
> > }
> >
> > IDLE PATCH:
> >
> > diff -ruN linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h
> > --- linux-2.6.32-5RR9/include/asm-generic/xenomai/syscall.h   2010-04-13 20:02:21.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/include/asm-generic/xenomai/syscall.h       2010-04-14 10:38:09.000000000 -0400
> > @@ -89,6 +89,8 @@
> >  #define __xn_exec_adaptive   0x40
> >  /* Do not restart syscall upon signal receipt. */
> >  #define __xn_exec_norestart  0x80
> > +/* Do not switch to secondary mode after syscall if thread has XNIDLE flag set (see #XNIDLE) */
> > +#define __xn_exec_norelax    0x100
> >  /* Context-agnostic syscall. Will actually run in Xenomai domain. */
> >  #define __xn_exec_any        0x0
> >  /* Short-hand for shadow init syscall. */
> > diff -ruN linux-2.6.32-5RR9/include/xenomai/native/task.h linux-2.6.32-5RR9-new/include/xenomai/native/task.h
> > --- linux-2.6.32-5RR9/include/xenomai/native/task.h   2010-04-13 20:02:21.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/include/xenomai/native/task.h       2010-04-14 10:38:09.000000000 -0400
> > @@ -52,6 +52,7 @@
> >  #define T_SHIELD   XNSHIELD   /**< See #XNSHIELD  */
> >  #define T_WARNSW   XNTRAPSW   /**< See #XNTRAPSW  */
> >  #define T_RPIOFF   XNRPIOFF   /**< See #XNRPIOFF  */
> > +#define T_IDLE     XNIDLE     /**< See #XNIDLE    */
> >  #define T_PRIMARY  0x00000200        /* Recycle internal bits status which */
> >  #define T_JOINABLE 0x00000400        /* won't be passed to the nucleus.  */
> >  /*! @} */ /* Ends doxygen-group native_task_status */
> > diff -ruN linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h
> > --- linux-2.6.32-5RR9/include/xenomai/nucleus/thread.h        2010-04-13 20:02:21.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/include/xenomai/nucleus/thread.h    2010-04-14 10:38:09.000000000 -0400
> > @@ -55,6 +55,7 @@
> >  #define XNSHIELD  0x00010000 /**< IRQ shield is enabled (shadow only) */
> >  #define XNTRAPSW  0x00020000 /**< Trap execution mode switches */
> >  #define XNRPIOFF  0x00040000 /**< Stop priority coupling (shadow only) */
> > +#define XNIDLE    0x00080000 /**< Switches to secondary mode after syscalls if not holding mutexes */
> >
> >  #define XNFPU     0x00100000 /**< Thread uses FPU */
> >  #define XNSHADOW  0x00200000 /**< Shadow thread */
> > @@ -90,7 +91,7 @@
> >  }
> >
> >  #define XNTHREAD_BLOCK_BITS   (XNSUSP|XNPEND|XNDELAY|XNDORMANT|XNRELAX|XNHELD)
> > -#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF)
> > +#define XNTHREAD_MODE_BITS    (XNLOCK|XNRRB|XNASDI|XNSHIELD|XNTRAPSW|XNRPIOFF|XNIDLE)
> >
> >  /* These state flags are available to the real-time interfaces */
> >  #define XNTHREAD_STATE_SPARE0  0x10000000
> > @@ -186,6 +187,8 @@
> >
> >      xnpqueue_t claimq;               /* Owned resources claimed by others (PIP) */
> >
> > +    int lockcnt;                     /* Mutexes which are currently locked by this thread */
> > +
> >      struct xnsynch *wchan;   /* Resource the thread pends on */
> >
> >      struct xnsynch *wwake;   /* Wait channel the thread was resumed from */
> > diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c
> > --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/shadow.c 2010-04-13 20:02:22.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/shadow.c     2010-04-14 18:04:20.000000000 -0400
> > @@ -1187,7 +1187,7 @@
> >  void xnshadow_relax(int notify)
> >  {
> >       xnthread_t *thread = xnpod_current_thread();
> > -     int prio;
> > +     int prio, policy;
> >       spl_t s;
> >
> >       XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
> > @@ -1217,9 +1217,9 @@
> >               xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
> >                           thread->name, xnthread_user_pid(thread));
> >
> > -     prio = normalize_priority(xnthread_current_priority(thread));
> > -     rthal_reenter_root(get_switch_lock_owner(),
> > -                        prio ? SCHED_FIFO : SCHED_NORMAL, prio);
> > +     prio = xnthread_test_state(thread, XNIDLE) ? 0 : normalize_priority(xnthread_current_priority(thread));
> > +     policy = xnthread_test_state(thread, XNIDLE) ? SCHED_IDLE : (prio ? SCHED_FIFO : SCHED_NORMAL);
> > +     rthal_reenter_root(get_switch_lock_owner(), policy, prio);
> >
> >       xnstat_counter_inc(&thread->stat.ssw);  /* Account for secondary mode switch. */
> >
> > @@ -2001,8 +2001,13 @@
> >
> >       if (xnpod_shadow_p() && signal_pending(p))
> >               request_syscall_restart(thread, regs, sysflags);
> > -     else if ((sysflags & __xn_exec_switchback) != 0 && switched)
> > -             xnshadow_harden();      /* -EPERM will be trapped later if needed. */
> > +     else if ((sysflags & __xn_exec_switchback) != 0 && switched) {
> > +             if (!xnthread_test_state(thread, XNIDLE) ||
> > +                 (xnthread_test_state(thread, XNIDLE) && (sysflags & __xn_exec_norelax) != 0))
> > +             xnshadow_harden();  /* -EPERM will be trapped later if needed. */
> > +     } else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() &&
> > +                        xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
> > +             xnshadow_relax(0);
> >
> >       return RTHAL_EVENT_STOP;
> >
> > @@ -2137,6 +2142,9 @@
> >               request_syscall_restart(xnshadow_thread(current), regs, sysflags);
> >       else if ((sysflags & __xn_exec_switchback) != 0 && switched)
> >               xnshadow_relax(0);
> > +     else if ((sysflags & __xn_exec_norelax) == 0 && xnpod_primary_p() &&
> > +                      xnpod_current_thread()->lockcnt == 0 && xnthread_test_state(thread, XNIDLE))
> > +             xnshadow_relax(0);
> >
> >       return RTHAL_EVENT_STOP;
> >  }
> > diff -ruN linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c
> > --- linux-2.6.32-5RR9/kernel/xenomai/nucleus/thread.c 2010-04-13 20:02:22.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/kernel/xenomai/nucleus/thread.c     2010-04-14 10:38:09.000000000 -0400
> > @@ -124,6 +124,7 @@
> >       thread->rpi = NULL;
> >  #endif /* CONFIG_XENO_OPT_PRIOCPL */
> >       initpq(&thread->claimq);
> > +     thread->lockcnt = 0;
> >
> >       xnarch_init_display_context(thread);
> >
> > diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c
> > --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/mutex.c     2010-04-13 20:02:22.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/mutex.c 2010-04-14 10:38:09.000000000 -0400
> > @@ -396,6 +396,8 @@
> >               /* xnsynch_sleep_on() might have stolen the resource,
> >                  so we need to put our internal data in sync. */
> >               mutex->lockcnt = 1;
> > +
> > +             thread->lockcnt++;
> >       }
> >
> >        unlock_and_exit:
> > @@ -462,6 +464,8 @@
> >       if (--mutex->lockcnt > 0)
> >               goto unlock_and_exit;
> >
> > +     xnpod_current_thread()->lockcnt--;
> > +
> >       if (xnsynch_wakeup_one_sleeper(&mutex->synch_base)) {
> >               mutex->lockcnt = 1;
> >               xnpod_schedule();
> > diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c
> > --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/syscall.c   2010-04-13 20:02:22.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/syscall.c       2010-04-14 10:38:09.000000000 -0400
> > @@ -3932,7 +3932,7 @@
> >       [__native_mutex_create] = {&__rt_mutex_create, __xn_exec_any},
> >       [__native_mutex_bind] = {&__rt_mutex_bind, __xn_exec_conforming},
> >       [__native_mutex_delete] = {&__rt_mutex_delete, __xn_exec_any},
> > -     [__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary},
> > +     [__native_mutex_acquire] = {&__rt_mutex_acquire, __xn_exec_primary|__xn_exec_norelax},
> >       [__native_mutex_release] = {&__rt_mutex_release, __xn_exec_primary},
> >       [__native_mutex_inquire] = {&__rt_mutex_inquire, __xn_exec_any},
> >       [__native_cond_create] = {&__rt_cond_create, __xn_exec_any},
> > diff -ruN linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c
> > --- linux-2.6.32-5RR9/kernel/xenomai/skins/native/task.c      2010-04-13 20:02:22.000000000 -0400
> > +++ linux-2.6.32-5RR9-new/kernel/xenomai/skins/native/task.c  2010-04-14 10:38:09.000000000 -0400
> > @@ -1498,7 +1498,7 @@
> >       }
> >
> >       if (((clrmask | setmask) &
> > -          ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW)) != 0)
> > +          ~(T_LOCK | T_RRB | T_NOSIG | T_SHIELD | T_WARNSW | T_IDLE)) != 0)
> >               return -EINVAL;
> >
> >       if (!xnpod_primary_p())
> >
> >
> > _______________________________________________
> > Xenomai-help mailing list
> > Xenomai-help@domain.hid
> > https://mail.gna.org/listinfo/xenomai-help
> 
> 
> --
> Philippe.
> 
> 

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Andreas Glatz wrote:
> Hi,
> 
> On Thu, Apr 15, 2010 at 06:46:58AM -0400, Philippe Gerum wrote:
>> On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:
>>
>>> One thing I've noticed though, and this is not related to the patch (I verified it on a
>>> vanilla Xenomai system): Consider the example I included. It prints average cycle times
>>> and the cycle time variance of the high priority task ("T2"). I noticed a big difference
>>> in the cycle time variance when switching the first task ("T1") to secondary mode with
>>> rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or
>>> SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :)
>>> Can someone give me a short explanation or point me somewhere to get an explanation for
>>> this behaviour? I didn't expect such a difference in variance:
>>>
>> Moving back to secondary mode is nothing more than triggering the
>> rescheduling procedure linux-wise, so the time credit for
>> SCHED_OTHER/IDLE tasks decay as usual during your work loop, preemption
>> by high priority task is more likely and so on. This variance is just
>> the sign that those tasks cannot ask for more than what their scheduling
>> policy grants.
>>
> 
> I think, I didn't express myself clearly enough. I was more puzzled about 
> the variance of the pure RT task (task w/o any secondary mode switches).
> So it seems that changing the scheduling policy of the "relaxed" task has
> an influence on the variance of the pure RT task. So the RT task seems
> to wait for the "relaxed" task. But where exacly does it wait for it? Mmmmm...

When waiting for the mutex?


-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Andreas Glatz]

On Thu, Apr 15, 2010 at 08:40:00AM -0400, Gilles Chanteperdrix wrote:
> Andreas Glatz wrote:
> > Hi,
> > 
> > On Thu, Apr 15, 2010 at 06:46:58AM -0400, Philippe Gerum wrote:
> >> On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:
> >>
> >>> One thing I've noticed though, and this is not related to the patch (I verified it on a
> >>> vanilla Xenomai system): Consider the example I included. It prints average cycle times
> >>> and the cycle time variance of the high priority task ("T2"). I noticed a big difference
> >>> in the cycle time variance when switching the first task ("T1") to secondary mode with
> >>> rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or
> >>> SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :)
> >>> Can someone give me a short explanation or point me somewhere to get an explanation for
> >>> this behaviour? I didn't expect such a difference in variance:
> >>>
> >> Moving back to secondary mode is nothing more than triggering the
> >> rescheduling procedure linux-wise, so the time credit for
> >> SCHED_OTHER/IDLE tasks decay as usual during your work loop, preemption
> >> by high priority task is more likely and so on. This variance is just
> >> the sign that those tasks cannot ask for more than what their scheduling
> >> policy grants.
> >>
> > 
> > I think, I didn't express myself clearly enough. I was more puzzled about 
> > the variance of the pure RT task (task w/o any secondary mode switches).
> > So it seems that changing the scheduling policy of the "relaxed" task has
> > an influence on the variance of the pure RT task. So the RT task seems
> > to wait for the "relaxed" task. But where exacly does it wait for it? Mmmmm...
> 
> When waiting for the mutex?
>

Ok. Maybe it's as ovvious as that. But I'm not quite there yet. Let me explain:
I know that the "relaxed" task ("T1") is in primary mode between rt_mutex_aquire()
and rt_mutex_release(). In other words when the pure RT task ("T2") starts waiting
on the mutex, "T1" should be primary mode and not be affected by the Linux 
scheduling policy. After "T1" releases the mutex it switches back to secondary mode.
So I would expect the variance to be constant. 

Thanks, Andreas

 

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Andreas Glatz wrote:
> On Thu, Apr 15, 2010 at 08:40:00AM -0400, Gilles Chanteperdrix wrote:
>> Andreas Glatz wrote:
>>> Hi,
>>>
>>> On Thu, Apr 15, 2010 at 06:46:58AM -0400, Philippe Gerum wrote:
>>>> On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:
>>>>
>>>>> One thing I've noticed though, and this is not related to the patch (I verified it on a
>>>>> vanilla Xenomai system): Consider the example I included. It prints average cycle times
>>>>> and the cycle time variance of the high priority task ("T2"). I noticed a big difference
>>>>> in the cycle time variance when switching the first task ("T1") to secondary mode with
>>>>> rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or
>>>>> SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :)
>>>>> Can someone give me a short explanation or point me somewhere to get an explanation for
>>>>> this behaviour? I didn't expect such a difference in variance:
>>>>>
>>>> Moving back to secondary mode is nothing more than triggering the
>>>> rescheduling procedure linux-wise, so the time credit for
>>>> SCHED_OTHER/IDLE tasks decay as usual during your work loop, preemption
>>>> by high priority task is more likely and so on. This variance is just
>>>> the sign that those tasks cannot ask for more than what their scheduling
>>>> policy grants.
>>>>
>>> I think, I didn't express myself clearly enough. I was more puzzled about 
>>> the variance of the pure RT task (task w/o any secondary mode switches).
>>> So it seems that changing the scheduling policy of the "relaxed" task has
>>> an influence on the variance of the pure RT task. So the RT task seems
>>> to wait for the "relaxed" task. But where exacly does it wait for it? Mmmmm...
>> When waiting for the mutex?
>>
> 
> Ok. Maybe it's as ovvious as that. But I'm not quite there yet. Let me explain:
> I know that the "relaxed" task ("T1") is in primary mode between rt_mutex_aquire()
> and rt_mutex_release(). In other words when the pure RT task ("T2") starts waiting
> on the mutex, "T1" should be primary mode and not be affected by the Linux 
> scheduling policy. After "T1" releases the mutex it switches back to secondary mode.
> So I would expect the variance to be constant. 

Right. Maybe the problem is that the first call made by a thread to
rt_printf causes a switch to secondary mode (auto mode, means that the
first call to rt_printf calls malloc). You can confirm that by arming
the T_WARNSW bit.


-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Andreas Glatz]

On Thu, Apr 15, 2010 at 09:15:58AM -0400, Gilles Chanteperdrix wrote:
> Andreas Glatz wrote:
> > On Thu, Apr 15, 2010 at 08:40:00AM -0400, Gilles Chanteperdrix wrote:
> >> Andreas Glatz wrote:
> >>> Hi,
> >>>
> >>> On Thu, Apr 15, 2010 at 06:46:58AM -0400, Philippe Gerum wrote:
> >>>> On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:
> >>>>
> >>>>> One thing I've noticed though, and this is not related to the patch (I verified it on a
> >>>>> vanilla Xenomai system): Consider the example I included. It prints average cycle times
> >>>>> and the cycle time variance of the high priority task ("T2"). I noticed a big difference
> >>>>> in the cycle time variance when switching the first task ("T1") to secondary mode with
> >>>>> rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or
> >>>>> SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :)
> >>>>> Can someone give me a short explanation or point me somewhere to get an explanation for
> >>>>> this behaviour? I didn't expect such a difference in variance:
> >>>>>
> >>>> Moving back to secondary mode is nothing more than triggering the
> >>>> rescheduling procedure linux-wise, so the time credit for
> >>>> SCHED_OTHER/IDLE tasks decay as usual during your work loop, preemption
> >>>> by high priority task is more likely and so on. This variance is just
> >>>> the sign that those tasks cannot ask for more than what their scheduling
> >>>> policy grants.
> >>>>
> >>> I think, I didn't express myself clearly enough. I was more puzzled about 
> >>> the variance of the pure RT task (task w/o any secondary mode switches).
> >>> So it seems that changing the scheduling policy of the "relaxed" task has
> >>> an influence on the variance of the pure RT task. So the RT task seems
> >>> to wait for the "relaxed" task. But where exacly does it wait for it? Mmmmm...
> >> When waiting for the mutex?
> >>
> > 
> > Ok. Maybe it's as ovvious as that. But I'm not quite there yet. Let me explain:
> > I know that the "relaxed" task ("T1") is in primary mode between rt_mutex_aquire()
> > and rt_mutex_release(). In other words when the pure RT task ("T2") starts waiting
> > on the mutex, "T1" should be primary mode and not be affected by the Linux 
> > scheduling policy. After "T1" releases the mutex it switches back to secondary mode.
> > So I would expect the variance to be constant. 
> 
> Right. Maybe the problem is that the first call made by a thread to
> rt_printf causes a switch to secondary mode (auto mode, means that the
> first call to rt_printf calls malloc). You can confirm that by arming
> the T_WARNSW bit.
>

Ok. You where right, too. The reason for the high variance (compared to the case
where both tasks always run in primary mode) comes from the fact that the pure 
RT task ("T2") starts blocking on rt_mutex_aquire() which in term boosts the 
priority of the "relaxed" task ("T1") running in primary mode (since it is between
rt_mutex_aquire/rt_mutex_release). 

At a first glance this made no sense to me since I thought that there shouldn't 
be any difference between the case where both tasks always run in primary mode 
and the case where "T1" switches to secondary mode only outside of 
rt_mutex_aquire/rt_mutex_release.

Now I found the reason: If I pass the flag T_PRIOFF to "T1" (the task which 
occasionally switches to secondary mode) I get the same variances for both cases
which I described in the previous paragraph. So the reason is clear: T2 not only
boosts the RT priority but also the Linux priority. Apparently, boosting the
Linux priority takes much longer and depends on the scheduling policy.

Is this a desired behaviour?

Andreas

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Andreas Glatz wrote:
> On Thu, Apr 15, 2010 at 09:15:58AM -0400, Gilles Chanteperdrix wrote:
>> Andreas Glatz wrote:
>>> On Thu, Apr 15, 2010 at 08:40:00AM -0400, Gilles Chanteperdrix wrote:
>>>> Andreas Glatz wrote:
>>>>> Hi,
>>>>>
>>>>> On Thu, Apr 15, 2010 at 06:46:58AM -0400, Philippe Gerum wrote:
>>>>>> On Wed, 2010-04-14 at 20:19 -0400, Andreas Glatz wrote:
>>>>>>
>>>>>>> One thing I've noticed though, and this is not related to the patch (I verified it on a
>>>>>>> vanilla Xenomai system): Consider the example I included. It prints average cycle times
>>>>>>> and the cycle time variance of the high priority task ("T2"). I noticed a big difference
>>>>>>> in the cycle time variance when switching the first task ("T1") to secondary mode with
>>>>>>> rt_task_set_mode() and setting the scheduler policy to either SCHED_FIFO, SCHED_IDLE or
>>>>>>> SCHED_NORMAL. I'm assuming someone asked this before and I didn't pay attention :)
>>>>>>> Can someone give me a short explanation or point me somewhere to get an explanation for
>>>>>>> this behaviour? I didn't expect such a difference in variance:
>>>>>>>
>>>>>> Moving back to secondary mode is nothing more than triggering the
>>>>>> rescheduling procedure linux-wise, so the time credit for
>>>>>> SCHED_OTHER/IDLE tasks decay as usual during your work loop, preemption
>>>>>> by high priority task is more likely and so on. This variance is just
>>>>>> the sign that those tasks cannot ask for more than what their scheduling
>>>>>> policy grants.
>>>>>>
>>>>> I think, I didn't express myself clearly enough. I was more puzzled about 
>>>>> the variance of the pure RT task (task w/o any secondary mode switches).
>>>>> So it seems that changing the scheduling policy of the "relaxed" task has
>>>>> an influence on the variance of the pure RT task. So the RT task seems
>>>>> to wait for the "relaxed" task. But where exacly does it wait for it? Mmmmm...
>>>> When waiting for the mutex?
>>>>
>>> Ok. Maybe it's as ovvious as that. But I'm not quite there yet. Let me explain:
>>> I know that the "relaxed" task ("T1") is in primary mode between rt_mutex_aquire()
>>> and rt_mutex_release(). In other words when the pure RT task ("T2") starts waiting
>>> on the mutex, "T1" should be primary mode and not be affected by the Linux 
>>> scheduling policy. After "T1" releases the mutex it switches back to secondary mode.
>>> So I would expect the variance to be constant. 
>> Right. Maybe the problem is that the first call made by a thread to
>> rt_printf causes a switch to secondary mode (auto mode, means that the
>> first call to rt_printf calls malloc). You can confirm that by arming
>> the T_WARNSW bit.
>>
> 
> Ok. You where right, too. The reason for the high variance (compared to the case
> where both tasks always run in primary mode) comes from the fact that the pure 
> RT task ("T2") starts blocking on rt_mutex_aquire() which in term boosts the 
> priority of the "relaxed" task ("T1") running in primary mode (since it is between
> rt_mutex_aquire/rt_mutex_release). 
> 
> At a first glance this made no sense to me since I thought that there shouldn't 
> be any difference between the case where both tasks always run in primary mode 
> and the case where "T1" switches to secondary mode only outside of 
> rt_mutex_aquire/rt_mutex_release.
> 
> Now I found the reason: If I pass the flag T_PRIOFF to "T1" (the task which 
> occasionally switches to secondary mode) I get the same variances for both cases
> which I described in the previous paragraph. So the reason is clear: T2 not only
> boosts the RT priority but also the Linux priority. Apparently, boosting the
> Linux priority takes much longer and depends on the scheduling policy.
> 
> Is this a desired behaviour?

The thing is that when you hold a mutex shared with a primary mode
thread, you should not switch to secondary mode. Xenomai 2.5 is able to
detect this behaviour and send you a signal in that case, if the primary
mode thread has the T_WARNSW bit armed. I have tried to explain this here:

http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai#Chasing_the_unwanted_mode_switches.


-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 933 bytes --]

Gilles Chanteperdrix wrote:
> Henri Roosen wrote:
>> Ok, I might give Jan's proposal a try and could be sufficient for us:a
>> Xenomai thread with non-RT scheduling class (didn't know that was
>> possible).
>>
>> We just need the thread in the Xenomai domain for being able to access
>> Xenomai resources, but want it to be scheduled together with the Linux
>> nice levels. A near IDLE nice level should be ok. We just don't want
>> that our Xenomai 'idle' thread with a while(1) would prevent Linux to
>> run.
> 
> That will happen with SCHED_OTHER if while(1) occurs while running in
> primary mode.
> 

Indeed. But that's in fact a deficit of our mapping of non-RT sched
policies on the Xenomai scheduler. We should not leave, e.g., a
SCHED_OTHER thread migrated in primary as long as it does not hold any
resources. We can't perform SCHED_OTHER scheduling in Xenomai, we have
to push this to Linux.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Henri Roosen wrote:
>>> Ok, I might give Jan's proposal a try and could be sufficient for us:a
>>> Xenomai thread with non-RT scheduling class (didn't know that was
>>> possible).
>>>
>>> We just need the thread in the Xenomai domain for being able to access
>>> Xenomai resources, but want it to be scheduled together with the Linux
>>> nice levels. A near IDLE nice level should be ok. We just don't want
>>> that our Xenomai 'idle' thread with a while(1) would prevent Linux to
>>> run.
>> That will happen with SCHED_OTHER if while(1) occurs while running in
>> primary mode.
>>
> 
> Indeed. But that's in fact a deficit of our mapping of non-RT sched
> policies on the Xenomai scheduler. We should not leave, e.g., a
> SCHED_OTHER thread migrated in primary as long as it does not hold any
> resources. We can't perform SCHED_OTHER scheduling in Xenomai, we have
> to push this to Linux.

You have the scheduling classes, you can in fact now implement
SCHED_OTHER the way you like. This SCHED_OTHER as a priority 0
SCHED_FIFO dates back from the time when we had not the scheduling classes.

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1311 bytes --]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Andreas Glatz wrote:
>>> Hi Philippe,
>>>
>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>
>> What prevents using a borderline thread (if you need to interact with
>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
> 
> Well, this does not really guarantee that the thread will run only when
> linux is idle. The thread will eat some cpu time, the nice level is not
> a strict priority, as you know.

Where do you really need anything stricter? It's the opposite of "I need
true 100% CPU for my task, and that forever."

> But in fact, I wonder why Andreas wants
> a new scheduling policy for xenomai, what is needed, is simply a
> SCHED_IDLE (maybe it exists ?!) policy for Linux.
> 

There is no such thing AFAIK. If you are concerned that some CPU
intensive low prio job eats too much CPU, you normally reduce its
nice-level and/or confine its CPU bandwidth via cgroups.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Andreas Glatz wrote:
>>>> Hi Philippe,
>>>>
>>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>>
>>> What prevents using a borderline thread (if you need to interact with
>>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>> Well, this does not really guarantee that the thread will run only when
>> linux is idle. The thread will eat some cpu time, the nice level is not
>> a strict priority, as you know.
> 
> Where do you really need anything stricter? It's the opposite of "I need
> true 100% CPU for my task, and that forever."
> 
>> But in fact, I wonder why Andreas wants
>> a new scheduling policy for xenomai, what is needed, is simply a
>> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>>
> 
> There is no such thing AFAIK. If you are concerned that some CPU
> intensive low prio job eats too much CPU, you normally reduce its
> nice-level and/or confine its CPU bandwidth via cgroups.

SCHED_IDLE exists.

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Jan Kiszka wrote:
>>>> Andreas Glatz wrote:
>>>>> Hi Philippe,
>>>>>
>>>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>>>
>>>> What prevents using a borderline thread (if you need to interact with
>>>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>>> Well, this does not really guarantee that the thread will run only when
>>> linux is idle. The thread will eat some cpu time, the nice level is not
>>> a strict priority, as you know.
>> Where do you really need anything stricter? It's the opposite of "I need
>> true 100% CPU for my task, and that forever."
>>
>>> But in fact, I wonder why Andreas wants
>>> a new scheduling policy for xenomai, what is needed, is simply a
>>> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>>>
>> There is no such thing AFAIK. If you are concerned that some CPU
>> intensive low prio job eats too much CPU, you normally reduce its
>> nice-level and/or confine its CPU bandwidth via cgroups.
> 
> SCHED_IDLE exists.
> 

Ah, as "nice 20". Same mechanism, just another level.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1763 bytes --]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Jan Kiszka wrote:
>>>>> Andreas Glatz wrote:
>>>>>> Hi Philippe,
>>>>>>
>>>>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>>>>
>>>>> What prevents using a borderline thread (if you need to interact with
>>>>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>>>> Well, this does not really guarantee that the thread will run only when
>>>> linux is idle. The thread will eat some cpu time, the nice level is not
>>>> a strict priority, as you know.
>>> Where do you really need anything stricter? It's the opposite of "I need
>>> true 100% CPU for my task, and that forever."
>>>
>>>> But in fact, I wonder why Andreas wants
>>>> a new scheduling policy for xenomai, what is needed, is simply a
>>>> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>>>>
>>> There is no such thing AFAIK. If you are concerned that some CPU
>>> intensive low prio job eats too much CPU, you normally reduce its
>>> nice-level and/or confine its CPU bandwidth via cgroups.
>> SCHED_IDLE exists.
>>
> 
> Ah, as "nice 20". Same mechanism, just another level.
> 

...and before trying something else:

"SCHED_IDLE: This is even weaker than nice 19, but its not a true
idle timer scheduler in order to avoid to get into priority
inversion problems which would deadlock the machine."

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Jan Kiszka wrote:
>>>> Gilles Chanteperdrix wrote:
>>>>> Jan Kiszka wrote:
>>>>>> Andreas Glatz wrote:
>>>>>>> Hi Philippe,
>>>>>>>
>>>>>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>>>>>
>>>>>> What prevents using a borderline thread (if you need to interact with
>>>>>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>>>>> Well, this does not really guarantee that the thread will run only when
>>>>> linux is idle. The thread will eat some cpu time, the nice level is not
>>>>> a strict priority, as you know.
>>>> Where do you really need anything stricter? It's the opposite of "I need
>>>> true 100% CPU for my task, and that forever."
>>>>
>>>>> But in fact, I wonder why Andreas wants
>>>>> a new scheduling policy for xenomai, what is needed, is simply a
>>>>> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>>>>>
>>>> There is no such thing AFAIK. If you are concerned that some CPU
>>>> intensive low prio job eats too much CPU, you normally reduce its
>>>> nice-level and/or confine its CPU bandwidth via cgroups.
>>> SCHED_IDLE exists.
>>>
>> Ah, as "nice 20". Same mechanism, just another level.
>>
> 
> ...and before trying something else:
> 
> "SCHED_IDLE: This is even weaker than nice 19, but its not a true
> idle timer scheduler in order to avoid to get into priority
> inversion problems which would deadlock the machine."

Mmmm, priority inversion? I thought the kernel had priority inheritance?

-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2193 bytes --]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Jan Kiszka wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Jan Kiszka wrote:
>>>>> Gilles Chanteperdrix wrote:
>>>>>> Jan Kiszka wrote:
>>>>>>> Andreas Glatz wrote:
>>>>>>>> Hi Philippe,
>>>>>>>>
>>>>>>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>>>>>>
>>>>>>> What prevents using a borderline thread (if you need to interact with
>>>>>>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>>>>>> Well, this does not really guarantee that the thread will run only when
>>>>>> linux is idle. The thread will eat some cpu time, the nice level is not
>>>>>> a strict priority, as you know.
>>>>> Where do you really need anything stricter? It's the opposite of "I need
>>>>> true 100% CPU for my task, and that forever."
>>>>>
>>>>>> But in fact, I wonder why Andreas wants
>>>>>> a new scheduling policy for xenomai, what is needed, is simply a
>>>>>> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>>>>>>
>>>>> There is no such thing AFAIK. If you are concerned that some CPU
>>>>> intensive low prio job eats too much CPU, you normally reduce its
>>>>> nice-level and/or confine its CPU bandwidth via cgroups.
>>>> SCHED_IDLE exists.
>>>>
>>> Ah, as "nice 20". Same mechanism, just another level.
>>>
>> ...and before trying something else:
>>
>> "SCHED_IDLE: This is even weaker than nice 19, but its not a true
>> idle timer scheduler in order to avoid to get into priority
>> inversion problems which would deadlock the machine."
> 
> Mmmm, priority inversion? I thought the kernel had priority inheritance?
> 

I do not recall the details, but I don't think non-RT priorities are
inherited. Moreover, user space locks without explicit inheritance set
(ie. the majority of locks) would still cause deadlocks, though not for
the kernel.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Jan Kiszka wrote:
>>>> Gilles Chanteperdrix wrote:
>>>>> Jan Kiszka wrote:
>>>>>> Gilles Chanteperdrix wrote:
>>>>>>> Jan Kiszka wrote:
>>>>>>>> Andreas Glatz wrote:
>>>>>>>>> Hi Philippe,
>>>>>>>>>
>>>>>>>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>>>>>>>
>>>>>>>> What prevents using a borderline thread (if you need to interact with
>>>>>>>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>>>>>>> Well, this does not really guarantee that the thread will run only when
>>>>>>> linux is idle. The thread will eat some cpu time, the nice level is not
>>>>>>> a strict priority, as you know.
>>>>>> Where do you really need anything stricter? It's the opposite of "I need
>>>>>> true 100% CPU for my task, and that forever."
>>>>>>
>>>>>>> But in fact, I wonder why Andreas wants
>>>>>>> a new scheduling policy for xenomai, what is needed, is simply a
>>>>>>> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>>>>>>>
>>>>>> There is no such thing AFAIK. If you are concerned that some CPU
>>>>>> intensive low prio job eats too much CPU, you normally reduce its
>>>>>> nice-level and/or confine its CPU bandwidth via cgroups.
>>>>> SCHED_IDLE exists.
>>>>>
>>>> Ah, as "nice 20". Same mechanism, just another level.
>>>>
>>> ...and before trying something else:
>>>
>>> "SCHED_IDLE: This is even weaker than nice 19, but its not a true
>>> idle timer scheduler in order to avoid to get into priority
>>> inversion problems which would deadlock the machine."
>> Mmmm, priority inversion? I thought the kernel had priority inheritance?
>>
> 
> I do not recall the details, but I don't think non-RT priorities are
> inherited. Moreover, user space locks without explicit inheritance set
> (ie. the majority of locks) would still cause deadlocks, though not for
> the kernel.

Well, of course. If a non real-time task shares a lock with a real-time
task, it has to have priority inheritance anyway. However
SCHED_IDLE/SCHED_OTHER/whatever is implemented.


-- 
					    Gilles.

Re: [Xenomai-help] New scheduler class

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2623 bytes --]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Jan Kiszka wrote:
>>>> Jan Kiszka wrote:
>>>>> Gilles Chanteperdrix wrote:
>>>>>> Jan Kiszka wrote:
>>>>>>> Gilles Chanteperdrix wrote:
>>>>>>>> Jan Kiszka wrote:
>>>>>>>>> Andreas Glatz wrote:
>>>>>>>>>> Hi Philippe,
>>>>>>>>>>
>>>>>>>>>> At the Xenomai Users Meeting last year I asked you if Xenomai would offer a possibility to lower the priority of certain Xenomai tasks below that of Linux. We need this feature since we have tasks in our RT application which should only run when Linux is idle (A statistics collection task which part of the RT application and hard to isolate from this application).
>>>>>>>>>>
>>>>>>>>> What prevents using a borderline thread (if you need to interact with
>>>>>>>>> blocking Xenomai services) with SCHED_OTHER and a Linux nice level of 19?
>>>>>>>> Well, this does not really guarantee that the thread will run only when
>>>>>>>> linux is idle. The thread will eat some cpu time, the nice level is not
>>>>>>>> a strict priority, as you know.
>>>>>>> Where do you really need anything stricter? It's the opposite of "I need
>>>>>>> true 100% CPU for my task, and that forever."
>>>>>>>
>>>>>>>> But in fact, I wonder why Andreas wants
>>>>>>>> a new scheduling policy for xenomai, what is needed, is simply a
>>>>>>>> SCHED_IDLE (maybe it exists ?!) policy for Linux.
>>>>>>>>
>>>>>>> There is no such thing AFAIK. If you are concerned that some CPU
>>>>>>> intensive low prio job eats too much CPU, you normally reduce its
>>>>>>> nice-level and/or confine its CPU bandwidth via cgroups.
>>>>>> SCHED_IDLE exists.
>>>>>>
>>>>> Ah, as "nice 20". Same mechanism, just another level.
>>>>>
>>>> ...and before trying something else:
>>>>
>>>> "SCHED_IDLE: This is even weaker than nice 19, but its not a true
>>>> idle timer scheduler in order to avoid to get into priority
>>>> inversion problems which would deadlock the machine."
>>> Mmmm, priority inversion? I thought the kernel had priority inheritance?
>>>
>> I do not recall the details, but I don't think non-RT priorities are
>> inherited. Moreover, user space locks without explicit inheritance set
>> (ie. the majority of locks) would still cause deadlocks, though not for
>> the kernel.
> 
> Well, of course. If a non real-time task shares a lock with a real-time
> task, it has to have priority inheritance anyway. However
> SCHED_IDLE/SCHED_OTHER/whatever is implemented.

Yes, but the deadlock concerns are mostly about non-RT tasks sharing
locks (e.g. in-kernel mutexes).

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] New scheduler class

[Andreas Glatz]

>
> The auto-relax feature is now available from -head (upcoming  
> 2.6.x). You
> will need all commits from
> http://git.xenomai.org/?p=xenomai- 
> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
> to
> http://git.xenomai.org/?p=xenomai- 
> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
>
> -- 
> Philippe.
>
>

Very nice. Will try it out soon on my G4 PB Aluminium.

Andreas

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
Tests with a basic application look good, however our bigger project
doesn't. I checked already if we release all mutexes; the base prio 0
thread thread released all of them but stays in primary domain after a
xenomai call.

I'll setup a clean environment, just to be sure, and will investigate
further tomorrow.

On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
>
>>
>> The auto-relax feature is now available from -head (upcoming
>> 2.6.x). You
>> will need all commits from
>> http://git.xenomai.org/?p=xenomai-
>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
>> to
>> http://git.xenomai.org/?p=xenomai-
>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
>>
>> --
>> Philippe.
>>
>>
>
> Very nice. Will try it out soon on my G4 PB Aluminium.
>
> Andreas
>
>
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
>

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

That was a problem with our build environment.

Patches work and the base-prio 0 task nicely switches to the Linux
domain and competes with the Linux threads for the processor. Now we
don't loose TCP connection to our target anymore.

Thanks!

Henri

On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
> Tests with a basic application look good, however our bigger project
> doesn't. I checked already if we release all mutexes; the base prio 0
> thread thread released all of them but stays in primary domain after a
> xenomai call.
>
> I'll setup a clean environment, just to be sure, and will investigate
> further tomorrow.
>
> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
>>
>>>
>>> The auto-relax feature is now available from -head (upcoming
>>> 2.6.x). You
>>> will need all commits from
>>> http://git.xenomai.org/?p=xenomai-
>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
>>> to
>>> http://git.xenomai.org/?p=xenomai-
>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
>>>
>>> --
>>> Philippe.
>>>
>>>
>>
>> Very nice. Will try it out soon on my G4 PB Aluminium.
>>
>> Andreas
>>
>>
>> _______________________________________________
>> Xenomai-help mailing list
>> Xenomai-help@domain.hid
>> https://mail.gna.org/listinfo/xenomai-help
>>
>

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

Hi,

We are using the back-ported auto-relax patches of Xenomai-head on the
2.5.6 release on Linux 2.6.32.15.

We noticed that our prio-0 shadowed task is not always auto-relaxed
and traced it back to a problem with rt_task_send/receive/reply. The
rt_task_send/receive/reply mechanism uses the xnsynch primitive to
implement its PIP. However, this mechanism only acquires the xysynch
primitive without releasing it. This results in the resource counter
never be 0 anymore and thus breaking the auto-relaxed which is based
on this resource counter.

I guess releasing the xnsynch resets the owner, which is not wanted either..
Any idea for a fix?

Thanks,
Henri.


On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
> That was a problem with our build environment.
>
> Patches work and the base-prio 0 task nicely switches to the Linux
> domain and competes with the Linux threads for the processor. Now we
> don't loose TCP connection to our target anymore.
>
> Thanks!
>
> Henri
>
> On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
>> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
>> Tests with a basic application look good, however our bigger project
>> doesn't. I checked already if we release all mutexes; the base prio 0
>> thread thread released all of them but stays in primary domain after a
>> xenomai call.
>>
>> I'll setup a clean environment, just to be sure, and will investigate
>> further tomorrow.
>>
>> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
>>>
>>>>
>>>> The auto-relax feature is now available from -head (upcoming
>>>> 2.6.x). You
>>>> will need all commits from
>>>> http://git.xenomai.org/?p=xenomai-
>>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
>>>> to
>>>> http://git.xenomai.org/?p=xenomai-
>>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
>>>>
>>>> --
>>>> Philippe.
>>>>
>>>>
>>>
>>> Very nice. Will try it out soon on my G4 PB Aluminium.
>>>
>>> Andreas
>>>
>>>
>>> _______________________________________________
>>> Xenomai-help mailing list
>>> Xenomai-help@domain.hid
>>> https://mail.gna.org/listinfo/xenomai-help
>>>
>>
>

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Mon, 2011-03-28 at 12:56 +0200, Henri Roosen wrote:
> Hi,
> 
> We are using the back-ported auto-relax patches of Xenomai-head on the
> 2.5.6 release on Linux 2.6.32.15.
> 
> We noticed that our prio-0 shadowed task is not always auto-relaxed
> and traced it back to a problem with rt_task_send/receive/reply. The
> rt_task_send/receive/reply mechanism uses the xnsynch primitive to
> implement its PIP. However, this mechanism only acquires the xysynch
> primitive without releasing it. This results in the resource counter
> never be 0 anymore and thus breaking the auto-relaxed which is based
> on this resource counter.
> 
> I guess releasing the xnsynch resets the owner, which is not wanted either..
> Any idea for a fix?
> 

The main issue is that this interface is totally at odds with the
fastsync support. Will have a look.

> Thanks,
> Henri.
> 
> 
> On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
> > That was a problem with our build environment.
> >
> > Patches work and the base-prio 0 task nicely switches to the Linux
> > domain and competes with the Linux threads for the processor. Now we
> > don't loose TCP connection to our target anymore.
> >
> > Thanks!
> >
> > Henri
> >
> > On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
> >> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
> >> Tests with a basic application look good, however our bigger project
> >> doesn't. I checked already if we release all mutexes; the base prio 0
> >> thread thread released all of them but stays in primary domain after a
> >> xenomai call.
> >>
> >> I'll setup a clean environment, just to be sure, and will investigate
> >> further tomorrow.
> >>
> >> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
> >>>
> >>>>
> >>>> The auto-relax feature is now available from -head (upcoming
> >>>> 2.6.x). You
> >>>> will need all commits from
> >>>> http://git.xenomai.org/?p=xenomai-
> >>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
> >>>> to
> >>>> http://git.xenomai.org/?p=xenomai-
> >>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
> >>>>
> >>>> --
> >>>> Philippe.
> >>>>
> >>>>
> >>>
> >>> Very nice. Will try it out soon on my G4 PB Aluminium.
> >>>
> >>> Andreas
> >>>
> >>>
> >>> _______________________________________________
> >>> Xenomai-help mailing list
> >>> Xenomai-help@domain.hid
> >>> https://mail.gna.org/listinfo/xenomai-help
> >>>
> >>
> >

-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Mon, 2011-03-28 at 12:56 +0200, Henri Roosen wrote:
> Hi,
> 
> We are using the back-ported auto-relax patches of Xenomai-head on the
> 2.5.6 release on Linux 2.6.32.15.
> 
> We noticed that our prio-0 shadowed task is not always auto-relaxed
> and traced it back to a problem with rt_task_send/receive/reply. The
> rt_task_send/receive/reply mechanism uses the xnsynch primitive to
> implement its PIP. However, this mechanism only acquires the xysynch
> primitive without releasing it. This results in the resource counter
> never be 0 anymore and thus breaking the auto-relaxed which is based
> on this resource counter.
> 
> I guess releasing the xnsynch resets the owner, which is not wanted either..
> Any idea for a fix?

Could you give a try at this one? TIA,
http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44

> 
> Thanks,
> Henri.
> 
> 
> On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
> > That was a problem with our build environment.
> >
> > Patches work and the base-prio 0 task nicely switches to the Linux
> > domain and competes with the Linux threads for the processor. Now we
> > don't loose TCP connection to our target anymore.
> >
> > Thanks!
> >
> > Henri
> >
> > On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
> >> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
> >> Tests with a basic application look good, however our bigger project
> >> doesn't. I checked already if we release all mutexes; the base prio 0
> >> thread thread released all of them but stays in primary domain after a
> >> xenomai call.
> >>
> >> I'll setup a clean environment, just to be sure, and will investigate
> >> further tomorrow.
> >>
> >> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
> >>>
> >>>>
> >>>> The auto-relax feature is now available from -head (upcoming
> >>>> 2.6.x). You
> >>>> will need all commits from
> >>>> http://git.xenomai.org/?p=xenomai-
> >>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
> >>>> to
> >>>> http://git.xenomai.org/?p=xenomai-
> >>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
> >>>>
> >>>> --
> >>>> Philippe.
> >>>>
> >>>>
> >>>
> >>> Very nice. Will try it out soon on my G4 PB Aluminium.
> >>>
> >>> Andreas
> >>>
> >>>
> >>> _______________________________________________
> >>> Xenomai-help mailing list
> >>> Xenomai-help@domain.hid
> >>> https://mail.gna.org/listinfo/xenomai-help
> >>>
> >>
> >

-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

[-- Attachment #1: Type: text/plain, Size: 3114 bytes --]

Hi Philippe,

Thanks for the patch! Unfortunately this does not resolve the 'stuck
in primary of prio 0 after rt_task_send'.

I'll attach the basic test application that reproduces it. All prints
should contain secondary mode because all tasks are prio 0 and the
auto-relax patches are compiled in.

The producer which does the rt_task_send() call does not auto-relax to
secondary mode.

Thanks,
Henri.

On Sun, May 1, 2011 at 5:01 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> On Mon, 2011-03-28 at 12:56 +0200, Henri Roosen wrote:
>> Hi,
>>
>> We are using the back-ported auto-relax patches of Xenomai-head on the
>> 2.5.6 release on Linux 2.6.32.15.
>>
>> We noticed that our prio-0 shadowed task is not always auto-relaxed
>> and traced it back to a problem with rt_task_send/receive/reply. The
>> rt_task_send/receive/reply mechanism uses the xnsynch primitive to
>> implement its PIP. However, this mechanism only acquires the xysynch
>> primitive without releasing it. This results in the resource counter
>> never be 0 anymore and thus breaking the auto-relaxed which is based
>> on this resource counter.
>>
>> I guess releasing the xnsynch resets the owner, which is not wanted either..
>> Any idea for a fix?
>
> Could you give a try at this one? TIA,
> http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44
>
>>
>> Thanks,
>> Henri.
>>
>>
>> On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
>> > That was a problem with our build environment.
>> >
>> > Patches work and the base-prio 0 task nicely switches to the Linux
>> > domain and competes with the Linux threads for the processor. Now we
>> > don't loose TCP connection to our target anymore.
>> >
>> > Thanks!
>> >
>> > Henri
>> >
>> > On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
>> >> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
>> >> Tests with a basic application look good, however our bigger project
>> >> doesn't. I checked already if we release all mutexes; the base prio 0
>> >> thread thread released all of them but stays in primary domain after a
>> >> xenomai call.
>> >>
>> >> I'll setup a clean environment, just to be sure, and will investigate
>> >> further tomorrow.
>> >>
>> >> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
>> >>>
>> >>>>
>> >>>> The auto-relax feature is now available from -head (upcoming
>> >>>> 2.6.x). You
>> >>>> will need all commits from
>> >>>> http://git.xenomai.org/?p=xenomai-
>> >>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
>> >>>> to
>> >>>> http://git.xenomai.org/?p=xenomai-
>> >>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
>> >>>>
>> >>>> --
>> >>>> Philippe.
>> >>>>
>> >>>>
>> >>>
>> >>> Very nice. Will try it out soon on my G4 PB Aluminium.
>> >>>
>> >>> Andreas
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Xenomai-help mailing list
>> >>> Xenomai-help@domain.hid
>> >>> https://mail.gna.org/listinfo/xenomai-help
>> >>>
>> >>
>> >
>
> --
> Philippe.
>
>
>

[-- Attachment #2: test_autorelax_tasksend.c --]
[-- Type: text/x-csrc, Size: 2364 bytes --]

#include <sys/mman.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <native/task.h>
#include <native/queue.h>
#include <asm-generic/xenomai/bits/current.h>

#define CONSUMER_TASK_PRI    0
#define CONSUMER_STACK_SIZE  8192

#define PRODUCER_TASK_PRI    0
#define PRODUCER_STACK_SIZE  8192

#define CONSUMER_WAIT 150
#define PRODUCER_TRIG 40

#define MAX_STRING_LEN 40

static const char *satch_s_tunes[] = {
    "Surfing With The Alien",
    "Lords of Karma",
    "Banana Mango",
    "Psycho Monkey",
    "Luminous Flesh Giants",
    "Moroccan Sunset",
    "Satch Boogie",
    "Flying In A Blue Dream",
    "Ride",
    "Summer Song",
    "Speed Of Light",
    "Crystal Planet",
    "Raspberry Jam Delta-V",
    "Champagne?",
    "Clouds Race Across The Sky",
    "Engines Of Creation"
};

static RT_TASK producer_task,
	       consumer_task;

void consumer(void *cookie)
{
    char buf[MAX_STRING_LEN];
    RT_TASK_MCB mcb;
    int flowid;
    int mode;

    for (;;)
	{
	rt_task_sleep(CONSUMER_WAIT);

	for (;;)
	    {
	    mcb.opcode = 0;	/* Dummy. */
	    mcb.data = (caddr_t)buf;
	    mcb.size = sizeof(buf);
	    flowid = rt_task_receive(&mcb,TM_NONBLOCK);
            mode = xeno_get_current_mode();
	    if (flowid < 0)
		break;

	    printf("Now playing %s... in %s mode.\n",buf, (mode & XNRELAX) ? "secondary" : "primary");
	    rt_task_reply(flowid,NULL);
	    }
	}
}

void producer(void *cookie)
{
    int next_msg = 0;
    RT_TASK_MCB mcb;
    const char *msg;
    int mode;

    for (;;)
	{
	rt_task_sleep(PRODUCER_TRIG);

	msg = satch_s_tunes[next_msg++];
	next_msg %= (sizeof(satch_s_tunes) / sizeof(satch_s_tunes[0]));

	mcb.opcode = 0;	/* Dummy. */
	mcb.data = (caddr_t)msg;
	mcb.size = strlen(msg) + 1; /* \0 must be copied. */
	rt_task_send(&consumer_task,&mcb,NULL,TM_INFINITE);
	mode = xeno_get_current_mode();
        printf("Producer mode %s\n", (mode & XNRELAX) ? "secondary" : "primary");
	}
}

int main(void)
{
    mlockall(MCL_CURRENT | MCL_FUTURE);
    rt_timer_set_mode(1000000);	/* Forc 1ms periodic tick. */

    rt_task_spawn(&consumer_task,
		  "ConsumerTask",
		  CONSUMER_STACK_SIZE,
		  CONSUMER_TASK_PRI,
		  0,
		  &consumer,
		  NULL);

    rt_task_spawn(&producer_task,
		  "ProducerTask",
		  PRODUCER_STACK_SIZE,
		  PRODUCER_TASK_PRI,
		  0,
		  &producer,
		  NULL);
    while (1);
    return 0;
}

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Mon, 2011-05-02 at 10:58 +0200, Henri Roosen wrote:
> Hi Philippe,
> 
> Thanks for the patch! Unfortunately this does not resolve the 'stuck
> in primary of prio 0 after rt_task_send'.
> 
> I'll attach the basic test application that reproduces it. All prints
> should contain secondary mode because all tasks are prio 0 and the
> auto-relax patches are compiled in.
> 
> The producer which does the rt_task_send() call does not auto-relax to
> secondary mode.

Your example code works with this added patch on top of both 2.5.x and
-head:
http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=82200f0b66908a5366a72355b3b70472031d68fb

> 
> Thanks,
> Henri.
> 
> On Sun, May 1, 2011 at 5:01 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> > On Mon, 2011-03-28 at 12:56 +0200, Henri Roosen wrote:
> >> Hi,
> >>
> >> We are using the back-ported auto-relax patches of Xenomai-head on the
> >> 2.5.6 release on Linux 2.6.32.15.
> >>
> >> We noticed that our prio-0 shadowed task is not always auto-relaxed
> >> and traced it back to a problem with rt_task_send/receive/reply. The
> >> rt_task_send/receive/reply mechanism uses the xnsynch primitive to
> >> implement its PIP. However, this mechanism only acquires the xysynch
> >> primitive without releasing it. This results in the resource counter
> >> never be 0 anymore and thus breaking the auto-relaxed which is based
> >> on this resource counter.
> >>
> >> I guess releasing the xnsynch resets the owner, which is not wanted either..
> >> Any idea for a fix?
> >
> > Could you give a try at this one? TIA,
> > http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44
> >
> >>
> >> Thanks,
> >> Henri.
> >>
> >>
> >> On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
> >> > That was a problem with our build environment.
> >> >
> >> > Patches work and the base-prio 0 task nicely switches to the Linux
> >> > domain and competes with the Linux threads for the processor. Now we
> >> > don't loose TCP connection to our target anymore.
> >> >
> >> > Thanks!
> >> >
> >> > Henri
> >> >
> >> > On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
> >> >> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
> >> >> Tests with a basic application look good, however our bigger project
> >> >> doesn't. I checked already if we release all mutexes; the base prio 0
> >> >> thread thread released all of them but stays in primary domain after a
> >> >> xenomai call.
> >> >>
> >> >> I'll setup a clean environment, just to be sure, and will investigate
> >> >> further tomorrow.
> >> >>
> >> >> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
> >> >>>
> >> >>>>
> >> >>>> The auto-relax feature is now available from -head (upcoming
> >> >>>> 2.6.x). You
> >> >>>> will need all commits from
> >> >>>> http://git.xenomai.org/?p=xenomai-
> >> >>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
> >> >>>> to
> >> >>>> http://git.xenomai.org/?p=xenomai-
> >> >>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
> >> >>>>
> >> >>>> --
> >> >>>> Philippe.
> >> >>>>
> >> >>>>
> >> >>>
> >> >>> Very nice. Will try it out soon on my G4 PB Aluminium.
> >> >>>
> >> >>> Andreas
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> Xenomai-help mailing list
> >> >>> Xenomai-help@domain.hid
> >> >>> https://mail.gna.org/listinfo/xenomai-help
> >> >>>
> >> >>
> >> >
> >
> > --
> > Philippe.
> >
> >
> >

-- 
Philippe.

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

On Thu, May 5, 2011 at 3:55 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> On Mon, 2011-05-02 at 10:58 +0200, Henri Roosen wrote:
>> Hi Philippe,
>>
>> Thanks for the patch! Unfortunately this does not resolve the 'stuck
>> in primary of prio 0 after rt_task_send'.
>>
>> I'll attach the basic test application that reproduces it. All prints
>> should contain secondary mode because all tasks are prio 0 and the
>> auto-relax patches are compiled in.
>>
>> The producer which does the rt_task_send() call does not auto-relax to
>> secondary mode.
>
> Your example code works with this added patch on top of both 2.5.x and
> -head:
> http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=82200f0b66908a5366a72355b3b70472031d68fb
>

Hi Philippe!
I tried this patch and can confirm it results in expected behaviour of
the prio 0 scheduling now, also in our main application.

Thanks a lot!
Henri

>>
>> Thanks,
>> Henri.
>>
>> On Sun, May 1, 2011 at 5:01 PM, Philippe Gerum <rpm@xenomai.org> wrote:
>> > On Mon, 2011-03-28 at 12:56 +0200, Henri Roosen wrote:
>> >> Hi,
>> >>
>> >> We are using the back-ported auto-relax patches of Xenomai-head on the
>> >> 2.5.6 release on Linux 2.6.32.15.
>> >>
>> >> We noticed that our prio-0 shadowed task is not always auto-relaxed
>> >> and traced it back to a problem with rt_task_send/receive/reply. The
>> >> rt_task_send/receive/reply mechanism uses the xnsynch primitive to
>> >> implement its PIP. However, this mechanism only acquires the xysynch
>> >> primitive without releasing it. This results in the resource counter
>> >> never be 0 anymore and thus breaking the auto-relaxed which is based
>> >> on this resource counter.
>> >>
>> >> I guess releasing the xnsynch resets the owner, which is not wanted either..
>> >> Any idea for a fix?
>> >
>> > Could you give a try at this one? TIA,
>> > http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44
>> >
>> >>
>> >> Thanks,
>> >> Henri.
>> >>
>> >>
>> >> On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
>> >> > That was a problem with our build environment.
>> >> >
>> >> > Patches work and the base-prio 0 task nicely switches to the Linux
>> >> > domain and competes with the Linux threads for the processor. Now we
>> >> > don't loose TCP connection to our target anymore.
>> >> >
>> >> > Thanks!
>> >> >
>> >> > Henri
>> >> >
>> >> > On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
>> >> >> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
>> >> >> Tests with a basic application look good, however our bigger project
>> >> >> doesn't. I checked already if we release all mutexes; the base prio 0
>> >> >> thread thread released all of them but stays in primary domain after a
>> >> >> xenomai call.
>> >> >>
>> >> >> I'll setup a clean environment, just to be sure, and will investigate
>> >> >> further tomorrow.
>> >> >>
>> >> >> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
>> >> >>>
>> >> >>>>
>> >> >>>> The auto-relax feature is now available from -head (upcoming
>> >> >>>> 2.6.x). You
>> >> >>>> will need all commits from
>> >> >>>> http://git.xenomai.org/?p=xenomai-
>> >> >>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
>> >> >>>> to
>> >> >>>> http://git.xenomai.org/?p=xenomai-
>> >> >>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
>> >> >>>>
>> >> >>>> --
>> >> >>>> Philippe.
>> >> >>>>
>> >> >>>>
>> >> >>>
>> >> >>> Very nice. Will try it out soon on my G4 PB Aluminium.
>> >> >>>
>> >> >>> Andreas
>> >> >>>
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> Xenomai-help mailing list
>> >> >>> Xenomai-help@domain.hid
>> >> >>> https://mail.gna.org/listinfo/xenomai-help
>> >> >>>
>> >> >>
>> >> >
>> >
>> > --
>> > Philippe.
>> >
>> >
>> >
>
> --
> Philippe.
>
>
>

Re: [Xenomai-help] New scheduler class

[Henri Roosen]

Hi,

We enabled CONFIG_XENO_OPT_DEBUG_NUCLEUS and get the following fatal bug:

 [   26.208785] Xenomai: fatal: bug at
kernel/xenomai/nucleus/synch.c:723 (xnthread_get_rescnt(lastowner) <
0)

This occurs at different places in our code, one of which is in the
rt-serial driver 16550A.c in line 1076. I don't think the code of this
driver is using the out_lock in a wrong way, so any idea what can be
wrong then?

We use Xenomai 2.5.6 on Linux 2.6.32.15 x86 with auto-relax patches from 'head':
http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
to
http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
and
http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=82200f0b66908a5366a72355b3b70472031d68fb
http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44

Thanks,
Henri.

On Thu, May 5, 2011 at 4:40 PM, Henri Roosen <henriroosen@domain.hid> wrote:
> On Thu, May 5, 2011 at 3:55 PM, Philippe Gerum <rpm@xenomai.org> wrote:
>> On Mon, 2011-05-02 at 10:58 +0200, Henri Roosen wrote:
>>> Hi Philippe,
>>>
>>> Thanks for the patch! Unfortunately this does not resolve the 'stuck
>>> in primary of prio 0 after rt_task_send'.
>>>
>>> I'll attach the basic test application that reproduces it. All prints
>>> should contain secondary mode because all tasks are prio 0 and the
>>> auto-relax patches are compiled in.
>>>
>>> The producer which does the rt_task_send() call does not auto-relax to
>>> secondary mode.
>>
>> Your example code works with this added patch on top of both 2.5.x and
>> -head:
>> http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=82200f0b66908a5366a72355b3b70472031d68fb
>>
>
> Hi Philippe!
> I tried this patch and can confirm it results in expected behaviour of
> the prio 0 scheduling now, also in our main application.
>
> Thanks a lot!
> Henri
>
>>>
>>> Thanks,
>>> Henri.
>>>
>>> On Sun, May 1, 2011 at 5:01 PM, Philippe Gerum <rpm@xenomai.org> wrote:
>>> > On Mon, 2011-03-28 at 12:56 +0200, Henri Roosen wrote:
>>> >> Hi,
>>> >>
>>> >> We are using the back-ported auto-relax patches of Xenomai-head on the
>>> >> 2.5.6 release on Linux 2.6.32.15.
>>> >>
>>> >> We noticed that our prio-0 shadowed task is not always auto-relaxed
>>> >> and traced it back to a problem with rt_task_send/receive/reply. The
>>> >> rt_task_send/receive/reply mechanism uses the xnsynch primitive to
>>> >> implement its PIP. However, this mechanism only acquires the xysynch
>>> >> primitive without releasing it. This results in the resource counter
>>> >> never be 0 anymore and thus breaking the auto-relaxed which is based
>>> >> on this resource counter.
>>> >>
>>> >> I guess releasing the xnsynch resets the owner, which is not wanted either..
>>> >> Any idea for a fix?
>>> >
>>> > Could you give a try at this one? TIA,
>>> > http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44
>>> >
>>> >>
>>> >> Thanks,
>>> >> Henri.
>>> >>
>>> >>
>>> >> On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
>>> >> > That was a problem with our build environment.
>>> >> >
>>> >> > Patches work and the base-prio 0 task nicely switches to the Linux
>>> >> > domain and competes with the Linux threads for the processor. Now we
>>> >> > don't loose TCP connection to our target anymore.
>>> >> >
>>> >> > Thanks!
>>> >> >
>>> >> > Henri
>>> >> >
>>> >> > On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
>>> >> >> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
>>> >> >> Tests with a basic application look good, however our bigger project
>>> >> >> doesn't. I checked already if we release all mutexes; the base prio 0
>>> >> >> thread thread released all of them but stays in primary domain after a
>>> >> >> xenomai call.
>>> >> >>
>>> >> >> I'll setup a clean environment, just to be sure, and will investigate
>>> >> >> further tomorrow.
>>> >> >>
>>> >> >> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
>>> >> >>>
>>> >> >>>>
>>> >> >>>> The auto-relax feature is now available from -head (upcoming
>>> >> >>>> 2.6.x). You
>>> >> >>>> will need all commits from
>>> >> >>>> http://git.xenomai.org/?p=xenomai-
>>> >> >>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
>>> >> >>>> to
>>> >> >>>> http://git.xenomai.org/?p=xenomai-
>>> >> >>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
>>> >> >>>>
>>> >> >>>> --
>>> >> >>>> Philippe.
>>> >> >>>>
>>> >> >>>>
>>> >> >>>
>>> >> >>> Very nice. Will try it out soon on my G4 PB Aluminium.
>>> >> >>>
>>> >> >>> Andreas
>>> >> >>>
>>> >> >>>
>>> >> >>> _______________________________________________
>>> >> >>> Xenomai-help mailing list
>>> >> >>> Xenomai-help@domain.hid
>>> >> >>> https://mail.gna.org/listinfo/xenomai-help
>>> >> >>>
>>> >> >>
>>> >> >
>>> >
>>> > --
>>> > Philippe.
>>> >
>>> >
>>> >
>>
>> --
>> Philippe.
>>
>>
>>
>

Re: [Xenomai-help] New scheduler class

[Philippe Gerum]

On Tue, 2011-06-28 at 11:26 +0200, Henri Roosen wrote:
> Hi,
> 
> We enabled CONFIG_XENO_OPT_DEBUG_NUCLEUS and get the following fatal bug:
> 
>  [   26.208785] Xenomai: fatal: bug at
> kernel/xenomai/nucleus/synch.c:723 (xnthread_get_rescnt(lastowner) <
> 0)
> 

This is a known bug, fix pending here.

> This occurs at different places in our code, one of which is in the
> rt-serial driver 16550A.c in line 1076. I don't think the code of this
> driver is using the out_lock in a wrong way, so any idea what can be
> wrong then?
> 
> We use Xenomai 2.5.6 on Linux 2.6.32.15 x86 with auto-relax patches from 'head':
> http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
> to
> http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
> and
> http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=82200f0b66908a5366a72355b3b70472031d68fb
> http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44
> 
> Thanks,
> Henri.
> 
> On Thu, May 5, 2011 at 4:40 PM, Henri Roosen <henriroosen@domain.hid> wrote:
> > On Thu, May 5, 2011 at 3:55 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> >> On Mon, 2011-05-02 at 10:58 +0200, Henri Roosen wrote:
> >>> Hi Philippe,
> >>>
> >>> Thanks for the patch! Unfortunately this does not resolve the 'stuck
> >>> in primary of prio 0 after rt_task_send'.
> >>>
> >>> I'll attach the basic test application that reproduces it. All prints
> >>> should contain secondary mode because all tasks are prio 0 and the
> >>> auto-relax patches are compiled in.
> >>>
> >>> The producer which does the rt_task_send() call does not auto-relax to
> >>> secondary mode.
> >>
> >> Your example code works with this added patch on top of both 2.5.x and
> >> -head:
> >> http://git.xenomai.org/?p=xenomai-head.git;a=commit;h=82200f0b66908a5366a72355b3b70472031d68fb
> >>
> >
> > Hi Philippe!
> > I tried this patch and can confirm it results in expected behaviour of
> > the prio 0 scheduling now, also in our main application.
> >
> > Thanks a lot!
> > Henri
> >
> >>>
> >>> Thanks,
> >>> Henri.
> >>>
> >>> On Sun, May 1, 2011 at 5:01 PM, Philippe Gerum <rpm@xenomai.org> wrote:
> >>> > On Mon, 2011-03-28 at 12:56 +0200, Henri Roosen wrote:
> >>> >> Hi,
> >>> >>
> >>> >> We are using the back-ported auto-relax patches of Xenomai-head on the
> >>> >> 2.5.6 release on Linux 2.6.32.15.
> >>> >>
> >>> >> We noticed that our prio-0 shadowed task is not always auto-relaxed
> >>> >> and traced it back to a problem with rt_task_send/receive/reply. The
> >>> >> rt_task_send/receive/reply mechanism uses the xnsynch primitive to
> >>> >> implement its PIP. However, this mechanism only acquires the xysynch
> >>> >> primitive without releasing it. This results in the resource counter
> >>> >> never be 0 anymore and thus breaking the auto-relaxed which is based
> >>> >> on this resource counter.
> >>> >>
> >>> >> I guess releasing the xnsynch resets the owner, which is not wanted either..
> >>> >> Any idea for a fix?
> >>> >
> >>> > Could you give a try at this one? TIA,
> >>> > http://git.xenomai.org/?p=xenomai-rpm.git;a=commit;h=becb13774ab687d016857f6a972f4124fbd10c44
> >>> >
> >>> >>
> >>> >> Thanks,
> >>> >> Henri.
> >>> >>
> >>> >>
> >>> >> On Tue, Oct 12, 2010 at 9:16 AM, Henri Roosen <henriroosen@domain.hid> wrote:
> >>> >> > That was a problem with our build environment.
> >>> >> >
> >>> >> > Patches work and the base-prio 0 task nicely switches to the Linux
> >>> >> > domain and competes with the Linux threads for the processor. Now we
> >>> >> > don't loose TCP connection to our target anymore.
> >>> >> >
> >>> >> > Thanks!
> >>> >> >
> >>> >> > Henri
> >>> >> >
> >>> >> > On Mon, Oct 11, 2010 at 5:35 PM, Henri Roosen <henriroosen@domain.hid> wrote:
> >>> >> >> Thanks for the patches Philippe. I applied them to xenomai 2.5.5.
> >>> >> >> Tests with a basic application look good, however our bigger project
> >>> >> >> doesn't. I checked already if we release all mutexes; the base prio 0
> >>> >> >> thread thread released all of them but stays in primary domain after a
> >>> >> >> xenomai call.
> >>> >> >>
> >>> >> >> I'll setup a clean environment, just to be sure, and will investigate
> >>> >> >> further tomorrow.
> >>> >> >>
> >>> >> >> On Mon, Oct 11, 2010 at 11:31 AM, Andreas Glatz <andi.glatz@domain.hid> wrote:
> >>> >> >>>
> >>> >> >>>>
> >>> >> >>>> The auto-relax feature is now available from -head (upcoming
> >>> >> >>>> 2.6.x). You
> >>> >> >>>> will need all commits from
> >>> >> >>>> http://git.xenomai.org/?p=xenomai-
> >>> >> >>>> head.git;a=commit;h=b75cec19387e561f82ac55595db8c993b049f071
> >>> >> >>>> to
> >>> >> >>>> http://git.xenomai.org/?p=xenomai-
> >>> >> >>>> head.git;a=commit;h=6653a9e8eb7339b749989bd74adc3ac3bd29e4da
> >>> >> >>>>
> >>> >> >>>> --
> >>> >> >>>> Philippe.
> >>> >> >>>>
> >>> >> >>>>
> >>> >> >>>
> >>> >> >>> Very nice. Will try it out soon on my G4 PB Aluminium.
> >>> >> >>>
> >>> >> >>> Andreas
> >>> >> >>>
> >>> >> >>>
> >>> >> >>> _______________________________________________
> >>> >> >>> Xenomai-help mailing list
> >>> >> >>> Xenomai-help@domain.hid
> >>> >> >>> https://mail.gna.org/listinfo/xenomai-help
> >>> >> >>>
> >>> >> >>
> >>> >> >
> >>> >
> >>> > --
> >>> > Philippe.
> >>> >
> >>> >
> >>> >
> >>
> >> --
> >> Philippe.
> >>
> >>
> >>
> >

-- 
Philippe.