[Xenomai-core] xenomai-forge: round-robin scheduling in pSOS skin

[Xenomai-core] xenomai-forge: round-robin scheduling in pSOS skin

[Ronny Meeus]

[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]

Hello

I'm are using the xenomai-forge pSOS skin (Mercury).
My application is running on a P4040 (Freescale PPC with 4 cores).
Some code snippets are put in this mail but the complete testcode is
also attached.

I have a test task that just consumes the CPU:

int run_test = 1;
static void perform_work(u_long counter,u_long b,u_long c,u_long d)
{
  int i;
  while (run_test) {
    for (i=0;i<100000;i++);
    (*(unsigned long*)counter)++;
  }
  while (1) tm_wkafter(1000);
}

If I create 2 instances of this task with the T_SLICE option set:

    t_create("WORK",10,0,0,0,&tid);
    t_start(tid,T_TSLICE, perform_work, args);


I see that only 1 task is consuming CPU.

# taskset 1 ./roundrobin.exe &
#    0"000.543| [main] SCHED_RT priorities => [1 .. 99]
   0"000.656| [main] SCHED_RT.99 reserved for IRQ emulation
   0"000.692| [main] SCHED_RT.98 reserved for scheduler-lock emulation
0 -> 6602
1 -> 0

If I adapt the code so that I call in my init the threadobj_start_rr
function, I see that the load is equally distributed over the 2
threads:

# taskset 1 ./roundrobin.exe &
#    0"000.557| [main] SCHED_RT priorities => [1 .. 99]
   0"000.672| [main] SCHED_RT.99 reserved for IRQ emulation
   0"000.708| [main] SCHED_RT.98 reserved for scheduler-lock emulation
0 -> 3290
1 -> 3291

Here are the questions:
- why is the threadobj_start_rr function not called from the context
of the init of the psos layer.
- why is the roundrobin implemented in this way? If the tasks would be
mapped on the SCHED_RR instead of the SCHED_FF the Linux scheduler
would take care of this.
On the other hand, once the threadobj_start_rr function is called from
my init, and I create the tasks in T_NOTSLICE mode, the time-slicing
is still done.

Thanks.

---
Ronny

[-- Attachment #2: roundrobin.c --]
[-- Type: text/x-csrc, Size: 1003 bytes --]

#include <stdio.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <psos/psos.h>

int run_test = 1;
static void perform_work(u_long counter,u_long b,u_long c,u_long d)
{
  int i;
  while (run_test) {
    for (i=0;i<100000;i++);
    (*(unsigned long*)counter)++;
  }
  while (1) tm_wkafter(1000);
}

static unsigned long counter[2];

static void test()
{
  u_long tid,args[4] = {0,0,0,0};
  int i;

  for (i=0;i<2;i++) {
    args[0] = (u_long)&(counter[i]);
    t_create("WORK",10,0,0,0,&tid);
    t_start(tid,T_TSLICE, perform_work, args);
  }
  tm_wkafter(10000);
  run_test = 0;

  for (i=0;i<2;i++) {
    printf("%i -> %lu\n",i,counter[i]);
  }
  while (1) tm_wkafter(1000);
}

int main(int argc,char *argv[], char**envp)
{
  u_long oldprio;
  struct timespec quantum;

  mlockall(MCL_CURRENT | MCL_FUTURE);
  copperplate_init(argc,argv);

  quantum.tv_sec = 0;
  quantum.tv_nsec = 5 * 1000 * 1000; // use 5ms
  threadobj_start_rr(&quantum);
  t_setpri(0,50,&oldprio);
  test();
  return 0;
}

Re: [Xenomai-core] xenomai-forge: round-robin scheduling in pSOS skin

[Ronny Meeus]

On Thu, Mar 8, 2012 at 3:30 PM, Ronny Meeus <ronny.meeus@domain.hid> wrote:
> Hello
>
> I'm are using the xenomai-forge pSOS skin (Mercury).
> My application is running on a P4040 (Freescale PPC with 4 cores).
> Some code snippets are put in this mail but the complete testcode is
> also attached.
>
> I have a test task that just consumes the CPU:
>
> int run_test = 1;
> static void perform_work(u_long counter,u_long b,u_long c,u_long d)
> {
>  int i;
>  while (run_test) {
>    for (i=0;i<100000;i++);
>    (*(unsigned long*)counter)++;
>  }
>  while (1) tm_wkafter(1000);
> }
>
> If I create 2 instances of this task with the T_SLICE option set:
>
>    t_create("WORK",10,0,0,0,&tid);
>    t_start(tid,T_TSLICE, perform_work, args);
>
>
> I see that only 1 task is consuming CPU.
>
> # taskset 1 ./roundrobin.exe &
> #    0"000.543| [main] SCHED_RT priorities => [1 .. 99]
>   0"000.656| [main] SCHED_RT.99 reserved for IRQ emulation
>   0"000.692| [main] SCHED_RT.98 reserved for scheduler-lock emulation
> 0 -> 6602
> 1 -> 0
>
> If I adapt the code so that I call in my init the threadobj_start_rr
> function, I see that the load is equally distributed over the 2
> threads:
>
> # taskset 1 ./roundrobin.exe &
> #    0"000.557| [main] SCHED_RT priorities => [1 .. 99]
>   0"000.672| [main] SCHED_RT.99 reserved for IRQ emulation
>   0"000.708| [main] SCHED_RT.98 reserved for scheduler-lock emulation
> 0 -> 3290
> 1 -> 3291
>
> Here are the questions:
> - why is the threadobj_start_rr function not called from the context
> of the init of the psos layer.
> - why is the roundrobin implemented in this way? If the tasks would be
> mapped on the SCHED_RR instead of the SCHED_FF the Linux scheduler
> would take care of this.
> On the other hand, once the threadobj_start_rr function is called from
> my init, and I create the tasks in T_NOTSLICE mode, the time-slicing
> is still done.
>
> Thanks.
>
> ---
> Ronny

Any comments on this?

---
Ronny

Re: [Xenomai-core] xenomai-forge: round-robin scheduling in pSOS skin

[Gilles Chanteperdrix]

On 03/15/2012 08:49 PM, Ronny Meeus wrote:
> On Thu, Mar 8, 2012 at 3:30 PM, Ronny Meeus <ronny.meeus@domain.hid> wrote:
>> Hello
>>
>> I'm are using the xenomai-forge pSOS skin (Mercury).
>> My application is running on a P4040 (Freescale PPC with 4 cores).
>> Some code snippets are put in this mail but the complete testcode is
>> also attached.
>>
>> I have a test task that just consumes the CPU:
>>
>> int run_test = 1;
>> static void perform_work(u_long counter,u_long b,u_long c,u_long d)
>> {
>>  int i;
>>  while (run_test) {
>>    for (i=0;i<100000;i++);
>>    (*(unsigned long*)counter)++;
>>  }
>>  while (1) tm_wkafter(1000);
>> }
>>
>> If I create 2 instances of this task with the T_SLICE option set:
>>
>>    t_create("WORK",10,0,0,0,&tid);
>>    t_start(tid,T_TSLICE, perform_work, args);
>>
>>
>> I see that only 1 task is consuming CPU.
>>
>> # taskset 1 ./roundrobin.exe &
>> #    0"000.543| [main] SCHED_RT priorities => [1 .. 99]
>>   0"000.656| [main] SCHED_RT.99 reserved for IRQ emulation
>>   0"000.692| [main] SCHED_RT.98 reserved for scheduler-lock emulation
>> 0 -> 6602
>> 1 -> 0
>>
>> If I adapt the code so that I call in my init the threadobj_start_rr
>> function, I see that the load is equally distributed over the 2
>> threads:
>>
>> # taskset 1 ./roundrobin.exe &
>> #    0"000.557| [main] SCHED_RT priorities => [1 .. 99]
>>   0"000.672| [main] SCHED_RT.99 reserved for IRQ emulation
>>   0"000.708| [main] SCHED_RT.98 reserved for scheduler-lock emulation
>> 0 -> 3290
>> 1 -> 3291
>>
>> Here are the questions:
>> - why is the threadobj_start_rr function not called from the context
>> of the init of the psos layer.
>> - why is the roundrobin implemented in this way? If the tasks would be
>> mapped on the SCHED_RR instead of the SCHED_FF the Linux scheduler
>> would take care of this.
>> On the other hand, once the threadobj_start_rr function is called from
>> my init, and I create the tasks in T_NOTSLICE mode, the time-slicing
>> is still done.
>>
>> Thanks.
>>
>> ---
>> Ronny
> 
> Any comments on this?

I am afraid you will have to wait for Philippe to have time to answer
you. I am a bit ignorant about the psos API, but most importantly
completely ignorant of the mercury. I am working on forge, but mostly
with cobalt.

-- 
                                                                Gilles.

Re: [Xenomai-core] xenomai-forge: round-robin scheduling in pSOS skin

[Philippe Gerum]

On 03/08/2012 03:30 PM, Ronny Meeus wrote:
> Hello
>
> I'm are using the xenomai-forge pSOS skin (Mercury).
> My application is running on a P4040 (Freescale PPC with 4 cores).
> Some code snippets are put in this mail but the complete testcode is
> also attached.
>
> I have a test task that just consumes the CPU:
>
> int run_test = 1;
> static void perform_work(u_long counter,u_long b,u_long c,u_long d)
> {
>    int i;
>    while (run_test) {
>      for (i=0;i<100000;i++);
>      (*(unsigned long*)counter)++;
>    }
>    while (1) tm_wkafter(1000);
> }
>
> If I create 2 instances of this task with the T_SLICE option set:
>
>      t_create("WORK",10,0,0,0,&tid);
>      t_start(tid,T_TSLICE, perform_work, args);
>
>
> I see that only 1 task is consuming CPU.
>
> # taskset 1 ./roundrobin.exe&
> #    0"000.543| [main] SCHED_RT priorities =>  [1 .. 99]
>     0"000.656| [main] SCHED_RT.99 reserved for IRQ emulation
>     0"000.692| [main] SCHED_RT.98 reserved for scheduler-lock emulation
> 0 ->  6602
> 1 ->  0
>
> If I adapt the code so that I call in my init the threadobj_start_rr
> function, I see that the load is equally distributed over the 2
> threads:
>
> # taskset 1 ./roundrobin.exe&
> #    0"000.557| [main] SCHED_RT priorities =>  [1 .. 99]
>     0"000.672| [main] SCHED_RT.99 reserved for IRQ emulation
>     0"000.708| [main] SCHED_RT.98 reserved for scheduler-lock emulation
> 0 ->  3290
> 1 ->  3291
>
> Here are the questions:
> - why is the threadobj_start_rr function not called from the context
> of the init of the psos layer.

Because threadobj_start_rr() was originally designed to activate 
round-robin for all threads (some RTOS like VxWorks expose that kind of 
API), not on a per-thread basis. This is not what pSOS wants.

The round-robin API is in state of flux for mercury, only the cobalt one 
is stable. This is why RR is not yet activated despite T_SLICE is 
recognized.

> - why is the roundrobin implemented in this way? If the tasks would be
> mapped on the SCHED_RR instead of the SCHED_FF the Linux scheduler
> would take care of this.

Nope. We need per-thread RR intervals, to manage multiple priority 
groups concurrently, and we also want to define that interval as we see 
fit for proper RTOS emulation. POSIX does not define anything like 
sched_set_rr_interval(), and the linux kernel applies a default fixed 
interval to all threads from the SCHED_RR class (100ms IIRC).

So we have to emulate SCHED_RR over SCHED_FIFO plus a per-thread virtual 
timer.

> On the other hand, once the threadobj_start_rr function is called from
> my init, and I create the tasks in T_NOTSLICE mode, the time-slicing
> is still done.

Because you called threadobj_start_rr().

>
> Thanks.
>
> ---
> Ronny
>
>
>
> _______________________________________________
> Xenomai-core mailing list
> Xenomai-core@domain.hid
> https://mail.gna.org/listinfo/xenomai-core


-- 
Philippe.

Re: [Xenomai-core] xenomai-forge: round-robin scheduling in pSOS skin

[Philippe Gerum]

On 03/08/2012 03:30 PM, Ronny Meeus wrote:

> # taskset 1 ./roundrobin.exe&
> #    0"000.557| [main] SCHED_RT priorities =>  [1 .. 99]
>     0"000.672| [main] SCHED_RT.99 reserved for IRQ emulation
>     0"000.708| [main] SCHED_RT.98 reserved for scheduler-lock emulation

Btw, the copperplate support you are currently using is seriously outdated.

-- 
Philippe.

Re: [Xenomai-help] [Xenomai-core] xenomai-forge: round-robin scheduling in pSOS skin

[Ronny Meeus]

[-- Attachment #1: Type: text/plain, Size: 5445 bytes --]

[ In my last mail I did not put xenomai-help in CC, corrected by this. ]

On Sat, Mar 17, 2012 at 11:52 PM, Ronny Meeus <ronny.meeus@domain.hid> wrote:
> On Sat, Mar 17, 2012 at 11:42 AM, Philippe Gerum <rpm@xenomai.org> wrote:
>> On 03/08/2012 03:30 PM, Ronny Meeus wrote:
>>>
>>> Hello
>>>
>>> I'm are using the xenomai-forge pSOS skin (Mercury).
>>> My application is running on a P4040 (Freescale PPC with 4 cores).
>>> Some code snippets are put in this mail but the complete testcode is
>>> also attached.
>>>
>>> I have a test task that just consumes the CPU:
>>>
>>> int run_test = 1;
>>> static void perform_work(u_long counter,u_long b,u_long c,u_long d)
>>> {
>>>   int i;
>>>   while (run_test) {
>>>     for (i=0;i<100000;i++);
>>>     (*(unsigned long*)counter)++;
>>>   }
>>>   while (1) tm_wkafter(1000);
>>> }
>>>
>>> If I create 2 instances of this task with the T_SLICE option set:
>>>
>>>     t_create("WORK",10,0,0,0,&tid);
>>>     t_start(tid,T_TSLICE, perform_work, args);
>>>
>>>
>>> I see that only 1 task is consuming CPU.
>>>
>>> # taskset 1 ./roundrobin.exe&
>>> #    0"000.543| [main] SCHED_RT priorities =>  [1 .. 99]
>>>    0"000.656| [main] SCHED_RT.99 reserved for IRQ emulation
>>>    0"000.692| [main] SCHED_RT.98 reserved for scheduler-lock emulation
>>> 0 ->  6602
>>> 1 ->  0
>>>
>>> If I adapt the code so that I call in my init the threadobj_start_rr
>>> function, I see that the load is equally distributed over the 2
>>> threads:
>>>
>>> # taskset 1 ./roundrobin.exe&
>>> #    0"000.557| [main] SCHED_RT priorities =>  [1 .. 99]
>>>    0"000.672| [main] SCHED_RT.99 reserved for IRQ emulation
>>>    0"000.708| [main] SCHED_RT.98 reserved for scheduler-lock emulation
>>> 0 ->  3290
>>> 1 ->  3291
>>>
>>> Here are the questions:
>>> - why is the threadobj_start_rr function not called from the context
>>> of the init of the psos layer.
>>
>>
>> Because threadobj_start_rr() was originally designed to activate round-robin
>> for all threads (some RTOS like VxWorks expose that kind of API), not on a
>> per-thread basis. This is not what pSOS wants.
>>
>> The round-robin API is in state of flux for mercury, only the cobalt one is
>> stable. This is why RR is not yet activated despite T_SLICE is recognized.
>>
>>
>>> - why is the roundrobin implemented in this way? If the tasks would be
>>> mapped on the SCHED_RR instead of the SCHED_FF the Linux scheduler
>>> would take care of this.
>>
>>
>> Nope. We need per-thread RR intervals, to manage multiple priority groups
>> concurrently, and we also want to define that interval as we see fit for
>> proper RTOS emulation. POSIX does not define anything like
>> sched_set_rr_interval(), and the linux kernel applies a default fixed
>> interval to all threads from the SCHED_RR class (100ms IIRC).
>>
>> So we have to emulate SCHED_RR over SCHED_FIFO plus a per-thread virtual
>> timer.
>>
>
> OK I understand.
>
>>
>>> On the other hand, once the threadobj_start_rr function is called from
>>> my init, and I create the tasks in T_NOTSLICE mode, the time-slicing
>>> is still done.
>>
>>
>> Because you called threadobj_start_rr().
>>
>
> Yes, indeed I called threadobj_start_rr to install the virtual timer
> and to attach the signal handler, but the tasks are not running in
> time-sliced mode. This means that the flag THREADOBJ_ROUNDROBIN is not
> set within the thread object so I do not understand why I still see
> the timeslicing being applied.
> It looks to me that once the timer fires and the signal handler is
> called, there is a context switch made anyhow.
>

I was studying the code in more detail this time and I observed
something which is strange to me.
In the function threadobj_start_rr, both the round-robin virtual timer
mechanism is installed and also the round-robin flag is set for all
tasks currently existing in the system (This is what I missed before).
Also from that moment on, all new tasks are running on round-robin
mode, independent of the flags specified during the task creation. The
only exception to this is that when a task calls t_mode to disable
round-robin, in this case the round-robin is disabled. So for me this
is not completely consistent and certainly not to what the user
expects.


I did a change and introduced a new function: threadobj_start_global_rr
The patch is attached to the mail.

I basically have split the installation of the signal handler and the
activation of the round-robin mode for all tasks into 2 parts.
The original function  threadobj_start_rr just installs the signal
handler and starts the timer with the interval that the user has
specified.
The newly introduced function (threadobj_start_global_rr) marks all
tasks as round-robin active and all future tasks will also be marked.
By splitting the logic of this function it is possible to work in a
mode where the tasks itself (based on the flags given during
task-creation or via t_mode) decides on the mode in which it wants to
run or the application can decide that the round-robin will be
activated globally (by calling  threadobj_start_global_rr once during
init for example). In the latter case the round-robin activation
because global for all threads in the application process.

I tested the change in my version and it looks like it behaves like it
should be.

Thanks
---
Ronny

[-- Attachment #2: global_rr.patch --]
[-- Type: application/octet-stream, Size: 656 bytes --]

--- a/lib/copperplate/threadobj.c	2011-10-12 12:11:04.000000000 +0200
+++ b/lib/copperplate/threadobj.c	2012-03-27 12:01:38.000000000 +0200
@@ -572,15 +572,22 @@
 	return 0;
 }
 
+int threadobj_start_global_rr(void)
+{
+	int ret;
+	ret = threadobj_set_rr(NULL, &global_quantum);
+	if (ret)
+		return __bt(ret);
+	return 0;
+}
+
 int threadobj_start_rr(struct timespec *quantum)
 {
 	struct itimerval value, ovalue;
 	struct sigaction sa;
 	int ret;
 
-	ret = threadobj_set_rr(NULL, quantum);
-	if (ret)
-		return __bt(ret);
+	global_quantum = *quantum;
 
 	value.it_interval.tv_sec = quantum->tv_sec;
 	value.it_interval.tv_usec = quantum->tv_nsec / 1000;