[Xenomai-help] bring down the system

[Xenomai-help] bring down the system

[Stefan Schaal]

Hi,

   attached is a little C program that spawns off a task which runs a  
heavy tiny math job. Despite my 8 processor machine, taking this one  
task to 99% CPU load slows down the entire computer, often such that  
there is no return except for a forced restart.

I run the xenomai-head git, updated on Oct. 29 the last time, on a  
i386 ubuntu system with kernel 2.6.29.5.

Does anybody know what is going wrong here, i.e., am I not permitted  
to take my CPUs to the max load?

Any help would be highly appreciated!!!

Best wishes,

-Stefan



Compile statement:

gcc -o xtest xeno_spawn_test.c -I/usr/xenomai/include -Wl,@/usr/ 
xenomai/lib/posix.wrappers -L/usr/xenomai/lib -lpthre
ad_rt -lpthread -lrt  -lnative

Run:

unix> xtest







------------------------------------------------  xeno_spawn_test.c  
---------------------------------------------
#include <sys/mman.h>
#include <pthread.h>
#include <stdio.h>

pthread_t cthread;

// a quick thread to run ....
void *
run_thread(void *dummy)
{
   int i;

   for (i=1; i<=1000; ++i) {
     printf("%d thread is runnning\n",i);
     sleep(1);
   }
   return NULL;
}

// main program
int
main() {

   int rc;
   pthread_attr_t pth_attr;

#ifdef __XENO__
   mlockall(MCL_CURRENT | MCL_FUTURE);
#endif

   if ((rc=pthread_attr_init(&pth_attr)))
       printf("pthread_attr_init returned %d\n",rc);

   if ((rc=pthread_create( &cthread, &pth_attr, run_thread, NULL)))
       printf("pthread_create returned with %d\n",rc);

   // wait a bit to get some print outs from the thread
   sleep(10);

   return 1;
}

Re: [Xenomai-help] bring down the system

[Stefan Schaal]

Sorry, I attached the wrong C-program in the previous posting ....  
here is the corrected version:

Hi,

  attached is a little C program that spawns off a task which runs a  
heavy tiny math job. Despite my 8 processor machine, taking this one  
task to 99% CPU load slows down the entire computer, often such that  
there is no return except for a forced restart.

I run the xenomai-head git, updated on Oct. 29 the last time, on a  
i386 ubuntu system with kernel 2.6.29.5.

Does anybody know what is going wrong here, i.e., am I not permitted  
to take my CPUs to the max load?

Any help would be highly appreciated!!!

Best wishes,

-Stefan



Compile statement:

gcc -o xtest xeno_spawn_test.c -I/usr/xenomai/include -Wl,@/usr/ 
xenomai/lib/posix.wrappers -L/usr/xenomai/lib -lpthre
ad_rt -lpthread -lrt  -lnative

Run:

unix> xtest



------------------------------------------------  xeno_spawn_test.c  
---------------------------------------------
#include <sys/mman.h>
#include <native/task.h>
#include <pthread.h>
#include <stdio.h>
#include <math.h>

RT_TASK task;

// a quick task run ....
void
run_task(void *dummy)
{
   int i;
   double foo = 1.0;

   for (i=1; i<=100000000; ++i)
     foo = foo*1.1;

   return;
}

// main program
int
main() {

   int rc;
   pthread_attr_t pth_attr;

   mlockall(MCL_CURRENT | MCL_FUTURE);

   // spawn the task
   if ((rc=rt_task_spawn(&task,"run_task",100000,1,T_JOINABLE|T_FPU| 
T_CPU(1),run_task,NULL)))
       printf("rt_task_spawn returned %d\n",rc);

   rt_task_join(&task);

   return 1;
}

Re: [Xenomai-help] bring down the system

[Wolfgang Denk]

Dear Stefan Schaal,

In message <C790DE41-A093-46C5-9F44-D3F0EA839CF5@domain.hid> you wrote:
> Sorry, I attached the wrong C-program in the previous posting ....  
> here is the corrected version:

I'm not sure what exactly you are trying to test, but I think you
should invest a little more time in your test cases.

>   attached is a little C program that spawns off a task which runs a  
> heavy tiny math job. Despite my 8 processor machine, taking this one  

Does it? Not for me.

> void
> run_task(void *dummy)
> {
>    int i;
>    double foo = 1.0;
> 
>    for (i=1; i<=100000000; ++i)
>      foo = foo*1.1;
> 
>    return;
> }

On x86, this code compiles for me into this:

        .text
        .p2align 4,,15
.globl run_task
        .type   run_task, @function
run_task:
        pushl   %ebp
        movl    %esp, %ebp
        popl    %ebp
        ret
        .size   run_task, .-run_task
        .ident  "GCC: (GNU) 4.4.1 20090725 (Red Hat 4.4.1-2)"
        .section        .note.GNU-stack,"",@progbits

Similar on ARM and Power.

As you can see, the compiler completely optimizes away your "tiny math
job". So what should be loading your CPU?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@domain.hid
"Nature is very un-American.  Nature never hurries."
- William George Jordan

Re: [Xenomai-help] bring down the system

[Stefan Schaal]

Hi Wolfgang,

  ok, good point. Let's replace the "math loop" with:

   for (i=1; i<=1000000000; ++i) {
     foo = foo*1.1;
     foo = foo/1.1;
     foo = pow(foo,1.0);
   }

   printf("foo=%f\n",foo);


Will this get your system to an almost stand-still?

-Stefan




On Nov 3, 2009, at 22:52, Wolfgang Denk wrote:

> Dear Stefan Schaal,
>
> In message <C790DE41-A093-46C5-9F44-D3F0EA839CF5@domain.hid> you wrote:
>> Sorry, I attached the wrong C-program in the previous posting ....
>> here is the corrected version:
>
> I'm not sure what exactly you are trying to test, but I think you
> should invest a little more time in your test cases.
>
>>  attached is a little C program that spawns off a task which runs a
>> heavy tiny math job. Despite my 8 processor machine, taking this one
>
> Does it? Not for me.
>
>> void
>> run_task(void *dummy)
>> {
>>   int i;
>>   double foo = 1.0;
>>
>>   for (i=1; i<=100000000; ++i)
>>     foo = foo*1.1;
>>
>>   return;
>> }
>
> On x86, this code compiles for me into this:
>
>        .text
>        .p2align 4,,15
> .globl run_task
>        .type   run_task, @function
> run_task:
>        pushl   %ebp
>        movl    %esp, %ebp
>        popl    %ebp
>        ret
>        .size   run_task, .-run_task
>        .ident  "GCC: (GNU) 4.4.1 20090725 (Red Hat 4.4.1-2)"
>        .section        .note.GNU-stack,"",@progbits
>
> Similar on ARM and Power.
>
> As you can see, the compiler completely optimizes away your "tiny math
> job". So what should be loading your CPU?
>
> Best regards,
>
> Wolfgang Denk
>
> -- 
> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@domain.hid
> "Nature is very un-American.  Nature never hurries."
> - William George Jordan
>

Re: [Xenomai-help] bring down the system

[Jonas Buchli]

wolfgang,

did you compile with exactly our compile statement or with additional 
flags (i.e. optimization)?

compiling with our statement, just throwing in the -S flag, i am getting 
a very getting different assembly code - while has been a while i have 
been reading assembly, it seems our loop is not getting optimized away.

anyway, the point here is not about if this is a meaningful program or 
not, but we are getting system lock ups and this one is a minimal 
program that does reproduce the problem for us.

best
jonas

Wolfgang Denk wrote:
> Dear Stefan Schaal,
> 
> In message <C790DE41-A093-46C5-9F44-D3F0EA839CF5@domain.hid> you wrote:
>> Sorry, I attached the wrong C-program in the previous posting ....  
>> here is the corrected version:
> 
> I'm not sure what exactly you are trying to test, but I think you
> should invest a little more time in your test cases.
> 
>>   attached is a little C program that spawns off a task which runs a  
>> heavy tiny math job. Despite my 8 processor machine, taking this one  
> 
> Does it? Not for me.
> 
>> void
>> run_task(void *dummy)
>> {
>>    int i;
>>    double foo = 1.0;
>>
>>    for (i=1; i<=100000000; ++i)
>>      foo = foo*1.1;
>>
>>    return;
>> }
> 
> On x86, this code compiles for me into this:
> 
>         .text
>         .p2align 4,,15
> .globl run_task
>         .type   run_task, @function
> run_task:
>         pushl   %ebp
>         movl    %esp, %ebp
>         popl    %ebp
>         ret
>         .size   run_task, .-run_task
>         .ident  "GCC: (GNU) 4.4.1 20090725 (Red Hat 4.4.1-2)"
>         .section        .note.GNU-stack,"",@progbits
> 
> Similar on ARM and Power.
> 
> As you can see, the compiler completely optimizes away your "tiny math
> job". So what should be loading your CPU?
> 
> Best regards,
> 
> Wolfgang Denk
> 

-- 
--------------------------------------------------------------
Jonas Buchli, Dr.sc., MSc EE
Computational Learning and Motor Control Lab
University of Southern California
http://www-clmc.usc.edu/
+1 (213) 740 67 17
--------------------------------------------------------------

Re: [Xenomai-help] bring down the system

[Wolfgang Denk]

Dear Jonas Buchli,

In message <4AF12B70.3020004@domain.hid> you wrote:
> 
> did you compile with exactly our compile statement or with additional 
> flags (i.e. optimization)?

With optimization, of course.

> compiling with our statement, just throwing in the -S flag, i am getting 
> a very getting different assembly code - while has been a while i have 
> been reading assembly, it seems our loop is not getting optimized away.
> 
> anyway, the point here is not about if this is a meaningful program or 
> not, but we are getting system lock ups and this one is a minimal 
> program that does reproduce the problem for us.

What sort of machine is this?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@domain.hid
Do you suppose the reason the ends of the `Intel Inside'  logo  don't
match up is that it was drawn on a Pentium?

Re: [Xenomai-help] bring down the system

[Jonas Buchli]

> In message <4AF12B70.3020004@domain.hid> you wrote:
>> did you compile with exactly our compile statement or with additional 
>> flags (i.e. optimization)?
> 
> With optimization, of course.

well, we did not want to burden with you the 70k lines of code, that 
even when compiled optimized leads to the lock ups, and tried to come up 
with this little test case, but to be useful it needs to be compiled 
with the statement provided i guess ;)

could you try if you can reproduce it when not compiled with optimization?

> What sort of machine is this?

it's a 8 cpu dell rack server (sorry i don't have the model at hand and 
am not at the same site as the machine right now).

i attach the content of /proc/cpuinfo what other info would you need?

thanks a lot for looking into this!
jonas


--------------------------------------------------
/proc/cpuinfo (only for one processor, omitted the others, there are 8)


processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Xeon(R) CPU           E5440  @ 2.83GHz
stepping	: 10
cpu MHz		: 2826.146
cache size	: 6144 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm 
constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx est tm2 
ssse3 cx16 xtpr pdcm dca sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority
bogomips	: 5652.29
clflush size	: 64
power management:


-- 
--------------------------------------------------------------
Jonas Buchli, Dr.sc., MSc EE
Computational Learning and Motor Control Lab
University of Southern California
http://www-clmc.usc.edu/
+1 (213) 740 67 17
--------------------------------------------------------------

Re: [Xenomai-help] bring down the system

[Gilles Chanteperdrix]

Stefan Schaal wrote:
> Sorry, I attached the wrong C-program in the previous posting ....  
> here is the corrected version:
> 
> Hi,
> 
>   attached is a little C program that spawns off a task which runs a  
> heavy tiny math job. Despite my 8 processor machine, taking this one  
> task to 99% CPU load slows down the entire computer, often such that  
> there is no return except for a forced restart.
> 
> I run the xenomai-head git, updated on Oct. 29 the last time, on a  
> i386 ubuntu system with kernel 2.6.29.5.
> 
> Does anybody know what is going wrong here, i.e., am I not permitted  
> to take my CPUs to the max load?

No, you can not do that. When a Xenomai task runs, Linux does not, and
for your system to work correctly, Linux should run from time to time,
so, the typical Xenomai setup is a periodic task which wakes up, does
some job, and goes back to sleep, leaving some time for linux to run.

-- 
                                          Gilles