[Xenomai-help] xenomai user app working very slowly

[Xenomai-help] xenomai user app working very slowly

[Landau, Bracha]

[-- Attachment #1: Type: text/plain, Size: 856 bytes --]

I converted my regular posix application to a xenomai-posix application, hoping to see an improvement in performance. However, the new xenomai application runs e-x-t-r-e-m-e-l-y slowly, and seems to take undue amounts of CPU time.

I configured the xenomai with -enable-smp and -enable-x86-sep. It is an Atom processor (a single cpu).

Any ideas why this is happening?

________________________________
This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes.
To protect the environment please do not print this e-mail unless necessary.

An NDS Group Limited company. www.nds.com

[-- Attachment #2: Type: text/html, Size: 5315 bytes --]

Re: [Xenomai-help] xenomai user app working very slowly

[Gilles Chanteperdrix]

Landau, Bracha wrote:
> I converted my regular posix application to a xenomai-posix
> application, hoping to see an improvement in performance. However,
> the new xenomai application runs e-x-t-r-e-m-e-l-y slowly, and seems
> to take undue amounts of CPU time.
> 
> I configured the xenomai with -enable-smp and -enable-x86-sep. It is
> an Atom processor (a single cpu).
> 
> Any ideas why this is happening?

Check:
http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai
and:
http://www.xenomai.org/index.php/FAQs#What_may_cause_a_freeze_or_machine_lockup.3F

If you do not find what is wrong, please isolate the issue you have in a
small self-contained test case, which we can compile and run to
investigate your issue.

I insist on "which we can compile", we are not interested in partial
pieces of code with ellipsis where we have to figure out what you add to
run your test.

-- 
Gilles Chanteperdrix, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [Xenomai-help] xenomai user app working very slowly

[Landau, Bracha]

After reading one of the links below, I added calls to pthread_setstacksize to limit memory usage, and this improved the timing.
However, I still have several problems with the application.
In a thread I have a call to select which waits on two message queues. Each of these is a real-time message queue. In the regular application this works fine, but in the xenomai application the select returns with both sets returning true to "FD_ISSET", when there really is nothing to read from the queue.
I did not find anything in the links that refers to problems with select (as long as they are both r/t descriptors, which they are).


-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@gmail.com]
Sent: Monday, January 11, 2010 7:50 PM
To: Landau, Bracha
Cc: xenomai-help@gna.org
Subject: Re: [Xenomai-help] xenomai user app working very slowly

Landau, Bracha wrote:
> I converted my regular posix application to a xenomai-posix
> application, hoping to see an improvement in performance. However,
> the new xenomai application runs e-x-t-r-e-m-e-l-y slowly, and seems
> to take undue amounts of CPU time.
>
> I configured the xenomai with -enable-smp and -enable-x86-sep. It is
> an Atom processor (a single cpu).
>
> Any ideas why this is happening?

Check:
http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai
and:
http://www.xenomai.org/index.php/FAQs#What_may_cause_a_freeze_or_machine_lockup.3F

If you do not find what is wrong, please isolate the issue you have in a
small self-contained test case, which we can compile and run to
investigate your issue.

I insist on "which we can compile", we are not interested in partial
pieces of code with ellipsis where we have to figure out what you add to
run your test.

--
                                            Gilles.

This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes.
To protect the environment please do not print this e-mail unless necessary.

An NDS Group Limited company. www.nds.com

Re: [Xenomai-help] xenomai user app working very slowly

[Gilles Chanteperdrix]

Landau, Bracha wrote:
> After reading one of the links below, I added calls to
> pthread_setstacksize to limit memory usage, and this improved the
> timing. However, I still have several problems with the application. 
> In a thread I have a call to select which waits on two message
> queues. Each of these is a real-time message queue. In the regular
> application this works fine, but in the xenomai application the
> select returns with both sets returning true to "FD_ISSET", when
> there really is nothing to read from the queue. I did not find
> anything in the links that refers to problems with select (as long as
> they are both r/t descriptors, which they are).

Please verify that you still have the same issue with Xenomai latest
stable release (v2.5.0), then send a test case which has this issue.

-- 
					    Gilles.

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Landau, Bracha]

I rebuilt with 2.5.0 and that particular problem is gone.
However, I see a different behavior between the regular implementation and the xenomai implementation:

I have a series of identical threads waiting, using select, on one of two message queues. One message queues is waited upon only by that thread, and the other is waited upon by all the threads. When the select returns the thread reads from the queue that has input ready.

I sent messages to the general queue (i.e., the one waited upon by multiple threads).

In the regular (non-xenomai) implementation, I see that the messages were processed more-or-less equally among all the waiting threads.

In the xenomai implementation, all the messages were processed by a single thread.

Is there any way to make the scheduling done more "fairly" in xenomai?

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
Sent: Tuesday, January 12, 2010 1:37 PM
To: Landau, Bracha
Cc: Gilles Chanteperdrix; xenomai-help@gna.org
Subject: Re: [Xenomai-help] xenomai user app working very slowly

Landau, Bracha wrote:
> After reading one of the links below, I added calls to
> pthread_setstacksize to limit memory usage, and this improved the
> timing. However, I still have several problems with the application.
> In a thread I have a call to select which waits on two message
> queues. Each of these is a real-time message queue. In the regular
> application this works fine, but in the xenomai application the
> select returns with both sets returning true to "FD_ISSET", when
> there really is nothing to read from the queue. I did not find
> anything in the links that refers to problems with select (as long as
> they are both r/t descriptors, which they are).

Please verify that you still have the same issue with Xenomai latest
stable release (v2.5.0), then send a test case which has this issue.

--
                                            Gilles.

This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes.
To protect the environment please do not print this e-mail unless necessary.

An NDS Group Limited company. www.nds.com

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Gilles Chanteperdrix]

Landau, Bracha wrote:
> I rebuilt with 2.5.0 and that particular problem is gone.
> However, I see a different behavior between the regular implementation and the xenomai implementation:
> 
> I have a series of identical threads waiting, using select, on one of two message queues. One message queues is waited upon only by that thread, and the other is waited upon by all the threads. When the select returns the thread reads from the queue that has input ready.
> 
> I sent messages to the general queue (i.e., the one waited upon by multiple threads).
> 
> In the regular (non-xenomai) implementation, I see that the messages were processed more-or-less equally among all the waiting threads.
> 
> In the xenomai implementation, all the messages were processed by a single thread.
> 
> Is there any way to make the scheduling done more "fairly" in xenomai?

The SCHED_FIFO policy is the contrary of fairness. Have you try to use
the SCHED_FIFO policy with Linux too?

There is not much point using several thread to select on the same file
descriptor. The point of select is rather the reverse: being able to
wait for several file descriptors with only one thread. If you have

If the thread have different priorities, the woken up thread is always
the thread with the highest priority.

Other than that, I do not really understand what you are talking about,
your description of the problem is rather vague. Please post a test case.

-- 
Gilles Chanteperdrix, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Landau, Bracha]

You say "There is not much point using several thread to select on the same file descriptor. The point of select is rather the reverse: being able to wait for several file descriptors with only one thread"
I rely on this in my design. Are you saying that Xenomai does not support having a few threads wait on the same file descriptor?

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@free-electrons.com]
Sent: Thursday, January 14, 2010 4:50 PM
To: Landau, Bracha
Cc: xenomai-help@gna.org
Subject: Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

Landau, Bracha wrote:
> I rebuilt with 2.5.0 and that particular problem is gone.
> However, I see a different behavior between the regular implementation and the xenomai implementation:
>
> I have a series of identical threads waiting, using select, on one of two message queues. One message queues is waited upon only by that thread, and the other is waited upon by all the threads. When the select returns the thread reads from the queue that has input ready.
>
> I sent messages to the general queue (i.e., the one waited upon by multiple threads).
>
> In the regular (non-xenomai) implementation, I see that the messages were processed more-or-less equally among all the waiting threads.
>
> In the xenomai implementation, all the messages were processed by a single thread.
>
> Is there any way to make the scheduling done more "fairly" in xenomai?

The SCHED_FIFO policy is the contrary of fairness. Have you try to use
the SCHED_FIFO policy with Linux too?

There is not much point using several thread to select on the same file
descriptor. The point of select is rather the reverse: being able to
wait for several file descriptors with only one thread. If you have

If the thread have different priorities, the woken up thread is always
the thread with the highest priority.

Other than that, I do not really understand what you are talking about,
your description of the problem is rather vague. Please post a test case.

--
Gilles Chanteperdrix, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes.
To protect the environment please do not print this e-mail unless necessary.

An NDS Group Limited company. www.nds.com

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Gilles Chanteperdrix]

Landau, Bracha wrote:
> You say "There is not much point using several thread to select on the same file descriptor. The point of select is rather the reverse: being able to wait for several file descriptors with only one thread"
> I rely on this in my design. Are you saying that Xenomai does not support having a few threads wait on the same file descriptor?

No, I say this is braindamaged design, because you will get lots of
spurious wakeups. But it should work.

-- 
Gilles Chanteperdrix, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Landau, Bracha wrote:
>> You say "There is not much point using several thread to select on the same file descriptor. The point of select is rather the reverse: being able to wait for several file descriptors with only one thread"
>> I rely on this in my design. Are you saying that Xenomai does not support having a few threads wait on the same file descriptor?
> 
> No, I say this is braindamaged design, because you will get lots of
> spurious wakeups. But it should work.

And to get it working, you have to be sure to put all the file
descriptors in non-blocking mode, and be ready for the read/recv
operation to fail with errno set to -EWOULDBLOCK, because if two threads
wake up because select signaled that the same file descriptor is ready,
one thread will win the race, and read will succeed, the other will not.
If the file descriptor is not in non-blocking mode, the second read will
block its caller.

-- 
					    Gilles.

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Landau, Bracha]

Each thread represents a different physical resource. The message queue feeding into the thread contains messages that may be serviced by any one of the resources.

1) Why would there be spurious wakeups?
2) I did the mq_open as nonblocking.
3) It did "work", but all the messages were serviced by a single thread. This is as opposed to the regular (non-xenomai) implementation, in which the messages are distributed evenly among the threads.



-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
Sent: Thursday, January 21, 2010 6:41 PM
To: Landau, Bracha
Cc: xenomai-help@gna.org
Subject: Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

Gilles Chanteperdrix wrote:
> Landau, Bracha wrote:
>> You say "There is not much point using several thread to select on the same file descriptor. The point of select is rather the reverse: being able to wait for several file descriptors with only one thread"
>> I rely on this in my design. Are you saying that Xenomai does not support having a few threads wait on the same file descriptor?
>
> No, I say this is braindamaged design, because you will get lots of
> spurious wakeups. But it should work.

And to get it working, you have to be sure to put all the file
descriptors in non-blocking mode, and be ready for the read/recv
operation to fail with errno set to -EWOULDBLOCK, because if two threads
wake up because select signaled that the same file descriptor is ready,
one thread will win the race, and read will succeed, the other will not.
If the file descriptor is not in non-blocking mode, the second read will
block its caller.

--
                                            Gilles.

This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@nds.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes.
To protect the environment please do not print this e-mail unless necessary.

An NDS Group Limited company. www.nds.com

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Gilles Chanteperdrix]

Landau, Bracha wrote:
> Each thread represents a different physical resource. The message
> queue feeding into the thread contains messages that may be serviced
> by any one of the resources.
> 
> 1) Why would there be spurious wakeups?

Read my mail again. I explained it.

> 2) I did the mq_open as nonblocking. 3) It did "work", but all the
> messages were serviced by a single thread. This is as opposed to the
> regular (non-xenomai) implementation, in which the messages are
> distributed evenly among the threads.

It is the second time you tell me this. And this mail is the third time
I ask you to show me some code proving your affirmations.

Maybe you are right, maybe you are not and your code is buggy.
Therefore, until I receive a valid test case proving your affirmations,
I will do nothing about them.

-- 
					    Gilles.

Re: [Xenomai-help] xenomai user app working very slowly - "select" issue

[Philippe Gerum]

On Thu, 2010-01-21 at 18:45 +0200, Landau, Bracha wrote:
> Each thread represents a different physical resource. The message queue feeding into the thread contains messages that may be serviced by any one of the resources.
> 
> 1) Why would there be spurious wakeups?
> 2) I did the mq_open as nonblocking.
> 3) It did "work", but all the messages were serviced by a sing

What you might look for, is the round-robin scheduling policy to divide
your timeline in multiple slots your different threads could run in. The
only upside of this would be to prevent any given thread to keep the CPU
busy for too long for processing a single message, at the expense of not
always processing the messages in strict FIFO ordering.

But if you want to stick to the FIFO policy to implement a thread pool,
for polling a single fd with threads that rarely block to handle a
message, I don't understand your design more than Gilles does, I'm
afraid.

> 
> 
> -----Original Message-----
> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org]
> Sent: Thursday, January 21, 2010 6:41 PM
> To: Landau, Bracha
> Cc: xenomai@xenomai.org
> Subject: Re: [Xenomai-help] xenomai user app working very slowly - "select" issue
> 
> Gilles Chanteperdrix wrote:
> > Landau, Bracha wrote:
> >> You say "There is not much point using several thread to select on the same file descriptor. The point of select is rather the reverse: being able to wait for several file descriptors with only one thread"
> >> I rely on this in my design. Are you saying that Xenomai does not support having a few threads wait on the same file descriptor?
> >
> > No, I say this is braindamaged design, because you will get lots of
> > spurious wakeups. But it should work.
> 
> And to get it working, you have to be sure to put all the file
> descriptors in non-blocking mode, and be ready for the read/recv
> operation to fail with errno set to -EWOULDBLOCK, because if two threads
> wake up because select signaled that the same file descriptor is ready,
> one thread will win the race, and read will succeed, the other will not.
> If the file descriptor is not in non-blocking mode, the second read will
> block its caller.
> 
> --
>                                             Gilles.
> 
> This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@domain.hid and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by NDS for employment and security purposes.
> To protect the environment please do not print this e-mail unless necessary.
> 
> An NDS Group Limited company. www.nds.com
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help


-- 
Philippe.