[Xenomai-help] Communication between RT and non-RT thread

[Xenomai-help] Communication between RT and non-RT thread

[Andrey Nechypurenko]

Hi Folks,

In my application I have Xenomai thread which collects and process
sensor data. In addition, I have non-RT thread which sends sensor data
over the network for visualization (with much lower time resolution,
picking let's say every 100th sample). Xenomai thread is the only one
which access sensors so there is a need to pass some samples to the
non-RT communication thread. Basically, It is a typical
producer/consumer situation which could be solved with thread-safe
queue. Obviously, introducing such queue with standard synchronization
mechanisms such as mutexes, etc. is not an option since it will
influence the RT thread which will be forced to wait for non-RT
thread.

While searching for the solution, I came to conclusion that lock-free
[1] queue implementation is the way to go. Searching for such
implementation which is also mature enough for real-life applications
and works on ARM, the only one I found was boost_lockfree [2] . It is
a great library, but it depends on boost, which in turns by default
pulls Python and a lot of other things which are not necessary
required on the embedded system (I am using BeagleBoard with Anstrom
Linux distribution).

I think, the situation I have (RT supplier/non-RT consumer) is quite
typical and I hope there should be a typical good solution for it. So
the question I would like to ask is whether someone already solved
such a problem and would appreciate any hints for good implementation
strategy.

[1] http://en.wikipedia.org/wiki/Non-blocking_algorithm
[2] http://tim.klingt.org/boost_lockfree

Thank you,
Andrey.

Re: [Xenomai-help] Communication between RT and non-RT thread

[Gilles Chanteperdrix]

On 07/06/2011 11:25 AM, Andrey Nechypurenko wrote:
> Hi Folks,
> 
> In my application I have Xenomai thread which collects and process
> sensor data. In addition, I have non-RT thread which sends sensor data
> over the network for visualization (with much lower time resolution,
> picking let's say every 100th sample). Xenomai thread is the only one
> which access sensors so there is a need to pass some samples to the
> non-RT communication thread. Basically, It is a typical
> producer/consumer situation which could be solved with thread-safe
> queue. Obviously, introducing such queue with standard synchronization
> mechanisms such as mutexes, etc. is not an option since it will
> influence the RT thread which will be forced to wait for non-RT
> thread.

You can use mutexes with priority inheritance to avoid this issue. But 
Xenomai has ready made queues for rt/non-rt communication, the rtipcs:

http://www.xenomai.org/documentation/xenomai-head/html/api/group__rtipc.html



> While searching for the solution, I came to conclusion that lock-free
> [1] queue implementation is the way to go. Searching for such
> implementation which is also mature enough for real-life applications
> and works on ARM, the only one I found was boost_lockfree [2] . It is
> a great library, but it depends on boost, which in turns by default
> pulls Python and a lot of other things which are not necessary
> required on the embedded system (I am using BeagleBoard with Anstrom
> Linux distribution).
> 
> I think, the situation I have (RT supplier/non-RT consumer) is quite
> typical and I hope there should be a typical good solution for it.

A lockless solution for this (one consumer, one producer) is the good 
old fifo.

-- 
                                                                Gilles.

Re: [Xenomai-help] Communication between RT and non-RT thread

[Andrey Nechypurenko]

Thanks Gilles for the quick response!

> You can use mutexes with priority inheritance to avoid this issue. But
> Xenomai has ready made queues for rt/non-rt communication, the rtipcs:
>
> http://www.xenomai.org/documentation/xenomai-head/html/api/group__rtipc.html

What makes me worry here is the IPC abbreviation - I have just one
process with multiple threads. So would not real IPC mechanism be the
overkill in this scenario? Or am I just misinterpret what IPC means
here?

> A lockless solution for this (one consumer, one producer) is the good
> old fifo.

I have one producer, but do not want to limit myself with just one
consumer (although currently there is only one). In addition, here I
am probably again confused by the terminology, but does not fifo you
mentioned here is another IPC mechanism? What I am seeking is the way
to communicate between two threads and benefit from the fact that they
share the same process memory.


Thanks,
Andrey.

Re: [Xenomai-help] Communication between RT and non-RT thread

[Gilles Chanteperdrix]

On 07/06/2011 01:42 PM, Andrey Nechypurenko wrote:
> Thanks Gilles for the quick response!
> 
>> You can use mutexes with priority inheritance to avoid this issue. But
>> Xenomai has ready made queues for rt/non-rt communication, the rtipcs:
>>
>> http://www.xenomai.org/documentation/xenomai-head/html/api/group__rtipc.html
> 
> What makes me worry here is the IPC abbreviation - I have just one
> process with multiple threads. So would not real IPC mechanism be the
> overkill in this scenario? Or am I just misinterpret what IPC means
> here?

Not really overkill. Data will be copied from and to an intermediate
buffer. But if you send pointers to data, these copies will simply
involve copying the pointer twice.

I would say it is lockless mechanism which are overkill. Because
usually, lockless mechanisms have a much more complicated
implementation, and so are much more prone to bugs. All this for what?
To avoid a small critical section?

> 
>> A lockless solution for this (one consumer, one producer) is the good
>> old fifo.
> 
> I have one producer, but do not want to limit myself with just one
> consumer (although currently there is only one). In addition, here I
> am probably again confused by the terminology, but does not fifo you
> mentioned here is another IPC mechanism? What I am seeking is the way
> to communicate between two threads and benefit from the fact that they
> share the same process memory.

I am talking about the ring buffer thing with head and tail pointers
where each thread (consumer, producer), moves only one pointer. I would
call this a "lockless fifo", though I do not know what the official name
is, but you get the idea. Again, you can send pointers through an IPC
instead of sending the data themselves, and you will benefit from the
fact that the two threads are running in the same memory space.

-- 
					    Gilles.

Re: [Xenomai-help] Communication between RT and non-RT thread

[Richard Cochran]

On Wed, Jul 06, 2011 at 02:19:47PM +0200, Gilles Chanteperdrix wrote:
> On 07/06/2011 01:42 PM, Andrey Nechypurenko wrote:
> > 
> > What makes me worry here is the IPC abbreviation - I have just one
> > process with multiple threads. So would not real IPC mechanism be the
> > overkill in this scenario? Or am I just misinterpret what IPC means
> > here?

...

> 
> I am talking about the ring buffer thing with head and tail pointers
> where each thread (consumer, producer), moves only one pointer. I would
> call this a "lockless fifo", though I do not know what the official name
> is, but you get the idea. Again, you can send pointers through an IPC
> instead of sending the data themselves, and you will benefit from the
> fact that the two threads are running in the same memory space.

In Andrey's case, I would simply use a ring buffer (fifo) shared
between one RT and one non-RT thread. No locking is needed, and it is
easy to implement. You don't need a library for that.

HTH,

Richard

Re: [Xenomai-help] Communication between RT and non-RT thread

[Anders Blomdell]

On 07/07/2011 09:31 AM, Richard Cochran wrote:
> On Wed, Jul 06, 2011 at 02:19:47PM +0200, Gilles Chanteperdrix wrote:
>> On 07/06/2011 01:42 PM, Andrey Nechypurenko wrote:
>>>
>>> What makes me worry here is the IPC abbreviation - I have just one
>>> process with multiple threads. So would not real IPC mechanism be the
>>> overkill in this scenario? Or am I just misinterpret what IPC means
>>> here?
>
> ...
>
>>
>> I am talking about the ring buffer thing with head and tail pointers
>> where each thread (consumer, producer), moves only one pointer. I would
>> call this a "lockless fifo", though I do not know what the official name
>> is, but you get the idea. Again, you can send pointers through an IPC
>> instead of sending the data themselves, and you will benefit from the
>> fact that the two threads are running in the same memory space.
>
> In Andrey's case, I would simply use a ring buffer (fifo) shared
> between one RT and one non-RT thread. No locking is needed, and it is
> easy to implement. You don't need a library for that.
If you want to do lockless passing of data, make sure that you have read 
and understood linux/Documentation/memory-barriers.txt

/Anders


-- 
Anders Blomdell                  Email: anders.blomdell@domain.hid
Department of Automatic Control
Lund University                  Phone:    +46 46 222 4625
P.O. Box 118                     Fax:      +46 46 138118
SE-221 00 Lund, Sweden

Re: [Xenomai-help] Communication between RT and non-RT thread

[Andrey Nechypurenko]

Thank you very much guys for suggestions!
The idea with simple ring buffer looks promising and I will try to
implement it this way.

Once again thank you!
Andrey.