[Xenomai-help] How to implement a GUI for RT apps?

[Xenomai-help] How to implement a GUI for RT apps?

[Vincent Levesque]

Hello,

I'd like to have your opinion on the best way to implement a GUI for a 
RT application. The standard approach seems to be to split of the 
application in two processes that communicate using shared memory or 
queues. I've been using a different approach that's closer to what's 
done in many non-realtime apps. I tried to put both the GUI and RT code 
in a single process using threads, shared objects and mutexes. It seemed 
to work fairly well for a while but it stopped working after switching 
to GCC 4.0 (while upgrading Ubuntu Linux). After spending a few days 
debugging this problem, I get the feeling that I'm doing something 
"illegal" with RTAI/fusion that worked by accident until the upgrade.

Here's what I've been doing... I create some objects, each with its own 
RT_MUTEX. I then create two RT tasks: one for the GUI and one for my 
hardware. Both taks have pointers to the shared objects. They call the 
objects' methods directly. The object locks or tries to lock its mutex 
in its methods, as appropriate. Both tasks are thus using the same 
RT_MUTEX structure. I've been careful to lock the mutexes only for short 
amounts of time and to only try to lock it in critical code.

To summarize,

1. What's the best way to implement a GUI for a RT app?
2. Is my approach "legal" in fusion/xenomai?
3. I remember reading that RTAI and X don't play well together. Is that 
the case with Xenomai? Will things be better if I use the two-processes 
approach?

Thank you,

Vincent Levesque
vleves@domain.hid

P.S. I'll be upgrading to Xenomai 2.0 asap. Thanks for your great work!

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2263 bytes --]

Vincent Levesque wrote:
> Hello,
> 
> I'd like to have your opinion on the best way to implement a GUI for a
> RT application. The standard approach seems to be to split of the
> application in two processes that communicate using shared memory or
> queues. I've been using a different approach that's closer to what's
> done in many non-realtime apps. I tried to put both the GUI and RT code
> in a single process using threads, shared objects and mutexes. It seemed
> to work fairly well for a while but it stopped working after switching
> to GCC 4.0 (while upgrading Ubuntu Linux). After spending a few days
> debugging this problem, I get the feeling that I'm doing something
> "illegal" with RTAI/fusion that worked by accident until the upgrade.
> 
> Here's what I've been doing... I create some objects, each with its own
> RT_MUTEX. I then create two RT tasks: one for the GUI and one for my
> hardware. Both taks have pointers to the shared objects. They call the
> objects' methods directly. The object locks or tries to lock its mutex
> in its methods, as appropriate. Both tasks are thus using the same
> RT_MUTEX structure. I've been careful to lock the mutexes only for short
> amounts of time and to only try to lock it in critical code.

...and hopefully do not call any non-rt functions while holding the
lock. Note that you can install a SIGXCPU handler to catch such
potential accidental violations. Likely not needed if you only copy some
data between the shared objects and your local buffers (all locked into
memory - mlockall?).

> 
> To summarize,
> 
> 1. What's the best way to implement a GUI for a RT app?

Depends too much on your requirements, GUI toolkit, etc. to give a
general statement here.

> 2. Is my approach "legal" in fusion/xenomai?

Sounds sane to me. Which fusion version are you using? Already tried to
update some component (kernel, patch, fusion/xenomai) and watch
potential changes?

> 3. I remember reading that RTAI and X don't play well together. Is that
> the case with Xenomai? Will things be better if I use the two-processes
> approach?

Not a general problem, but a specific one related to certain graphic
cards and drivers (don't ask me which, we are only using head-less or
console-based boxes here).

Jan

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Vincent Levesque]

Hi Jan,

>...and hopefully do not call any non-rt functions while holding the
>lock. Note that you can install a SIGXCPU handler to catch such
>potential accidental violations. Likely not needed if you only copy some
>data between the shared objects and your local buffers (all locked into
>memory - mlockall?).
>  
>
What do you mean by non-rt function? System calls? Functions that take a 
lot of time to run? Unless I made some mistakes, I'm only doing simple 
computations and copying data while mutexes are locked. I'm also using 
mlockall().

What would be the effect otherwise? I assume it would slow things down 
(by taking too long or switching to secondary mode) and possibly starve 
the rest of the system (freeze). I'm currently getting segfaults...

>>To summarize,
>>
>>1. What's the best way to implement a GUI for a RT app?
>>    
>>
>
>Depends too much on your requirements, GUI toolkit, etc. to give a
>general statement here.
>  
>
Good point. I guess the real question is: what are my options?

A few words about my needs. I'm a grad student in a research lab 
specializing in experimental haptic devices (computer interfaces for the 
sense of touch). I've been writing RT software to control different 
pieces of hardware. The software is for the most part for prototypes and 
will never leave the lab. The specs are constantly changing and I need 
to be able to continuously refactor and extend my code. My GUIs are 
mainly used to modify parameters and determine what goes on in RT 
control loops. I also give feedback with widgets and opengl drawings.

I could use the traditional decoupled approach (two processes) but that 
imposes some limits on what can be done fast. It essentially means that 
I need to design some kind of communication protocol between my RT and 
"non-RT" processes using queues or shared  memory. It's not that 
difficult but it complicates things and discourages changes to the 
software once finished. This approach can also quickly lead to very ugly 
hacks and workarounds when you can't afford to spend much time 
refactoring. It's not uncommon to see huge "do-it-all" structures with 
obsolete fields being passed around to save development time.

The approach that I'm using is more risky but it has many advantages 
(if/when it works). After all, there's a reason why multithreaded apps 
and frameworks like CORBA are popular...

Some information about my development env.:

OS: Ubuntu Linux 5.04 / 5.10
Kernel: 2.6.13 with ipipe adeos patch
RT: RTAI/fusion 0.9
Language: C++
GUI: gtkmm and gtkglextmm

>>2. Is my approach "legal" in fusion/xenomai?
>>    
>>
>
>Sounds sane to me. Which fusion version are you using? Already tried to
>update some component (kernel, patch, fusion/xenomai) and watch
>potential changes?
>  
>
Just to be sure we're clear: I have two tasks reusing the same RT_MUTEX 
structure, not two structure with the same content. That is allowed?

I didn't try to change much at this point. In fact I had to go back to 
my old linux distribution for the time being. I was using RTAI/fusion 
0.9. I switched to Xenomai 2.0 last night.

>>3. I remember reading that RTAI and X don't play well together. Is that
>>the case with Xenomai? Will things be better if I use the two-processes
>>approach?
>>    
>>
>
>Not a general problem, but a specific one related to certain graphic
>cards and drivers (don't ask me which, we are only using head-less or
>console-based boxes here).
>
That's good to know. I think I figured out why my program was sometimes 
freezing *before* the upgrade. It was unrelated to any problems with X.

Another question: Is there any restrictions on the use of the "new" 
operator in a RT task (or after calling mlockall)? That's the other 
possible culprit I had in mind....

Thanks a lot for your help!

Vincent Levesque
vleves@domain.hid

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 4979 bytes --]

Vincent Levesque wrote:
> Hi Jan,
> 
>> ...and hopefully do not call any non-rt functions while holding the
>> lock. Note that you can install a SIGXCPU handler to catch such
>> potential accidental violations. Likely not needed if you only copy some
>> data between the shared objects and your local buffers (all locked into
>> memory - mlockall?).
>>  
>>
> What do you mean by non-rt function? System calls? Functions that take a

Specifically Linux system calls (hidden mallocs e.g.), as they can
trigger sporadic latencies. Long running pure userspace code is no
problem as long as your system design is prepared for it.

> lot of time to run? Unless I made some mistakes, I'm only doing simple
> computations and copying data while mutexes are locked. I'm also using
> mlockall().
> 
> What would be the effect otherwise? I assume it would slow things down
> (by taking too long or switching to secondary mode) and possibly starve
> the rest of the system (freeze). I'm currently getting segfaults...

A segfaults or freezing are not the reactions one has to expect on
breaking the real-time context with system calls etc. It may be a result
when timing assumptions suddenly do not match any more and the own code
breaks somehow.

> 
>>> To summarize,
>>>
>>> 1. What's the best way to implement a GUI for a RT app?
>>>   
>>
>>
>> Depends too much on your requirements, GUI toolkit, etc. to give a
>> general statement here.
>>  
>>
> Good point. I guess the real question is: what are my options?
> 
> A few words about my needs. I'm a grad student in a research lab
> specializing in experimental haptic devices (computer interfaces for the
> sense of touch). I've been writing RT software to control different
> pieces of hardware. The software is for the most part for prototypes and
> will never leave the lab. The specs are constantly changing and I need
> to be able to continuously refactor and extend my code. My GUIs are
> mainly used to modify parameters and determine what goes on in RT
> control loops. I also give feedback with widgets and opengl drawings.
> 
> I could use the traditional decoupled approach (two processes) but that
> imposes some limits on what can be done fast. It essentially means that
> I need to design some kind of communication protocol between my RT and
> "non-RT" processes using queues or shared  memory. It's not that
> difficult but it complicates things and discourages changes to the
> software once finished. This approach can also quickly lead to very ugly
> hacks and workarounds when you can't afford to spend much time
> refactoring. It's not uncommon to see huge "do-it-all" structures with
> obsolete fields being passed around to save development time.

Ack.

> 
> The approach that I'm using is more risky but it has many advantages
> (if/when it works). After all, there's a reason why multithreaded apps
> and frameworks like CORBA are popular...
> 
> Some information about my development env.:
> 
> OS: Ubuntu Linux 5.04 / 5.10
> Kernel: 2.6.13 with ipipe adeos patch
> RT: RTAI/fusion 0.9
> Language: C++
> GUI: gtkmm and gtkglextmm
> 
>>> 2. Is my approach "legal" in fusion/xenomai?
>>>   
>>
>>
>> Sounds sane to me. Which fusion version are you using? Already tried to
>> update some component (kernel, patch, fusion/xenomai) and watch
>> potential changes?
>>  
>>
> Just to be sure we're clear: I have two tasks reusing the same RT_MUTEX
> structure, not two structure with the same content. That is allowed?

Has the same effect. RT_MUTEX effectively contains a handle to the
related kernel object (it's the point - but don't tell it anyone else
;)). As long as the content is identical you can duplicate or share
RT_MUTEX as you like to.

> 
> I didn't try to change much at this point. In fact I had to go back to
> my old linux distribution for the time being. I was using RTAI/fusion
> 0.9. I switched to Xenomai 2.0 last night.
> 
>>> 3. I remember reading that RTAI and X don't play well together. Is that
>>> the case with Xenomai? Will things be better if I use the two-processes
>>> approach?
>>>   
>>
>>
>> Not a general problem, but a specific one related to certain graphic
>> cards and drivers (don't ask me which, we are only using head-less or
>> console-based boxes here).
>>
> That's good to know. I think I figured out why my program was sometimes
> freezing *before* the upgrade. It was unrelated to any problems with X.
> 
> Another question: Is there any restrictions on the use of the "new"
> operator in a RT task (or after calling mlockall)? That's the other
> possible culprit I had in mind....

Ha! The old problem with C++ in RT contexts ;). It IS a problem as long
as you do not overload it and control it's actual behaviour. Otherwise,
it's nondeterministic, e.i. it may also call the kernel and ask for more
space...

Generic advice: pre-allocate your objects or put temporal ones on the
stack. Use pools of pre-allocated objects if you really depend on some
dynamic.

Jan

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Ignacio García Pérez]

> 1. What's the best way to implement a GUI for a RT app?

Make it non-RT. The "U" in "GUI" is user. Users are not real time.

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 295 bytes --]

Ignacio García Pérez wrote:
>>1. What's the best way to implement a GUI for a RT app?
> 
> 
> Make it non-RT. The "U" in "GUI" is user. Users are not real time.
> 

Not true. That thread must be mapped to Xenomai in order to use
rt-mutexes. Otherwise, one has to use rt-pipes.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Ignacio García Pérez]

Jan Kiszka wrote:

>Ignacio García Pérez wrote:
>  
>
>>>1. What's the best way to implement a GUI for a RT app?
>>>      
>>>
>>Make it non-RT. The "U" in "GUI" is user. Users are not real time.
>>
>>    
>>
>
>Not true. That thread must be mapped to Xenomai in order to use
>rt-mutexes. Otherwise, one has to use rt-pipes.
>
>Jan
>
>  
>
I was talking design. Anything that requires user interaction is by
definition non-realtime, so,  if you have the choice to leave it out of
the RT domain, do it. You'll save a lot of headaches.

Of course, if your GUI is so tightly coupled to your RT processing that
you cannot do without rt_mutexes and/or other sync primitives, then you
must map it to Xenomai.

Nacho.

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2211 bytes --]

Ignacio García Pérez wrote:
> Jan Kiszka wrote:
> 
> 
>>Ignacio García Pérez wrote:
>> 
>>
>>
>>>>1. What's the best way to implement a GUI for a RT app?
>>>>     
>>>>
>>>
>>>Make it non-RT. The "U" in "GUI" is user. Users are not real time.
>>>
>>>   
>>>
>>
>>Not true. That thread must be mapped to Xenomai in order to use
>>rt-mutexes. Otherwise, one has to use rt-pipes.
>>
>>Jan
>>
>> 
>>
> 
> I was talking design. Anything that requires user interaction is by
> definition non-realtime, so,  if you have the choice to leave it out of

Do you happen to design ATMs as well? They interact with me, a user, but
also not in real-time. I typically have to type my PIN at least two
times to get all digits recognised. ;)

> the RT domain, do it. You'll save a lot of headaches.
> 
> Of course, if your GUI is so tightly coupled to your RT processing that
> you cannot do without rt_mutexes and/or other sync primitives, then you
> must map it to Xenomai.

Basically, we are discussing here where to put the "bridge" between RT
and non-RT. Some instance in your system has to be a bridge which
switches between both worlds in order to get data in and out of the RT
domain. Such bridge can be a rt-pipe (and related variants) or an
explicitly coded version at application layer. I don't see why it should
be bad design or cause troubles to create Xenomai RT threads like this:

while (!finished) {
    do_a_lot_of_non-rt_stuff;

    rt_mutex_lock();
    copy_data_between_rt_and_local_buffers;
    rt_mutex_unlock();

    do_more_non_rt_stuff;
}

Only when the code between rt_mutex_lock and unlock gets too complicated
and intransparent, your design needs a revision.

Ok, an option that does not require context migrations is lock-less
synchronisation. For example, one may maintain revision counters of data
structures which get updated on changes by the RT part. The non-RT part
can then spin on reading a consistent version until the revision on
entry equals the one on exit. And double-buffering can be used to write
data structures into the RT domain. But this can be more complex and is
not feasible in each and every scenario.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Vincent Levesque]

Hi Jan and Ignacio,

>>>Not true. That thread must be mapped to Xenomai in order to use
>>>rt-mutexes. Otherwise, one has to use rt-pipes.
>>>      
>>>
That's the impression I had. Is there anything I should do in the GUI 
task to make it closer to a non-RT thread? Is there any way, for 
example, to make it remain in secondary mode? I already set the task to 
a low priority.

>>the RT domain, do it. You'll save a lot of headaches.
>>
>>Of course, if your GUI is so tightly coupled to your RT processing that
>>you cannot do without rt_mutexes and/or other sync primitives, then you
>>must map it to Xenomai.
>>    
>>
As I'm explaining in my previous post, I could decouple the GUI from the 
RT side but the tight coupling has its advantages for my type of 
application.

>Ok, an option that does not require context migrations is lock-less
>synchronisation. For example, one may maintain revision counters of data
>structures which get updated on changes by the RT part. The non-RT part
>can then spin on reading a consistent version until the revision on
>entry equals the one on exit. And double-buffering can be used to write
>data structures into the RT domain. But this can be more complex and is
>not feasible in each and every scenario.
>  
>
That's very interesting. I'm not 100% clear on what you mean though. If 
I understand correctly, I could have a single RT task and a normal 
thread (or the main thread) for the GUI. I could then use the fact that 
the RT task cannot be interupted by the GUI thread (?) to make sure data 
structures do not get corrupted. Do I get this right?

Thanks again,

Vincent Levesque
vleves@domain.hid

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2198 bytes --]

Vincent Levesque wrote:
> Hi Jan and Ignacio,
>
>>>> Not true. That thread must be mapped to Xenomai in order to use
>>>> rt-mutexes. Otherwise, one has to use rt-pipes.
>>>>
>
> That's the impression I had. Is there anything I should do in the GUI
> task to make it closer to a non-RT thread? Is there any way, for
> example, to make it remain in secondary mode? I already set the task to
> a low priority.

You can explicitly switch the mode (rt_task_set_mode), but this is
generally not needed.

>> Ok, an option that does not require context migrations is lock-less
>> synchronisation. For example, one may maintain revision counters of data
>> structures which get updated on changes by the RT part. The non-RT part
>> can then spin on reading a consistent version until the revision on
>> entry equals the one on exit. And double-buffering can be used to write
>> data structures into the RT domain. But this can be more complex and is
>> not feasible in each and every scenario.
>>
>>
> That's very interesting. I'm not 100% clear on what you mean though. If
> I understand correctly, I could have a single RT task and a normal
> thread (or the main thread) for the GUI. I could then use the fact that
> the RT task cannot be interupted by the GUI thread (?) to make sure data
> structures do not get corrupted. Do I get this right?

Partially. On a single-CPU system, you can make use of the fact that
RT-tasks can always complete their modifications before a non-RT-task
gets the CPU again. But this is a fragile approach if you think of
upcoming dual-core systems.

For common systems it is safer to track modifications based on some
revision counter the changing instance increments when it touches data.
The interrupted reader can than check if things changed by 1. saving the
counter, 2. reading the data, 3. comparing the counter against the
original value. If both value are the same, the read data is consistent.
Otherwise the reader has to retry. Note that this approach implies that
the read data is never in a state which can break the reader
(inconsistent pointers e.g.).

But I must confess we also once made use of the UP characteristic for
some real-time application...

Jan

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [Xenomai-help] How to implement a GUI for RT apps?

[Hannes Mayer]

Hi all!

I must admit that I've done only RT stuff without GUI so far (well,
one with ncurses, but I guess that one doesn't count here ;-)

Just to get started - instead of theoretical talk only - it'll
be great if the one or another long time list-member could post
an examples to get the feet of the non-GUI folks wet.

Thanks a bunch :-)

Best regards,
Hannes.