[Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Guvenc Gulce]

Hello

I have the following questions regarding the rt_pipe behavior in Xenomai. 

-> Why is not possible to use rt_pipe_monitor() Native API call from an userspace RT Task ?

-> What is the solution if async notifications in rt-pipe context are needed in userspace RT Tasks ?

-> Is there any plan to implement something similar to linux's epoll system call for the rt-pipes used 
in userspace RT tasks ?

Thanks & Regards

Guvenc


      

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Philippe Gerum]

On Tue, 2009-06-16 at 15:23 -0700, Guvenc Gulce wrote: 
> Hello
> 
> I have the following questions regarding the rt_pipe behavior in Xenomai. 
> 
> -> Why is not possible to use rt_pipe_monitor() Native API call from an userspace RT Task ?

Because this API is aimed at providing help to drivers implementing the
kernel space endpoint of a pipe connection; the driver/module may then
make userland benefit from this in a way or another, but it is not the
primary intent.

Additionally, the pipe monitors are based on function calls for
notification, which is a real issue in a dual kernel system, when the
caller is not the linux kernel, but the co-kernel. We would have to
reinstate the userland address space context as fast as possible just
for the time needed to run the handler, and we would have to do so from
a real-time context that could be utterly unsafe from the linux kernel
POV.

There is another option we have been digging for some time to emulate
such kind of callouts using an asynchronous approach under the hood,
that would allow us to support real-time signals crossing address spaces
as well. But this is still on the drawing board.

> 
> -> What is the solution if async notifications in rt-pipe context are needed in userspace RT Tasks ?
> 
> 

If you don't care about real-timeliness of the receiver (i.e. userland
does read() and not rt_pipe_read/receive()), then SIGIO is available,
but maybe this is not what you want if your receiver is actually a RT
task. Otherwise, you will have to resort to a server task reading the
pipe to emulate an asynchronous behavior out of a synchronous construct,
I'm afraid.

As you have probably understood already, building a full real-time to
real-time data path using pipes is not possible; this said, this is not
the purpose of this API anyway, which has been designed for real-time to
non-RT communication.

> 
> 
> -> Is there any plan to implement something similar to linux's epoll system call for the rt-pipes used 
> in userspace RT tasks ?
> 

No plans yet, even if not objection to merge this either.

> Thanks & Regards
> 
> Guvenc
> 
> 
>       
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
-- 
Philippe.

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Gilles Chanteperdrix]

Philippe Gerum wrote:
> On Tue, 2009-06-16 at 15:23 -0700, Guvenc Gulce wrote: 
>> Hello
>>
>> I have the following questions regarding the rt_pipe behavior in Xenomai. 
>>
>> -> Why is not possible to use rt_pipe_monitor() Native API call from an userspace RT Task ?
> 
> Because this API is aimed at providing help to drivers implementing the
> kernel space endpoint of a pipe connection; the driver/module may then
> make userland benefit from this in a way or another, but it is not the
> primary intent.
> 
> Additionally, the pipe monitors are based on function calls for
> notification, which is a real issue in a dual kernel system, when the
> caller is not the linux kernel, but the co-kernel. We would have to
> reinstate the userland address space context as fast as possible just
> for the time needed to run the handler, and we would have to do so from
> a real-time context that could be utterly unsafe from the linux kernel
> POV.
> 
> There is another option we have been digging for some time to emulate
> such kind of callouts using an asynchronous approach under the hood,
> that would allow us to support real-time signals crossing address spaces
> as well. But this is still on the drawing board.
> 
>> -> What is the solution if async notifications in rt-pipe context are needed in userspace RT Tasks ?
>>
>>
> 
> If you don't care about real-timeliness of the receiver (i.e. userland
> does read() and not rt_pipe_read/receive()), then SIGIO is available,
> but maybe this is not what you want if your receiver is actually a RT
> task. Otherwise, you will have to resort to a server task reading the
> pipe to emulate an asynchronous behavior out of a synchronous construct,
> I'm afraid.
> 
> As you have probably understood already, building a full real-time to
> real-time data path using pipes is not possible; this said, this is not
> the purpose of this API anyway, which has been designed for real-time to
> non-RT communication.
> 
>>
>> -> Is there any plan to implement something similar to linux's epoll system call for the rt-pipes used 
>> in userspace RT tasks ?
>>
> 
> No plans yet, even if not objection to merge this either.

Actually, we already have xnselect, which could be used to implement
this. But we would have to implement a native interface using it, and
modify the rt-pipes to plug into the xnselect API (this has already been
done, but not merged into head).

-- 
                                          Gilles

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Peter Soetens]

On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
>
> As you have probably understood already, building a full real-time to
> real-time data path using pipes is not possible; this said, this is not
> the purpose of this API anyway, which has been designed for real-time to
> non-RT communication.

I was hoping to use rt_pipe + select in real-time context to implement
a data receiving server for real-time inter-process communication. Is
this possible ?
What would happen if the real-time clients open the pipes as rt_pipes
and start sending
data in ? What's the alternative to listen in real-time to many
connections from
a single thread ?

Peter

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Gilles Chanteperdrix]

Peter Soetens wrote:
> On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
>> As you have probably understood already, building a full real-time to
>> real-time data path using pipes is not possible; this said, this is not
>> the purpose of this API anyway, which has been designed for real-time to
>> non-RT communication.
> 
> I was hoping to use rt_pipe + select in real-time context to implement
> a data receiving server for real-time inter-process communication. Is
> this possible ?

No, currently, select works with posix skin descriptors and RTDM
descriptors, and rt-pipes is a native skin interface, both were not
meant to be mixed.

So, there are several ways to fix this:
- add a mapping of the xnpipe API to the posix skin (or RTDM), I do not
really know how to do this, probably the best thing would be a special
kind of socketpair, which returns both descriptors;
- add a mapping of the xnselect API to the native skin, this would allow
us the freedom to implement an interface without the long scan of the
fd_set in application which scales badly, though scalability may not be
an issue with select due to the limit of 1024 descriptors;
- have a great unified file descriptor support, which would allow to mix
interfaces.

> What would happen if the real-time clients open the pipes as rt_pipes
> and start sending
> data in ? What's the alternative to listen in real-time to many
> connections from
> a single thread ?
> 
> Peter
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help


-- 
                                          Gilles

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Gilles Chanteperdrix]

Peter Soetens wrote:
> On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
>> As you have probably understood already, building a full real-time to
>> real-time data path using pipes is not possible; this said, this is not
>> the purpose of this API anyway, which has been designed for real-time to
>> non-RT communication.
> 
> I was hoping to use rt_pipe + select in real-time context to implement
> a data receiving server for real-time inter-process communication. Is
> this possible ?
> What would happen if the real-time clients open the pipes as rt_pipes
> and start sending
> data in ? What's the alternative to listen in real-time to many
> connections from
> a single thread ?

Note that posix message queues already work for inter-process
communications, with support for select. Depending on your needs, this
may be sufficient: you will need Xenomai threads on both sides, but the
non real-time one may use the SCHED_OTHER policy.

I was thinking, maybe we could map the xnpipe to a special flag in mq_open ?

-- 
                                          Gilles

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Peter Soetens]

On Wed, Jun 17, 2009 at 12:30, Gilles
Chanteperdrix<gilles.chanteperdrix@xenomai.org> wrote:
> Peter Soetens wrote:
>> On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
>>> As you have probably understood already, building a full real-time to
>>> real-time data path using pipes is not possible; this said, this is not
>>> the purpose of this API anyway, which has been designed for real-time to
>>> non-RT communication.
>>
>> I was hoping to use rt_pipe + select in real-time context to implement
>> a data receiving server for real-time inter-process communication. Is
>> this possible ?
>> What would happen if the real-time clients open the pipes as rt_pipes
>> and start sending
>> data in ? What's the alternative to listen in real-time to many
>> connections from
>> a single thread ?
>
> Note that posix message queues already work for inter-process
> communications, with support for select. Depending on your needs, this
> may be sufficient: you will need Xenomai threads on both sides, but the
> non real-time one may use the SCHED_OTHER policy.

Thanks, I missed that. I thought that the mqd_t would not be
compatible with file descriptors required by select. I think I can
settle with Xenomai<->Xenomai only communication, since it's all
running on the same host and our library initialises every thread as a
Xenomai native task. It was a bit confusing which ipc primitive to use
in which situation.  We're still looking for the Holy grale (Any<->Any
+ select), but we're getting close :-)

>
> I was thinking, maybe we could map the xnpipe to a special flag in mq_open ?

If POSIX compatibility is put up front, I don't think this will happen ?

Peter

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Gilles Chanteperdrix]

Peter Soetens wrote:
> On Wed, Jun 17, 2009 at 12:30, Gilles
> Chanteperdrix<gilles.chanteperdrix@xenomai.org> wrote:
>> Peter Soetens wrote:
>>> On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
>>>> As you have probably understood already, building a full real-time to
>>>> real-time data path using pipes is not possible; this said, this is not
>>>> the purpose of this API anyway, which has been designed for real-time to
>>>> non-RT communication.
>>> I was hoping to use rt_pipe + select in real-time context to implement
>>> a data receiving server for real-time inter-process communication. Is
>>> this possible ?
>>> What would happen if the real-time clients open the pipes as rt_pipes
>>> and start sending
>>> data in ? What's the alternative to listen in real-time to many
>>> connections from
>>> a single thread ?
>> Note that posix message queues already work for inter-process
>> communications, with support for select. Depending on your needs, this
>> may be sufficient: you will need Xenomai threads on both sides, but the
>> non real-time one may use the SCHED_OTHER policy.
> 
> Thanks, I missed that. I thought that the mqd_t would not be
> compatible with file descriptors required by select. I think I can
> settle with Xenomai<->Xenomai only communication, since it's all
> running on the same host and our library initialises every thread as a
> Xenomai native task. It was a bit confusing which ipc primitive to use
> in which situation.  We're still looking for the Holy grale (Any<->Any
> + select), but we're getting close :-)
> 
>> I was thinking, maybe we could map the xnpipe to a special flag in mq_open ?
> 
> If POSIX compatibility is put up front, I don't think this will happen ?

The problem is that the only "POSIX way" I see to add a posix mapping
for the xnpipe API, is to add a non portable extension somewhere. It
could be a non portable message queues flag, or a non-portable socket
protocol. The problem with the socket protocol, is that we would open
one pipe end with the socket() call, and the other one with
open("/dev/rtpipe42"), which is a little bit cumbersome and why I
suggested to use socketpair instead, which would hide the call to open.
On the other hand, the same application does not necessarily need to
access the two sides of the pipe.

As for the holy grail you are looking for, message queues have
any<->any, but do not allow non real-time select. A mapping of xnpipe
would be asymetric, but would allow non real-time select on the non
real-time side, and real-time select on the real-time side.

By the way, I recently made a guide where I put some ideas on porting
POSIX applications to Xenomai, it mentions these issues:
http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai

-- 
                                          Gilles

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Peter Soetens]

On Wed, Jun 17, 2009 at 14:25, Gilles
Chanteperdrix<gilles.chanteperdrix@xenomai.org> wrote:
> Peter Soetens wrote:
>> On Wed, Jun 17, 2009 at 12:30, Gilles
>> Chanteperdrix<gilles.chanteperdrix@xenomai.org> wrote:
>>> Peter Soetens wrote:
>>>> On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
>>>>> As you have probably understood already, building a full real-time to
>>>>> real-time data path using pipes is not possible; this said, this is not
>>>>> the purpose of this API anyway, which has been designed for real-time to
>>>>> non-RT communication.
>>>> I was hoping to use rt_pipe + select in real-time context to implement
>>>> a data receiving server for real-time inter-process communication. Is
>>>> this possible ?
>>>> What would happen if the real-time clients open the pipes as rt_pipes
>>>> and start sending
>>>> data in ? What's the alternative to listen in real-time to many
>>>> connections from
>>>> a single thread ?
>>> Note that posix message queues already work for inter-process
>>> communications, with support for select. Depending on your needs, this
>>> may be sufficient: you will need Xenomai threads on both sides, but the
>>> non real-time one may use the SCHED_OTHER policy.
>>
>> Thanks, I missed that. I thought that the mqd_t would not be
>> compatible with file descriptors required by select. I think I can
>> settle with Xenomai<->Xenomai only communication, since it's all
>> running on the same host and our library initialises every thread as a
>> Xenomai native task. It was a bit confusing which ipc primitive to use
>> in which situation.  We're still looking for the Holy grale (Any<->Any
>> + select), but we're getting close :-)
>>
>>> I was thinking, maybe we could map the xnpipe to a special flag in mq_open ?
>>
>> If POSIX compatibility is put up front, I don't think this will happen ?
>
> The problem is that the only "POSIX way" I see to add a posix mapping
> for the xnpipe API, is to add a non portable extension somewhere. It
> could be a non portable message queues flag, or a non-portable socket
> protocol. The problem with the socket protocol, is that we would open
> one pipe end with the socket() call, and the other one with
> open("/dev/rtpipe42"), which is a little bit cumbersome and why I
> suggested to use socketpair instead, which would hide the call to open.
> On the other hand, the same application does not necessarily need to
> access the two sides of the pipe.

This is going 'slightly' over my head. I can understand the problem,
but not the solution. Probably this was meant for Philippe :-)

>
> As for the holy grail you are looking for, message queues have
> any<->any, but do not allow non real-time select. A mapping of xnpipe
> would be asymetric, but would allow non real-time select on the non
> real-time side, and real-time select on the real-time side.

I think I get it, but I'll start experimenting with some combinations
and see how I can get most out of it. As I said, I can live with a
Xeno<->Xeno + select.

>
> By the way, I recently made a guide where I put some ideas on porting
> POSIX applications to Xenomai, it mentions these issues:
> http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai

*very* well written. Thank you for refreshing these issues !

Peter

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Philippe Gerum]

On Wed, 2009-06-17 at 12:30 +0200, Gilles Chanteperdrix wrote:
> Peter Soetens wrote:
> > On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
> >> As you have probably understood already, building a full real-time to
> >> real-time data path using pipes is not possible; this said, this is not
> >> the purpose of this API anyway, which has been designed for real-time to
> >> non-RT communication.
> > 
> > I was hoping to use rt_pipe + select in real-time context to implement
> > a data receiving server for real-time inter-process communication. Is
> > this possible ?
> > What would happen if the real-time clients open the pipes as rt_pipes
> > and start sending
> > data in ? What's the alternative to listen in real-time to many
> > connections from
> > a single thread ?
> 
> Note that posix message queues already work for inter-process
> communications, with support for select. Depending on your needs, this
> may be sufficient: you will need Xenomai threads on both sides, but the
> non real-time one may use the SCHED_OTHER policy.
> 
> I was thinking, maybe we could map the xnpipe to a special flag in mq_open ?
> 

Albeit the message pipes are most of the time used in datagram mode, it
can also be used in byte stream mode, which would not fit the mq
semantics that nicely (I'm a bit afraid of potential overhead as well).

Actually, we do have a pivotal API we could use further, and this is
RTDM. We could have a set of socket-based RTDM drivers that implement
different RT communication domains for IPC, one emulating a real-time
socketpair for local IPC as you mentioned for instance, another one
would bear the message pipe semantics etc.

RTDM already complements all other APIs, so there would be no need to
extend those to get message pipes with select() support, and we could
even go as far as deprecating the existing message pipe API from the
native skin, at some point. Besides, all RTOS emulators we support work
fine side-by-side with either the POSIX or native skins, which in turn
do support the RTDM model. So basically, we write one IPC driver, it
works for the whole set of APIs we have, without any change. Maybe it's
time for ksrc/drivers/ipc or something?

-- 
Philippe.

Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

[Guvenc Gulce]

Hi all
Thanks for the replies so far. Maybe I should give some background info regarding my needs and why I have 
asked this question at the first place so that you guys may have some ideas how to solve such a situation 
in an elegant way by using the current APIs of Xenomai.

I am planning to develop a mechanism which allows me to use Linux Network Sockets(send/receive) from an RT Task 
running in linux userspace. (I am aware of the RTNet but I don't really need real timeliness and 
determinism on the wire.) What I need is that my RT Task in the userspace does not lose its real timeliness
at any cost. My Understanding is that calling Linux APIs (e.g. socket(), bind() etc.) from RT Tasks 
disturb the real timeliness of the RT task. 

Is there something already developed to meet such a need in Xenomai context ? 
If not, what would be the best way to approach such a problem ?

Here are few ideas I have :

* RT Task(Userspace) <---Connnected with RT Pipes---> Linux Task(Userspace, does the socket API calls)

The problem with this approach is that RT Pipe API is not fully available in userspace and ugly 
O(n) Algorithms should be used at some places. (Synchron/Serial pipe handling)

* RT Task(Userspace) <---Connected with Posix System Calls--> RTDM (Kernel Space) <-- RT Pipes --> 
Some other Kernel Module(Kernel space, does the Socket API calls)

I am not sure whether such an approach would work at all. Maybe RT Pipes are not necessary at all 
and RTDM can do the Linux Socket API calls, in other words, RTDM would wrap Socket API calls with 
some additional logic. Would such an approach disturb the real timeliness of the original caller RT Task? RT Task 
calls RTDM which then calls some linux stuff which may cause a switch to secondary domain ? 

There are also a lot of similar APIs in Native and Posix skins. Would it be correct to say that one 
of them can be randomly picked, if there aren't concerns of legacy code compatibility issues. They 
are somewhat confusing and it is not really clear which Skin in which case really makes sense. 
Native Skin has RT-Pipes and Queues and on the other hand Posix has only Queues. Can the Queues 
also be used for RT Task <--> Non-RT Task communication as in the case of RT Pipes ? 

Thanks in advance for the comments

Guvenc  






----- Original Message ----
From: Philippe Gerum <rpm@xenomai.org>
To: Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
Cc: xenomai@xenomai.org
Sent: Thursday, June 18, 2009 12:41:53 AM
Subject: Re: [Xenomai-help] Native API rt_pipe_monitor() call can not be called from an RT Task in linux userspace ?

On Wed, 2009-06-17 at 12:30 +0200, Gilles Chanteperdrix wrote:
> Peter Soetens wrote:
> > On Wed, Jun 17, 2009 at 10:09, Philippe Gerum<rpm@xenomai.org> wrote:
> >> As you have probably understood already, building a full real-time to
> >> real-time data path using pipes is not possible; this said, this is not
> >> the purpose of this API anyway, which has been designed for real-time to
> >> non-RT communication.
> > 
> > I was hoping to use rt_pipe + select in real-time context to implement
> > a data receiving server for real-time inter-process communication. Is
> > this possible ?
> > What would happen if the real-time clients open the pipes as rt_pipes
> > and start sending
> > data in ? What's the alternative to listen in real-time to many
> > connections from
> > a single thread ?
> 
> Note that posix message queues already work for inter-process
> communications, with support for select. Depending on your needs, this
> may be sufficient: you will need Xenomai threads on both sides, but the
> non real-time one may use the SCHED_OTHER policy.
> 
> I was thinking, maybe we could map the xnpipe to a special flag in mq_open ?
> 

Albeit the message pipes are most of the time used in datagram mode, it
can also be used in byte stream mode, which would not fit the mq
semantics that nicely (I'm a bit afraid of potential overhead as well).

Actually, we do have a pivotal API we could use further, and this is
RTDM. We could have a set of socket-based RTDM drivers that implement
different RT communication domains for IPC, one emulating a real-time
socketpair for local IPC as you mentioned for instance, another one
would bear the message pipe semantics etc.

RTDM already complements all other APIs, so there would be no need to
extend those to get message pipes with select() support, and we could
even go as far as deprecating the existing message pipe API from the
native skin, at some point. Besides, all RTOS emulators we support work
fine side-by-side with either the POSIX or native skins, which in turn
do support the RTDM model. So basically, we write one IPC driver, it
works for the whole set of APIs we have, without any change. Maybe it's
time for ksrc/drivers/ipc or something?

-- 
Philippe.



_______________________________________________
Xenomai-help mailing list
Xenomai-help@domain.hid
https://mail.gna.org/listinfo/xenomai-help