[Xenomai-core] select: native tasks with posix skin mqueues

[Xenomai-core] select: native tasks with posix skin mqueues

[Peter Soetens]

[-- Attachment #1: Type: text/plain, Size: 1937 bytes --]

Hi,

I'm creating my RT threads using the native API and I'm creating
mqueues, wrapped to the pthread_rt library.
I can read and write the mqueue (and it goes through Xenomai), but
when I select() on a receiving mqd_t, the select() calls returns that
there is data available on the mq (it fills in the FD_SET), but keeps
doing so even when it's empty (the select() is in a loop). Also, it's
modeswitching like nuts.

I found out that the __wrap_select is correctly called, but returns
-EPERM. Kernel sources indicate that this is caused by
pse51_current_thread() alias thread2pthread() returning null. Since
EPERM is returned to userspace, the __real_select is called from user
space, causing the mode switches and bad behaviour. This is almost
certainly the thing that native + RTDM + select() is seeing too.

My mqueues-only work probably because mq.c only uses
pse51_current_thread() in the mq_notify function. I'm guessing that
mq_notify would also not work in combination with native skin.

I had two options in fixing this: add a xnselector to the native task
struct or to the nucleus xnthread_t. I choose the latter, such that
every skin kan use select() + RTDM and migrate gradualy to the RTDM
and/or Posix skin.
I needed to free the xnselector structure in xnpod_delete_thread() , I
chose a spot, but it causes a segfault in my native thread (which did
the select) during program cleanup. Any advice ? Also, maybe we should
separate select() from the posix skin and put it in a separate place
(in RTDM as rtdm_select() ?), such that we can start building around
it (posix just forwards to rtdm_select() then).

A second patch was necessary to return the timeout case properly to
userspace (independent of first patch).

Tested with native + posix loaded and mq. If you never quit your
application, this works :-)

(maybe we discuss this better further on xenomai-core)

Thanks for the wonderful XUM-2009 experience btw !

Peter

[-- Attachment #2: 0001-Move-posix-selector-in-nucleus-for-every-skin-to-use.patch --]
[-- Type: text/x-diff, Size: 4261 bytes --]

From 0380298181f2926e6abe05e6b3d0b02389892a7c Mon Sep 17 00:00:00 2001
From: Peter Soetens <peter@thesourceworks.com>
Date: Thu, 1 Oct 2009 15:57:54 +0200
Subject: [PATCH] Move posix selector in nucleus for every skin to use.
 This patch makes the select implementation in syscall.c independent of the posix skin.

---
 include/nucleus/thread.h   |    3 +++
 ksrc/nucleus/pod.c         |    6 ++++++
 ksrc/nucleus/thread.c      |    2 ++
 ksrc/skins/posix/syscall.c |    6 +++---
 ksrc/skins/posix/thread.c  |    7 -------
 ksrc/skins/posix/thread.h  |    4 ----
 6 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/include/nucleus/thread.h b/include/nucleus/thread.h
index 3eefb53..88a14cc 100644
--- a/include/nucleus/thread.h
+++ b/include/nucleus/thread.h
@@ -142,6 +142,7 @@ struct xnthread;
 struct xnsched;
 struct xnsynch;
 struct xnrpi;
+struct xnselector;
 
 typedef struct xnthrops {
 
@@ -208,6 +209,8 @@ typedef struct xnthread {
 	xnstat_exectime_t lastperiod; /* Interval marker for execution time reports */
     } stat;
 
+    struct xnselector *selector;/* For select. */
+
     int errcode;		/* Local errno */
 
     xnasr_t asr;		/* Asynchronous service routine */
diff --git a/ksrc/nucleus/pod.c b/ksrc/nucleus/pod.c
index 9348ce1..2dd08ca 100644
--- a/ksrc/nucleus/pod.c
+++ b/ksrc/nucleus/pod.c
@@ -41,6 +41,7 @@
 #include <nucleus/registry.h>
 #include <nucleus/module.h>
 #include <nucleus/stat.h>
+#include <nucleus/select.h>
 #include <asm/xenomai/bits/pod.h>
 
 /* debug support */
@@ -1204,6 +1205,11 @@ void xnpod_delete_thread(xnthread_t *thread)
 	xntimer_destroy(&thread->rtimer);
 	xntimer_destroy(&thread->ptimer);
 
+	if (thread->selector) {
+		xnselector_destroy(thread->selector);
+		thread->selector = NULL;
+	}
+
 	if (xnthread_test_state(thread, XNPEND))
 		xnsynch_forget_sleeper(thread);
 
diff --git a/ksrc/nucleus/thread.c b/ksrc/nucleus/thread.c
index 5fceaec..ed9722e 100644
--- a/ksrc/nucleus/thread.c
+++ b/ksrc/nucleus/thread.c
@@ -95,6 +95,8 @@ int xnthread_init(xnthread_t *thread,
 	thread->asr = XNTHREAD_INVALID_ASR;
 	thread->asrlevel = 0;
 
+	thread->selector = NULL;
+
 	thread->iprio = prio;
 	thread->bprio = prio;
 	thread->cprio = prio;
diff --git a/ksrc/skins/posix/syscall.c b/ksrc/skins/posix/syscall.c
index d936f21..f4afd29 100644
--- a/ksrc/skins/posix/syscall.c
+++ b/ksrc/skins/posix/syscall.c
@@ -2382,12 +2382,12 @@ static int __select(struct task_struct *curr, struct pt_regs *regs)
 	xntmode_t mode = XN_RELATIVE;
 	struct xnselector *selector;
 	struct timeval tv;
-	pthread_t thread;
+	xnthread_t* thread;
 	int i, err, nfds;
 	size_t fds_size;
 
-	thread = pse51_current_thread();
-	if (!thread)
+	thread = xnpod_current_thread();
+	if ( !thread )
 		return -EPERM;
 
 	if (__xn_reg_arg5(regs)) {
diff --git a/ksrc/skins/posix/thread.c b/ksrc/skins/posix/thread.c
index ad4aaa5..6e89625 100644
--- a/ksrc/skins/posix/thread.c
+++ b/ksrc/skins/posix/thread.c
@@ -78,12 +78,6 @@ static void thread_delete_hook(xnthread_t *xnthread)
 	pse51_mark_deleted(thread);
 	pse51_signal_cleanup_thread(thread);
 	pse51_timer_cleanup_thread(thread);
-#ifdef CONFIG_XENO_OPT_POSIX_SELECT
-	if (thread->selector) {
-		xnselector_destroy(thread->selector);
-		thread->selector = NULL;
-	}
-#endif /* CONFIG_XENO_OPT_POSIX_SELECT */
 
 	switch (thread_getdetachstate(thread)) {
 	case PTHREAD_CREATE_DETACHED:
@@ -216,7 +210,6 @@ int pthread_create(pthread_t *tid,
 	pse51_signal_init_thread(thread, cur);
 	pse51_tsd_init_thread(thread);
 	pse51_timer_init_thread(thread);
-	thread->selector = NULL;
 
 	if (thread->attr.policy == SCHED_RR) {
 		xnthread_time_slice(&thread->threadbase) = pse51_time_slice;
diff --git a/ksrc/skins/posix/thread.h b/ksrc/skins/posix/thread.h
index 33b14fd..fc8c6fe 100644
--- a/ksrc/skins/posix/thread.h
+++ b/ksrc/skins/posix/thread.h
@@ -21,7 +21,6 @@
 #define _POSIX_THREAD_H
 
 #include <posix/internal.h>
-#include <nucleus/select.h>
 
 typedef unsigned long long pse51_sigset_t;
 
@@ -91,9 +90,6 @@ struct pse51_thread {
     /* For timers. */
     xnqueue_t timersq;
     
-    /* For select. */
-    struct xnselector *selector;
-
 #ifdef CONFIG_XENO_OPT_PERVASIVE
     struct pse51_hkey hkey;
 #endif /* CONFIG_XENO_OPT_PERVASIVE */
-- 
1.6.0.4


[-- Attachment #3: 0002-posix-Fix-__wrap_select-when-timeout-happens.patch --]
[-- Type: text/x-diff, Size: 802 bytes --]

From c406e7ce881927c2744d60503f0710d9b91fa5f4 Mon Sep 17 00:00:00 2001
From: Peter Soetens <peter@thesourceworks.com>
Date: Thu, 1 Oct 2009 16:20:51 +0200
Subject: [PATCH] posix: Fix __wrap_select() when timeout happens.

---
 src/skins/posix/select.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/src/skins/posix/select.c b/src/skins/posix/select.c
index 3076d9c..2104009 100644
--- a/src/skins/posix/select.c
+++ b/src/skins/posix/select.c
@@ -23,9 +23,15 @@ int __wrap_select (int __nfds, fd_set *__restrict __readfds,
 		return __real_select(__nfds, __readfds,
 				     __writefds, __exceptfds, __timeout);
 
+	/* timeout */
+	if (err == 0)
+		return 0;
+
+	/* success */
 	if (err > 0)
 		return err;
 
+	/* failure */
 	errno = -err;
 	return -1;
 }
-- 
1.6.0.4

Re: [Xenomai-core] [Xenomai-help] select: native tasks with posix skin mqueues

[Gilles Chanteperdrix]

Peter Soetens wrote:
> Hi,
> 
> I'm creating my RT threads using the native API and I'm creating
> mqueues, wrapped to the pthread_rt library.
> I can read and write the mqueue (and it goes through Xenomai), but
> when I select() on a receiving mqd_t, the select() calls returns that
> there is data available on the mq (it fills in the FD_SET), but keeps
> doing so even when it's empty (the select() is in a loop). Also, it's
> modeswitching like nuts.
> 
> I found out that the __wrap_select is correctly called, but returns
> -EPERM. Kernel sources indicate that this is caused by
> pse51_current_thread() alias thread2pthread() returning null. Since
> EPERM is returned to userspace, the __real_select is called from user
> space, causing the mode switches and bad behaviour. This is almost
> certainly the thing that native + RTDM + select() is seeing too.
> 
> My mqueues-only work probably because mq.c only uses
> pse51_current_thread() in the mq_notify function. I'm guessing that
> mq_notify would also not work in combination with native skin.
> 
> I had two options in fixing this: add a xnselector to the native task
> struct or to the nucleus xnthread_t. I choose the latter, such that
> every skin kan use select() + RTDM and migrate gradualy to the RTDM
> and/or Posix skin.
> I needed to free the xnselector structure in xnpod_delete_thread() , I
> chose a spot, but it causes a segfault in my native thread (which did
> the select) during program cleanup. Any advice ? Also, maybe we should
> separate select() from the posix skin and put it in a separate place
> (in RTDM as rtdm_select() ?), such that we can start building around
> it (posix just forwards to rtdm_select() then).
> 
> A second patch was necessary to return the timeout case properly to
> userspace (independent of first patch).
> 
> Tested with native + posix loaded and mq. If you never quit your
> application, this works :-)
> 
> (maybe we discuss this better further on xenomai-core)

Ok. Got it now. My idea with that the nucleus service xnselect could be
used to implement select-like services which would have different
semantics depending on the skins.

So, the select service with posix semantics was reserved to posix skin
threads.

-- 
                                          Gilles

Re: [Xenomai-core] [Xenomai-help] select: native tasks with posix skin mqueues

[Peter Soetens]

On Thu, Oct 1, 2009 at 16:47, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> Peter Soetens wrote:
>> Hi,
>>
>> I'm creating my RT threads using the native API and I'm creating
>> mqueues, wrapped to the pthread_rt library.
>> I can read and write the mqueue (and it goes through Xenomai), but
>> when I select() on a receiving mqd_t, the select() calls returns that
>> there is data available on the mq (it fills in the FD_SET), but keeps
>> doing so even when it's empty (the select() is in a loop). Also, it's
>> modeswitching like nuts.
>>
>> I found out that the __wrap_select is correctly called, but returns
>> -EPERM. Kernel sources indicate that this is caused by
>> pse51_current_thread() alias thread2pthread() returning null. Since
>> EPERM is returned to userspace, the __real_select is called from user
>> space, causing the mode switches and bad behaviour. This is almost
>> certainly the thing that native + RTDM + select() is seeing too.
>>
>> My mqueues-only work probably because mq.c only uses
>> pse51_current_thread() in the mq_notify function. I'm guessing that
>> mq_notify would also not work in combination with native skin.
>>
>> I had two options in fixing this: add a xnselector to the native task
>> struct or to the nucleus xnthread_t. I choose the latter, such that
>> every skin kan use select() + RTDM and migrate gradualy to the RTDM
>> and/or Posix skin.
>> I needed to free the xnselector structure in xnpod_delete_thread() , I
>> chose a spot, but it causes a segfault in my native thread (which did
>> the select) during program cleanup. Any advice ? Also, maybe we should
>> separate select() from the posix skin and put it in a separate place
>> (in RTDM as rtdm_select() ?), such that we can start building around
>> it (posix just forwards to rtdm_select() then).
>>
>> A second patch was necessary to return the timeout case properly to
>> userspace (independent of first patch).
>>
>> Tested with native + posix loaded and mq. If you never quit your
>> application, this works :-)
>>
>> (maybe we discuss this better further on xenomai-core)
>
> Ok. Got it now. My idea with that the nucleus service xnselect could be
> used to implement select-like services which would have different
> semantics depending on the skins.
>
> So, the select service with posix semantics was reserved to posix skin
> threads.

Yes. The segfaults I'm seeing is not related to the cleanup of my
xnselector struct in xnpod_delete_thread, because when removing the
cleanup code, still leads to the segfault. Probably Posix does
something special to let the thread leave select() earlier.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff59a224b in pthread_cancel () from /lib/libpthread.so.0
(gdb) bt
#0  0x00007ffff59a224b in pthread_cancel () from /lib/libpthread.so.0
#1  0x00007ffff6ae3ed5 in rt_task_delete (task=0x787388) at task.c:213

Peter

Re: [Xenomai-core] [Xenomai-help] select: native tasks with posix skin mqueues

[Gilles Chanteperdrix]

Peter Soetens wrote:
> On Thu, Oct 1, 2009 at 16:47, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
>> Peter Soetens wrote:
>>> Hi,
>>>
>>> I'm creating my RT threads using the native API and I'm creating
>>> mqueues, wrapped to the pthread_rt library.
>>> I can read and write the mqueue (and it goes through Xenomai), but
>>> when I select() on a receiving mqd_t, the select() calls returns that
>>> there is data available on the mq (it fills in the FD_SET), but keeps
>>> doing so even when it's empty (the select() is in a loop). Also, it's
>>> modeswitching like nuts.
>>>
>>> I found out that the __wrap_select is correctly called, but returns
>>> -EPERM. Kernel sources indicate that this is caused by
>>> pse51_current_thread() alias thread2pthread() returning null. Since
>>> EPERM is returned to userspace, the __real_select is called from user
>>> space, causing the mode switches and bad behaviour. This is almost
>>> certainly the thing that native + RTDM + select() is seeing too.
>>>
>>> My mqueues-only work probably because mq.c only uses
>>> pse51_current_thread() in the mq_notify function. I'm guessing that
>>> mq_notify would also not work in combination with native skin.
>>>
>>> I had two options in fixing this: add a xnselector to the native task
>>> struct or to the nucleus xnthread_t. I choose the latter, such that
>>> every skin kan use select() + RTDM and migrate gradualy to the RTDM
>>> and/or Posix skin.
>>> I needed to free the xnselector structure in xnpod_delete_thread() , I
>>> chose a spot, but it causes a segfault in my native thread (which did
>>> the select) during program cleanup. Any advice ? Also, maybe we should
>>> separate select() from the posix skin and put it in a separate place
>>> (in RTDM as rtdm_select() ?), such that we can start building around
>>> it (posix just forwards to rtdm_select() then).
>>>
>>> A second patch was necessary to return the timeout case properly to
>>> userspace (independent of first patch).
>>>
>>> Tested with native + posix loaded and mq. If you never quit your
>>> application, this works :-)
>>>
>>> (maybe we discuss this better further on xenomai-core)
>> Ok. Got it now. My idea with that the nucleus service xnselect could be
>> used to implement select-like services which would have different
>> semantics depending on the skins.
>>
>> So, the select service with posix semantics was reserved to posix skin
>> threads.
> 
> Yes. The segfaults I'm seeing is not related to the cleanup of my
> xnselector struct in xnpod_delete_thread, because when removing the
> cleanup code, still leads to the segfault. Probably Posix does
> something special to let the thread leave select() earlier.

To know whether the bug comes from your code or from an unseen bug in
xnselect implementation (there is a suspicious access to the xnselector
structure when waking up), could you try the same test with the original
support, simply using posix skin threads.

Other than that, the support is really tied to the posix skin: this
version of select will only accept file descriptors which were returned
by the posix skin or the rtdm skin. So, I am afraid making it a generic
service is a bit hard.

-- 
                                          Gilles

Re: [Xenomai-core] [Xenomai-help] select: native tasks with posix skin mqueues

[Peter Soetens]

On Thu, Oct 1, 2009 at 17:34, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> Peter Soetens wrote:
>> On Thu, Oct 1, 2009 at 16:47, Gilles Chanteperdrix
>> <gilles.chanteperdrix@xenomai.org> wrote:
>>> Peter Soetens wrote:
>>>> Hi,
>>>>
>>>> I'm creating my RT threads using the native API and I'm creating
>>>> mqueues, wrapped to the pthread_rt library.
>>>> I can read and write the mqueue (and it goes through Xenomai), but
>>>> when I select() on a receiving mqd_t, the select() calls returns that
>>>> there is data available on the mq (it fills in the FD_SET), but keeps
>>>> doing so even when it's empty (the select() is in a loop). Also, it's
>>>> modeswitching like nuts.
>>>>
>>>> I found out that the __wrap_select is correctly called, but returns
>>>> -EPERM. Kernel sources indicate that this is caused by
>>>> pse51_current_thread() alias thread2pthread() returning null. Since
>>>> EPERM is returned to userspace, the __real_select is called from user
>>>> space, causing the mode switches and bad behaviour. This is almost
>>>> certainly the thing that native + RTDM + select() is seeing too.
>>>>
>>>> My mqueues-only work probably because mq.c only uses
>>>> pse51_current_thread() in the mq_notify function. I'm guessing that
>>>> mq_notify would also not work in combination with native skin.
>>>>
>>>> I had two options in fixing this: add a xnselector to the native task
>>>> struct or to the nucleus xnthread_t. I choose the latter, such that
>>>> every skin kan use select() + RTDM and migrate gradualy to the RTDM
>>>> and/or Posix skin.
>>>> I needed to free the xnselector structure in xnpod_delete_thread() , I
>>>> chose a spot, but it causes a segfault in my native thread (which did
>>>> the select) during program cleanup. Any advice ? Also, maybe we should
>>>> separate select() from the posix skin and put it in a separate place
>>>> (in RTDM as rtdm_select() ?), such that we can start building around
>>>> it (posix just forwards to rtdm_select() then).
>>>>
>>>> A second patch was necessary to return the timeout case properly to
>>>> userspace (independent of first patch).
>>>>
>>>> Tested with native + posix loaded and mq. If you never quit your
>>>> application, this works :-)
>>>>
>>>> (maybe we discuss this better further on xenomai-core)
>>> Ok. Got it now. My idea with that the nucleus service xnselect could be
>>> used to implement select-like services which would have different
>>> semantics depending on the skins.
>>>
>>> So, the select service with posix semantics was reserved to posix skin
>>> threads.
>>
>> Yes. The segfaults I'm seeing is not related to the cleanup of my
>> xnselector struct in xnpod_delete_thread, because when removing the
>> cleanup code, still leads to the segfault. Probably Posix does
>> something special to let the thread leave select() earlier.
>
> To know whether the bug comes from your code or from an unseen bug in
> xnselect implementation (there is a suspicious access to the xnselector
> structure when waking up), could you try the same test with the original
> support, simply using posix skin threads.

Will do next week.

>
> Other than that, the support is really tied to the posix skin: this
> version of select will only accept file descriptors which were returned
> by the posix skin or the rtdm skin. So, I am afraid making it a generic
> service is a bit hard.

I don't have a clear view like you have, but clearly, being able to
use select() on the rtdm user API while using any other skin than
Posix looks like a big plus to me. I thought this was also in the line
of what Philippe was thinking of, ie to center our file descriptors
around the rtdm. Moving select() to rtdm, and having the Posix skin as
the first 'compatible with rtdm' skin, changes the perspective for the
better imho.

I'm not looking for making select() generic for all skins, but
adapting the existing skins to opt-in for using 'rtdm_select()', which
is something we can merge in gradually. Do you think this is
feasible/desired ?

Peter

Re: [Xenomai-core] [Xenomai-help] select: native tasks with posix skin mqueues

[Gilles Chanteperdrix]

Peter Soetens wrote:
> On Thu, Oct 1, 2009 at 17:34, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
>> Peter Soetens wrote:
>>> On Thu, Oct 1, 2009 at 16:47, Gilles Chanteperdrix
>>> <gilles.chanteperdrix@xenomai.org> wrote:
>>>> Peter Soetens wrote:
>>>>> Hi,
>>>>>
>>>>> I'm creating my RT threads using the native API and I'm creating
>>>>> mqueues, wrapped to the pthread_rt library.
>>>>> I can read and write the mqueue (and it goes through Xenomai), but
>>>>> when I select() on a receiving mqd_t, the select() calls returns that
>>>>> there is data available on the mq (it fills in the FD_SET), but keeps
>>>>> doing so even when it's empty (the select() is in a loop). Also, it's
>>>>> modeswitching like nuts.
>>>>>
>>>>> I found out that the __wrap_select is correctly called, but returns
>>>>> -EPERM. Kernel sources indicate that this is caused by
>>>>> pse51_current_thread() alias thread2pthread() returning null. Since
>>>>> EPERM is returned to userspace, the __real_select is called from user
>>>>> space, causing the mode switches and bad behaviour. This is almost
>>>>> certainly the thing that native + RTDM + select() is seeing too.
>>>>>
>>>>> My mqueues-only work probably because mq.c only uses
>>>>> pse51_current_thread() in the mq_notify function. I'm guessing that
>>>>> mq_notify would also not work in combination with native skin.
>>>>>
>>>>> I had two options in fixing this: add a xnselector to the native task
>>>>> struct or to the nucleus xnthread_t. I choose the latter, such that
>>>>> every skin kan use select() + RTDM and migrate gradualy to the RTDM
>>>>> and/or Posix skin.
>>>>> I needed to free the xnselector structure in xnpod_delete_thread() , I
>>>>> chose a spot, but it causes a segfault in my native thread (which did
>>>>> the select) during program cleanup. Any advice ? Also, maybe we should
>>>>> separate select() from the posix skin and put it in a separate place
>>>>> (in RTDM as rtdm_select() ?), such that we can start building around
>>>>> it (posix just forwards to rtdm_select() then).
>>>>>
>>>>> A second patch was necessary to return the timeout case properly to
>>>>> userspace (independent of first patch).
>>>>>
>>>>> Tested with native + posix loaded and mq. If you never quit your
>>>>> application, this works :-)
>>>>>
>>>>> (maybe we discuss this better further on xenomai-core)
>>>> Ok. Got it now. My idea with that the nucleus service xnselect could be
>>>> used to implement select-like services which would have different
>>>> semantics depending on the skins.
>>>>
>>>> So, the select service with posix semantics was reserved to posix skin
>>>> threads.
>>> Yes. The segfaults I'm seeing is not related to the cleanup of my
>>> xnselector struct in xnpod_delete_thread, because when removing the
>>> cleanup code, still leads to the segfault. Probably Posix does
>>> something special to let the thread leave select() earlier.
>> To know whether the bug comes from your code or from an unseen bug in
>> xnselect implementation (there is a suspicious access to the xnselector
>> structure when waking up), could you try the same test with the original
>> support, simply using posix skin threads.
> 
> Will do next week.
> 
>> Other than that, the support is really tied to the posix skin: this
>> version of select will only accept file descriptors which were returned
>> by the posix skin or the rtdm skin. So, I am afraid making it a generic
>> service is a bit hard.
> 
> I don't have a clear view like you have, but clearly, being able to
> use select() on the rtdm user API while using any other skin than
> Posix looks like a big plus to me.

This was also the big plan when pushing the xnselect service in the nucleus.

> I thought this was also in the line
> of what Philippe was thinking of, ie to center our file descriptors
> around the rtdm. Moving select() to rtdm, and having the Posix skin as
> the first 'compatible with rtdm' skin, changes the perspective for the
> better imho.
> 
> I'm not looking for making select() generic for all skins, but
> adapting the existing skins to opt-in for using 'rtdm_select()', which
> is something we can merge in gradually. Do you think this is
> feasible/desired ?

I think rtdm_select has to be different from select. For the simple
reason that there is currently a translation
posix rtdm fd == rtdm fd + 1024 - 128

To fix this, we will have to wait for something that we are dreaming for
some time, a unified file descriptor abstraction in the nucleus. But
seeing the 2.5 todo list thread, you will have understood that we are
not going to make it for 2.5...



-- 
					    Gilles.

Re: [Xenomai-core] [Xenomai-help] select: native tasks with posix skin mqueues

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Peter Soetens wrote:
>> On Thu, Oct 1, 2009 at 17:34, Gilles Chanteperdrix
>> <gilles.chanteperdrix@xenomai.org> wrote:
>>> Peter Soetens wrote:
>>>> On Thu, Oct 1, 2009 at 16:47, Gilles Chanteperdrix
>>>> <gilles.chanteperdrix@xenomai.org> wrote:
>>>>> Peter Soetens wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I'm creating my RT threads using the native API and I'm creating
>>>>>> mqueues, wrapped to the pthread_rt library.
>>>>>> I can read and write the mqueue (and it goes through Xenomai), but
>>>>>> when I select() on a receiving mqd_t, the select() calls returns that
>>>>>> there is data available on the mq (it fills in the FD_SET), but keeps
>>>>>> doing so even when it's empty (the select() is in a loop). Also, it's
>>>>>> modeswitching like nuts.
>>>>>>
>>>>>> I found out that the __wrap_select is correctly called, but returns
>>>>>> -EPERM. Kernel sources indicate that this is caused by
>>>>>> pse51_current_thread() alias thread2pthread() returning null. Since
>>>>>> EPERM is returned to userspace, the __real_select is called from user
>>>>>> space, causing the mode switches and bad behaviour. This is almost
>>>>>> certainly the thing that native + RTDM + select() is seeing too.
>>>>>>
>>>>>> My mqueues-only work probably because mq.c only uses
>>>>>> pse51_current_thread() in the mq_notify function. I'm guessing that
>>>>>> mq_notify would also not work in combination with native skin.
>>>>>>
>>>>>> I had two options in fixing this: add a xnselector to the native task
>>>>>> struct or to the nucleus xnthread_t. I choose the latter, such that
>>>>>> every skin kan use select() + RTDM and migrate gradualy to the RTDM
>>>>>> and/or Posix skin.
>>>>>> I needed to free the xnselector structure in xnpod_delete_thread() , I
>>>>>> chose a spot, but it causes a segfault in my native thread (which did
>>>>>> the select) during program cleanup. Any advice ? Also, maybe we should
>>>>>> separate select() from the posix skin and put it in a separate place
>>>>>> (in RTDM as rtdm_select() ?), such that we can start building around
>>>>>> it (posix just forwards to rtdm_select() then).
>>>>>>
>>>>>> A second patch was necessary to return the timeout case properly to
>>>>>> userspace (independent of first patch).
>>>>>>
>>>>>> Tested with native + posix loaded and mq. If you never quit your
>>>>>> application, this works :-)
>>>>>>
>>>>>> (maybe we discuss this better further on xenomai-core)
>>>>> Ok. Got it now. My idea with that the nucleus service xnselect could be
>>>>> used to implement select-like services which would have different
>>>>> semantics depending on the skins.
>>>>>
>>>>> So, the select service with posix semantics was reserved to posix skin
>>>>> threads.
>>>> Yes. The segfaults I'm seeing is not related to the cleanup of my
>>>> xnselector struct in xnpod_delete_thread, because when removing the
>>>> cleanup code, still leads to the segfault. Probably Posix does
>>>> something special to let the thread leave select() earlier.
>>> To know whether the bug comes from your code or from an unseen bug in
>>> xnselect implementation (there is a suspicious access to the xnselector
>>> structure when waking up), could you try the same test with the original
>>> support, simply using posix skin threads.
>> Will do next week.
>>
>>> Other than that, the support is really tied to the posix skin: this
>>> version of select will only accept file descriptors which were returned
>>> by the posix skin or the rtdm skin. So, I am afraid making it a generic
>>> service is a bit hard.
>> I don't have a clear view like you have, but clearly, being able to
>> use select() on the rtdm user API while using any other skin than
>> Posix looks like a big plus to me.
> 
> This was also the big plan when pushing the xnselect service in the nucleus.
> 
>> I thought this was also in the line
>> of what Philippe was thinking of, ie to center our file descriptors
>> around the rtdm. Moving select() to rtdm, and having the Posix skin as
>> the first 'compatible with rtdm' skin, changes the perspective for the
>> better imho.
>>
>> I'm not looking for making select() generic for all skins, but
>> adapting the existing skins to opt-in for using 'rtdm_select()', which
>> is something we can merge in gradually. Do you think this is
>> feasible/desired ?
> 
> I think rtdm_select has to be different from select. For the simple
> reason that there is currently a translation
> posix rtdm fd == rtdm fd + 1024 - 128
> 
> To fix this, we will have to wait for something that we are dreaming for
> some time, a unified file descriptor abstraction in the nucleus. But
> seeing the 2.5 todo list thread, you will have understood that we are
> not going to make it for 2.5...

Do not get me wrong. We can fix select to work with non-posix threads
without the unified file descriptor support, the way you did it.
However, this select will still only work for file descriptors obtained
using posix skin wrapped service, not with file descriptors obtained
directly from the rtdm skin. So, it would make little sense to move it
out of the posix skin.

-- 
					    Gilles.

Re: [Xenomai-core] select: native tasks with posix skin mqueues

[Gilles Chanteperdrix]

Peter Soetens wrote:
> Hi,
> 
> I'm creating my RT threads using the native API and I'm creating
> mqueues, wrapped to the pthread_rt library.
> I can read and write the mqueue (and it goes through Xenomai), but
> when I select() on a receiving mqd_t, the select() calls returns that
> there is data available on the mq (it fills in the FD_SET), but keeps
> doing so even when it's empty (the select() is in a loop). Also, it's
> modeswitching like nuts.
> 
> I found out that the __wrap_select is correctly called, but returns
> -EPERM. Kernel sources indicate that this is caused by
> pse51_current_thread() alias thread2pthread() returning null. Since
> EPERM is returned to userspace, the __real_select is called from user
> space, causing the mode switches and bad behaviour. This is almost
> certainly the thing that native + RTDM + select() is seeing too.
> 
> My mqueues-only work probably because mq.c only uses
> pse51_current_thread() in the mq_notify function. I'm guessing that
> mq_notify would also not work in combination with native skin.
> 
> I had two options in fixing this: add a xnselector to the native task
> struct or to the nucleus xnthread_t. I choose the latter, such that
> every skin kan use select() + RTDM and migrate gradualy to the RTDM
> and/or Posix skin.
> I needed to free the xnselector structure in xnpod_delete_thread() , I
> chose a spot, but it causes a segfault in my native thread (which did
> the select) during program cleanup. Any advice ? Also, maybe we should
> separate select() from the posix skin and put it in a separate place
> (in RTDM as rtdm_select() ?), such that we can start building around
> it (posix just forwards to rtdm_select() then).
> 
> A second patch was necessary to return the timeout case properly to
> userspace (independent of first patch).
> 
> Tested with native + posix loaded and mq. If you never quit your
> application, this works :-)

Hi,

I have included a lightly modified version of this patch on head, I do
not see any crash.  However, I have some doubts about the current
implementation: calling xnselector_destroy() opens opportunities for a
rescheduling, which I am not sure is really what we want in the middle
of xnpod_delete_thread(). Philippe, what do you think?

Regards.

-- 
					    Gilles.

Re: [Xenomai-core] select: native tasks with posix skin mqueues

[Philippe Gerum]

On Thu, 2009-11-05 at 02:46 +0100, Gilles Chanteperdrix wrote:
> Peter Soetens wrote:
> > Hi,
> > 
> > I'm creating my RT threads using the native API and I'm creating
> > mqueues, wrapped to the pthread_rt library.
> > I can read and write the mqueue (and it goes through Xenomai), but
> > when I select() on a receiving mqd_t, the select() calls returns that
> > there is data available on the mq (it fills in the FD_SET), but keeps
> > doing so even when it's empty (the select() is in a loop). Also, it's
> > modeswitching like nuts.
> > 
> > I found out that the __wrap_select is correctly called, but returns
> > -EPERM. Kernel sources indicate that this is caused by
> > pse51_current_thread() alias thread2pthread() returning null. Since
> > EPERM is returned to userspace, the __real_select is called from user
> > space, causing the mode switches and bad behaviour. This is almost
> > certainly the thing that native + RTDM + select() is seeing too.
> > 
> > My mqueues-only work probably because mq.c only uses
> > pse51_current_thread() in the mq_notify function. I'm guessing that
> > mq_notify would also not work in combination with native skin.
> > 
> > I had two options in fixing this: add a xnselector to the native task
> > struct or to the nucleus xnthread_t. I choose the latter, such that
> > every skin kan use select() + RTDM and migrate gradualy to the RTDM
> > and/or Posix skin.
> > I needed to free the xnselector structure in xnpod_delete_thread() , I
> > chose a spot, but it causes a segfault in my native thread (which did
> > the select) during program cleanup. Any advice ? Also, maybe we should
> > separate select() from the posix skin and put it in a separate place
> > (in RTDM as rtdm_select() ?), such that we can start building around
> > it (posix just forwards to rtdm_select() then).
> > 
> > A second patch was necessary to return the timeout case properly to
> > userspace (independent of first patch).
> > 
> > Tested with native + posix loaded and mq. If you never quit your
> > application, this works :-)
> 
> Hi,
> 
> I have included a lightly modified version of this patch on head, I do
> not see any crash.  However, I have some doubts about the current
> implementation: calling xnselector_destroy() opens opportunities for a
> rescheduling, which I am not sure is really what we want in the middle
> of xnpod_delete_thread(). Philippe, what do you think?
> 

Same as you do: that would break.

> Regards.
> 


-- 
Philippe.

Re: [Xenomai-core] select: native tasks with posix skin mqueues

[Peter Soetens]

On Thu, Nov 5, 2009 at 02:46, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
>
> Peter Soetens wrote:
> > Hi,
> >
> > I'm creating my RT threads using the native API and I'm creating
> > mqueues, wrapped to the pthread_rt library.
> > I can read and write the mqueue (and it goes through Xenomai), but
> > when I select() on a receiving mqd_t, the select() calls returns that
> > there is data available on the mq (it fills in the FD_SET), but keeps
> > doing so even when it's empty (the select() is in a loop). Also, it's
> > modeswitching like nuts.
> >
> > I found out that the __wrap_select is correctly called, but returns
> > -EPERM. Kernel sources indicate that this is caused by
> > pse51_current_thread() alias thread2pthread() returning null. Since
> > EPERM is returned to userspace, the __real_select is called from user
> > space, causing the mode switches and bad behaviour. This is almost
> > certainly the thing that native + RTDM + select() is seeing too.
> >
> > My mqueues-only work probably because mq.c only uses
> > pse51_current_thread() in the mq_notify function. I'm guessing that
> > mq_notify would also not work in combination with native skin.
> >
> > I had two options in fixing this: add a xnselector to the native task
> > struct or to the nucleus xnthread_t. I choose the latter, such that
> > every skin kan use select() + RTDM and migrate gradualy to the RTDM
> > and/or Posix skin.
> > I needed to free the xnselector structure in xnpod_delete_thread() , I
> > chose a spot, but it causes a segfault in my native thread (which did
> > the select) during program cleanup. Any advice ? Also, maybe we should
> > separate select() from the posix skin and put it in a separate place
> > (in RTDM as rtdm_select() ?), such that we can start building around
> > it (posix just forwards to rtdm_select() then).
> >
> > A second patch was necessary to return the timeout case properly to
> > userspace (independent of first patch).
> >
> > Tested with native + posix loaded and mq. If you never quit your
> > application, this works :-)
>
> Hi,
>
> I have included a lightly modified version of this patch on head, I do
> not see any crash.  However, I have some doubts about the current
> implementation: calling xnselector_destroy() opens opportunities for a
> rescheduling, which I am not sure is really what we want in the middle
> of xnpod_delete_thread(). Philippe, what do you think?

I'll test this patch this week too. It seems you forgot to apply the
second patch, which can go straight into the 2.4 head, since it fixes
a bug in the select() wrapping code when timeouts are used. See
parent's 0002-posix-Fix-__wrap_select-when-timeout-happens.patch

Peter

Re: [Xenomai-core] select: native tasks with posix skin mqueues

[Gilles Chanteperdrix]

Peter Soetens wrote:
> On Thu, Nov 5, 2009 at 02:46, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
>> Peter Soetens wrote:
>>> Hi,
>>>
>>> I'm creating my RT threads using the native API and I'm creating
>>> mqueues, wrapped to the pthread_rt library.
>>> I can read and write the mqueue (and it goes through Xenomai), but
>>> when I select() on a receiving mqd_t, the select() calls returns that
>>> there is data available on the mq (it fills in the FD_SET), but keeps
>>> doing so even when it's empty (the select() is in a loop). Also, it's
>>> modeswitching like nuts.
>>>
>>> I found out that the __wrap_select is correctly called, but returns
>>> -EPERM. Kernel sources indicate that this is caused by
>>> pse51_current_thread() alias thread2pthread() returning null. Since
>>> EPERM is returned to userspace, the __real_select is called from user
>>> space, causing the mode switches and bad behaviour. This is almost
>>> certainly the thing that native + RTDM + select() is seeing too.
>>>
>>> My mqueues-only work probably because mq.c only uses
>>> pse51_current_thread() in the mq_notify function. I'm guessing that
>>> mq_notify would also not work in combination with native skin.
>>>
>>> I had two options in fixing this: add a xnselector to the native task
>>> struct or to the nucleus xnthread_t. I choose the latter, such that
>>> every skin kan use select() + RTDM and migrate gradualy to the RTDM
>>> and/or Posix skin.
>>> I needed to free the xnselector structure in xnpod_delete_thread() , I
>>> chose a spot, but it causes a segfault in my native thread (which did
>>> the select) during program cleanup. Any advice ? Also, maybe we should
>>> separate select() from the posix skin and put it in a separate place
>>> (in RTDM as rtdm_select() ?), such that we can start building around
>>> it (posix just forwards to rtdm_select() then).
>>>
>>> A second patch was necessary to return the timeout case properly to
>>> userspace (independent of first patch).
>>>
>>> Tested with native + posix loaded and mq. If you never quit your
>>> application, this works :-)
>> Hi,
>>
>> I have included a lightly modified version of this patch on head, I do
>> not see any crash.  However, I have some doubts about the current
>> implementation: calling xnselector_destroy() opens opportunities for a
>> rescheduling, which I am not sure is really what we want in the middle
>> of xnpod_delete_thread(). Philippe, what do you think?
> 
> I'll test this patch this week too. It seems you forgot to apply the
> second patch, which can go straight into the 2.4 head, since it fixes
> a bug in the select() wrapping code when timeouts are used. See
> parent's 0002-posix-Fix-__wrap_select-when-timeout-happens.patch

Yes, I forgot it, will apply soon, thanks.

-- 
                                          Gilles

Re: [Xenomai-core] select: native tasks with posix skin mqueues

[Peter Soetens]

On Mon, Nov 30, 2009 at 15:20, Peter Soetens <peter@domain.hid> wrote:
> On Thu, Nov 5, 2009 at 02:46, Gilles Chanteperdrix
> <gilles.chanteperdrix@xenomai.org> wrote:
>>
>> Peter Soetens wrote:
>> > Hi,
>> >
>> > I'm creating my RT threads using the native API and I'm creating
>> > mqueues, wrapped to the pthread_rt library.
>> > I can read and write the mqueue (and it goes through Xenomai), but
>> > when I select() on a receiving mqd_t, the select() calls returns that
>> > there is data available on the mq (it fills in the FD_SET), but keeps
>> > doing so even when it's empty (the select() is in a loop). Also, it's
>> > modeswitching like nuts.
>> >
>> > I found out that the __wrap_select is correctly called, but returns
>> > -EPERM. Kernel sources indicate that this is caused by
>> > pse51_current_thread() alias thread2pthread() returning null. Since
>> > EPERM is returned to userspace, the __real_select is called from user
>> > space, causing the mode switches and bad behaviour. This is almost
>> > certainly the thing that native + RTDM + select() is seeing too.
>> >
>> > My mqueues-only work probably because mq.c only uses
>> > pse51_current_thread() in the mq_notify function. I'm guessing that
>> > mq_notify would also not work in combination with native skin.
>> >
>> > I had two options in fixing this: add a xnselector to the native task
>> > struct or to the nucleus xnthread_t. I choose the latter, such that
>> > every skin kan use select() + RTDM and migrate gradualy to the RTDM
>> > and/or Posix skin.
>> > I needed to free the xnselector structure in xnpod_delete_thread() , I
>> > chose a spot, but it causes a segfault in my native thread (which did
>> > the select) during program cleanup. Any advice ? Also, maybe we should
>> > separate select() from the posix skin and put it in a separate place
>> > (in RTDM as rtdm_select() ?), such that we can start building around
>> > it (posix just forwards to rtdm_select() then).
>> >
>> > A second patch was necessary to return the timeout case properly to
>> > userspace (independent of first patch).
>> >
>> > Tested with native + posix loaded and mq. If you never quit your
>> > application, this works :-)
>>
>> Hi,
>>
>> I have included a lightly modified version of this patch on head, I do
>> not see any crash.  However, I have some doubts about the current
>> implementation: calling xnselector_destroy() opens opportunities for a
>> rescheduling, which I am not sure is really what we want in the middle
>> of xnpod_delete_thread(). Philippe, what do you think?
>
> I'll test this patch this week too. It seems you forgot to apply the
> second patch, which can go straight into the 2.4 head, since it fixes
> a bug in the select() wrapping code when timeouts are used. See
> parent's 0002-posix-Fix-__wrap_select-when-timeout-happens.patch

I tested the 2.5 branch with this patch and it works here too.

Peter