[Xenomai-help] Mixing skin calls (cancellation points)

[Xenomai-help] Mixing skin calls (cancellation points)

[Steve Deiters]

Is it in general valid to mix calls from different skins?  In
particular, can I expect the native skin calls to work as POSIX
cancellation points?  For example, if I have a POSIX thread with a
pthread_t identifier which I know calls rt_cond_wait somewhere, will it
work as a cancellation point when I call pthread_cancel?

The example program attached will not cancel if I have CANCEL_DEFERRED
defined as 1 at the top, which sets the cancel type to deferred.  This
is on a PowerPC with Xenomai 2.5.3.  I think this worked with Xenomai
2.4.10.


--------


#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
#include <native/task.h>
#include <native/cond.h>

#define CANCEL_DEFERRED 1

RT_MUTEX mutex;
RT_COND cond;
pthread_t thread;

void *entry(void *cookie)
{
   fprintf(stderr, "Starting thread\n");

#if CANCEL_DEFERRED
   pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL);
#else
   pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
#endif

   rt_mutex_acquire(&mutex, TM_INFINITE);
   rt_cond_wait(&cond, &mutex, TM_INFINITE);
   rt_mutex_release(&mutex);

   return NULL;
}

int main(int argc, char **argv)
{
   int ret;
   pthread_attr_t attr;

   if((ret = rt_mutex_create(&mutex, NULL)))
   {
      errno = -ret;
      perror("rt_mutex_create");
      exit(EXIT_FAILURE);
   }

   if((ret = rt_cond_create(&cond, NULL)))
   {
      errno = -ret;
      perror("rt_cond_create");
      exit(EXIT_FAILURE);
   }

   pthread_attr_init(&attr);
   pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);

   if(pthread_create(&thread, &attr, entry, NULL))
   {
      perror("pthread_create");
      exit(EXIT_FAILURE);
   }

   fprintf(stderr, "Sleeping 5 seconds\n");
   sleep(5);
   fprintf(stderr, "Woke up, cancelling thread\n");

   pthread_cancel(thread);
   pthread_join(thread, NULL);

   return 0;
}

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Gilles Chanteperdrix]

Steve Deiters wrote:
> Is it in general valid to mix calls from different skins?  In
> particular, can I expect the native skin calls to work as POSIX
> cancellation points?  For example, if I have a POSIX thread with a
> pthread_t identifier which I know calls rt_cond_wait somewhere, will it
> work as a cancellation point when I call pthread_cancel?
> 
> The example program attached will not cancel if I have CANCEL_DEFERRED
> defined as 1 at the top, which sets the cancel type to deferred.  This
> is on a PowerPC with Xenomai 2.5.3.  I think this worked with Xenomai
> 2.4.10.
> 
> 
> --------
> 
> 
> #include <stdio.h>
> #include <errno.h>
> #include <stdlib.h>
> #include <unistd.h>
> #include <pthread.h>
> #include <native/task.h>
> #include <native/cond.h>
> 
> #define CANCEL_DEFERRED 1
> 
> RT_MUTEX mutex;
> RT_COND cond;
> pthread_t thread;
> 
> void *entry(void *cookie)
> {
>    fprintf(stderr, "Starting thread\n");
> 
> #if CANCEL_DEFERRED
>    pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL);
> #else
>    pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
> #endif
> 
>    rt_mutex_acquire(&mutex, TM_INFINITE);
>    rt_cond_wait(&cond, &mutex, TM_INFINITE);
>    rt_mutex_release(&mutex);
> 
>    return NULL;
> }

This program would also deadlock with plain POSIX: you are missing a
cleanup handler which releases the mutex.

-- 
					    Gilles.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Steve Deiters]

> -----Original Message-----
> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org
> Sent: Tuesday, July 06, 2010 1:42 PM
> To: Steve Deiters
> Cc: xenomai@xenomai.org
> Subject: Re: [Xenomai-help] Mixing skin calls (cancellation points)
> 
> Steve Deiters wrote:
> > Is it in general valid to mix calls from different skins?  In 
> > particular, can I expect the native skin calls to work as POSIX 
> > cancellation points?  For example, if I have a POSIX thread with a 
> > pthread_t identifier which I know calls rt_cond_wait 
> somewhere, will 
> > it work as a cancellation point when I call pthread_cancel?
> > 
> > The example program attached will not cancel if I have 
> CANCEL_DEFERRED 
> > defined as 1 at the top, which sets the cancel type to 
> deferred.  This 
> > is on a PowerPC with Xenomai 2.5.3.  I think this worked 
> with Xenomai 
> > 2.4.10.
> > 
> > 
> > --------
> > 
> > 
> > #include <stdio.h>
> > #include <errno.h>
> > #include <stdlib.h>
> > #include <unistd.h>
> > #include <pthread.h>
> > #include <native/task.h>
> > #include <native/cond.h>
> > 
> > #define CANCEL_DEFERRED 1
> > 
> > RT_MUTEX mutex;
> > RT_COND cond;
> > pthread_t thread;
> > 
> > void *entry(void *cookie)
> > {
> >    fprintf(stderr, "Starting thread\n");
> > 
> > #if CANCEL_DEFERRED
> >    pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL); #else
> >    pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL); #endif
> > 
> >    rt_mutex_acquire(&mutex, TM_INFINITE);
> >    rt_cond_wait(&cond, &mutex, TM_INFINITE);
> >    rt_mutex_release(&mutex);
> > 
> >    return NULL;
> > }
> 
> This program would also deadlock with plain POSIX: you are 
> missing a cleanup handler which releases the mutex.
> 
> -- 
> 					    Gilles.
> 

My situation here is that I have some library routines, which were
written using the native skin.  It was done like this so users of the
library were not forced to link against the POSIX skin.  Not to say that
they would use the libc POSIX implementation, just that they may want to
use only the native skin.

Is there a deadlock issue here if I have the cancellation as deferred?
Won't the rt_cond_wait already release the mutex.  This is similar to
pthread_cond_wait which is a cancellation point in POSIX.  I realize the
potential for deadlock if it asynchronous which is why I want the
cancellation as deferred.

If I ever do need to do something like this, can I use the
rt_task_add_hook looking at T_HOOK_DELETE in place of a POSIX cleanup
handler?

This still doesn't say whether or not I can rely on the rt_cond_wait as
a cancellation point.  If not I may have only two options: rewrite the
library using the POSIX skin and force the users to use the POSIX skin,
or force the user to use the native skin or avoid using pthread_cancel.
Neither one is an appealing option.

Thanks.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Gilles Chanteperdrix]

Steve Deiters wrote:
>> -----Original Message-----
>> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
>> Sent: Tuesday, July 06, 2010 1:42 PM
>> To: Steve Deiters
>> Cc: xenomai@xenomai.org
>> Subject: Re: [Xenomai-help] Mixing skin calls (cancellation points)
>>
>> Steve Deiters wrote:
>>> Is it in general valid to mix calls from different skins?  In 
>>> particular, can I expect the native skin calls to work as POSIX 
>>> cancellation points?  For example, if I have a POSIX thread with a 
>>> pthread_t identifier which I know calls rt_cond_wait 
>> somewhere, will 
>>> it work as a cancellation point when I call pthread_cancel?
>>>
>>> The example program attached will not cancel if I have 
>> CANCEL_DEFERRED 
>>> defined as 1 at the top, which sets the cancel type to 
>> deferred.  This 
>>> is on a PowerPC with Xenomai 2.5.3.  I think this worked 
>> with Xenomai 
>>> 2.4.10.
>>>
>>>
>>> --------
>>>
>>>
>>> #include <stdio.h>
>>> #include <errno.h>
>>> #include <stdlib.h>
>>> #include <unistd.h>
>>> #include <pthread.h>
>>> #include <native/task.h>
>>> #include <native/cond.h>
>>>
>>> #define CANCEL_DEFERRED 1
>>>
>>> RT_MUTEX mutex;
>>> RT_COND cond;
>>> pthread_t thread;
>>>
>>> void *entry(void *cookie)
>>> {
>>>    fprintf(stderr, "Starting thread\n");
>>>
>>> #if CANCEL_DEFERRED
>>>    pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED, NULL); #else
>>>    pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL); #endif
>>>
>>>    rt_mutex_acquire(&mutex, TM_INFINITE);
>>>    rt_cond_wait(&cond, &mutex, TM_INFINITE);
>>>    rt_mutex_release(&mutex);
>>>
>>>    return NULL;
>>> }
>> This program would also deadlock with plain POSIX: you are 
>> missing a cleanup handler which releases the mutex.
>>
>> -- 
>> 					    Gilles.
>>
> 
> My situation here is that I have some library routines, which were
> written using the native skin.  It was done like this so users of the
> library were not forced to link against the POSIX skin.  Not to say that
> they would use the libc POSIX implementation, just that they may want to
> use only the native skin.
> 
> Is there a deadlock issue here if I have the cancellation as deferred?
> Won't the rt_cond_wait already release the mutex.  This is similar to
> pthread_cond_wait which is a cancellation point in POSIX.  I realize the
> potential for deadlock if it asynchronous which is why I want the
> cancellation as deferred.

pthread_cond_wait, when cancelled, re-acquires the mutex, so, if you
want to release it, you have to pthread_cleanup_push a handler which
releases it. This is standard practice with the pthread API.
rt_cond_wait should behave the same way. Anyway, I do not think this is
your problem since no other thread has the mutex in your case.

> 
> If I ever do need to do something like this, can I use the
> rt_task_add_hook looking at T_HOOK_DELETE in place of a POSIX cleanup
> handler?

Basically, hook do not really work in user-space. So, you have to use
pthread_cleanup_push. pthread_cleanup_push is a function of the glibc,
not of Xenomai posix skin. So, you do not need linking with Xenomai
posix skin to use it.

> 
> This still doesn't say whether or not I can rely on the rt_cond_wait as
> a cancellation point.  If not I may have only two options: rewrite the
> library using the POSIX skin and force the users to use the POSIX skin,
> or force the user to use the native skin or avoid using pthread_cancel.
> Neither one is an appealing option.

It should be a cancellation point, but my point was that you may had the
impression that it was not because the thread was in fact blocked in the
mutex re-acquisition. Anyway, it is probably not your issue, so, we are
going to check your test case, and tell you more later.

> 
> Thanks.
> 
> 


-- 
					    Gilles.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Gilles Chanteperdrix]

Steve Deiters wrote:
> Is it in general valid to mix calls from different skins?  In
> particular, can I expect the native skin calls to work as POSIX
> cancellation points?  For example, if I have a POSIX thread with a
> pthread_t identifier which I know calls rt_cond_wait somewhere, will it
> work as a cancellation point when I call pthread_cancel?
> 
> The example program attached will not cancel if I have CANCEL_DEFERRED
> defined as 1 at the top, which sets the cancel type to deferred.  This
> is on a PowerPC with Xenomai 2.5.3.  I think this worked with Xenomai
> 2.4.10.

Hi,

I had a look at the code, and chances are, that in fact it never worked.
The native skin threads are created with asynchronous type, and the rest
 of the skin assumes that it is always the case. To kill a native skin
thread, you are supposed to use rt_task_delete instead of
pthread_cancel, and if you did not change the thread cancellation type,
it will work. If you are interested in waiting for the thread deletion,
you can create the thread joinable with T_JOINABLE, then wait for it
with rt_task_join.

We can, of course, modify the code to allow changing the native thread
cancellation type. But the question is: what is the use-case?


-- 
					    Gilles.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Steve Deiters]

> -----Original Message-----
> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org
> Sent: Wednesday, July 07, 2010 7:50 AM
> To: Steve Deiters
> Cc: xenomai@xenomai.org
> Subject: Re: [Xenomai-help] Mixing skin calls (cancellation points)
> 
> Steve Deiters wrote:
> > Is it in general valid to mix calls from different skins?  In 
> > particular, can I expect the native skin calls to work as POSIX 
> > cancellation points?  For example, if I have a POSIX thread with a 
> > pthread_t identifier which I know calls rt_cond_wait 
> somewhere, will 
> > it work as a cancellation point when I call pthread_cancel?
> > 
> > The example program attached will not cancel if I have 
> CANCEL_DEFERRED 
> > defined as 1 at the top, which sets the cancel type to 
> deferred.  This 
> > is on a PowerPC with Xenomai 2.5.3.  I think this worked 
> with Xenomai 
> > 2.4.10.
> 
> Hi,
> 
> I had a look at the code, and chances are, that in fact it 
> never worked.
> The native skin threads are created with asynchronous type, 
> and the rest  of the skin assumes that it is always the case. 
> To kill a native skin thread, you are supposed to use 
> rt_task_delete instead of pthread_cancel, and if you did not 
> change the thread cancellation type, it will work. If you are 
> interested in waiting for the thread deletion, you can create 
> the thread joinable with T_JOINABLE, then wait for it with 
> rt_task_join.
> 
> We can, of course, modify the code to allow changing the 
> native thread cancellation type. But the question is: what is 
> the use-case?
> 
> 
> -- 
> 					    Gilles.


I was not creating a native thread though.  I was creating a POSIX
thread, which used some native skin calls.  I do not have a RT_TASK in
this case to even use with rt_task_delete, unless there is something to
convert from a pthread_t to a RT_TASK I'm not aware of.  In any case I'm
wanting to cancel a POSIX thread, not a native one.

My usage case is basically as I described previously.  I have some code
sectioned off in a static library.  This code uses only native calls for
stuff like mutexes and condition variable using the assumption that the
end application using the library need not link against the POSIX skin.
In one of the applications it was using a POSIX skin, with POSIX
threads, and was cancelling a thread that was blocked with rt_cond_wait.

Just out of curiosity, is there anything analogous to pthread cleanup
handlers in the native skin?  You mentioned earlier that you cannot use
the rt_add_hook in userspace.  So is there anyway then to protect
against the situation described earlier, where you delete a thread
waiting on a condition variable.  Will the thread reacquire the mutex
before deletion as in the POSIX case and does the rt_task_delete make
sure not to delete the thread while it is holding the mutex in this
case?

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Gilles Chanteperdrix]

Steve Deiters wrote:
>> -----Original Message-----
>> From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
>> Sent: Wednesday, July 07, 2010 7:50 AM
>> To: Steve Deiters
>> Cc: xenomai@xenomai.org
>> Subject: Re: [Xenomai-help] Mixing skin calls (cancellation points)
>>
>> Steve Deiters wrote:
>>> Is it in general valid to mix calls from different skins?  In 
>>> particular, can I expect the native skin calls to work as POSIX 
>>> cancellation points?  For example, if I have a POSIX thread with a 
>>> pthread_t identifier which I know calls rt_cond_wait 
>> somewhere, will 
>>> it work as a cancellation point when I call pthread_cancel?
>>>
>>> The example program attached will not cancel if I have 
>> CANCEL_DEFERRED 
>>> defined as 1 at the top, which sets the cancel type to 
>> deferred.  This 
>>> is on a PowerPC with Xenomai 2.5.3.  I think this worked 
>> with Xenomai 
>>> 2.4.10.
>> Hi,
>>
>> I had a look at the code, and chances are, that in fact it 
>> never worked.
>> The native skin threads are created with asynchronous type, 
>> and the rest  of the skin assumes that it is always the case. 
>> To kill a native skin thread, you are supposed to use 
>> rt_task_delete instead of pthread_cancel, and if you did not 
>> change the thread cancellation type, it will work. If you are 
>> interested in waiting for the thread deletion, you can create 
>> the thread joinable with T_JOINABLE, then wait for it with 
>> rt_task_join.
>>
>> We can, of course, modify the code to allow changing the 
>> native thread cancellation type. But the question is: what is 
>> the use-case?
>>
>>
>> -- 
>> 					    Gilles.
> 
> 
> I was not creating a native thread though.  I was creating a POSIX
> thread, which used some native skin calls.  I do not have a RT_TASK in
> this case to even use with rt_task_delete, unless there is something to
> convert from a pthread_t to a RT_TASK I'm not aware of.  In any case I'm
> wanting to cancel a POSIX thread, not a native one.

The point is that rt_cond_wait assumes that the caller is a native skin
thread, so that it is running with the asynchronous type.

> 
> My usage case is basically as I described previously.  I have some code
> sectioned off in a static library.  This code uses only native calls for
> stuff like mutexes and condition variable using the assumption that the
> end application using the library need not link against the POSIX skin.
> In one of the applications it was using a POSIX skin, with POSIX
> threads, and was cancelling a thread that was blocked with rt_cond_wait.
> 
> Just out of curiosity, is there anything analogous to pthread cleanup
> handlers in the native skin?  You mentioned earlier that you cannot use
> the rt_add_hook in userspace.  So is there anyway then to protect
> against the situation described earlier, where you delete a thread
> waiting on a condition variable.  Will the thread reacquire the mutex
> before deletion as in the POSIX case and does the rt_task_delete make
> sure not to delete the thread while it is holding the mutex in this
> case?

I already answered that: yes, the thread will re-acquire the mutex. The
pthread_cleanup_push/pthread_cleanup_pop services can be called from a
native skin thread, they are part of the glibc, not of Xenomai posix
skin, so you do not need to link against the Xenomai posix skin library.

Anyway, the proper way to fix this, which we use in the POSIX skin, is
to set the cancellation type to asynchronous just before calling
rt_cond_wait, saving the previoys cancellation type, then restore the
cancellation type after rt_cond_wait.


-- 
					    Gilles.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Steve Deiters]

> I already answered that: yes, the thread will re-acquire the 
> mutex. The pthread_cleanup_push/pthread_cleanup_pop services 
> can be called from a native skin thread, they are part of the 
> glibc, not of Xenomai posix skin, so you do not need to link 
> against the Xenomai posix skin library.
> 
> Anyway, the proper way to fix this, which we use in the POSIX 
> skin, is to set the cancellation type to asynchronous just 
> before calling rt_cond_wait, saving the previoys cancellation 
> type, then restore the cancellation type after rt_cond_wait.

This is probably the best solution for me.  Can I also use
pthread_setcanceltype without linking to the POSIX skin or is there some
method in the native skin to change the cancellation type?  Can I assume
anything not wrapped in the posix.wrappers is provided by glibc?

Thanks.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Gilles Chanteperdrix]

Steve Deiters wrote:
>> I already answered that: yes, the thread will re-acquire the 
>> mutex. The pthread_cleanup_push/pthread_cleanup_pop services 
>> can be called from a native skin thread, they are part of the 
>> glibc, not of Xenomai posix skin, so you do not need to link 
>> against the Xenomai posix skin library.
>>
>> Anyway, the proper way to fix this, which we use in the POSIX 
>> skin, is to set the cancellation type to asynchronous just 
>> before calling rt_cond_wait, saving the previoys cancellation 
>> type, then restore the cancellation type after rt_cond_wait.
> 
> This is probably the best solution for me.  Can I also use
> pthread_setcanceltype without linking to the POSIX skin or is there some
> method in the native skin to change the cancellation type?  Can I assume
> anything not wrapped in the posix.wrappers is provided by glibc?

Yes, anything not wrapped in posix.wrappers is provided by glibc. It is
safe to call pthread_setcanceltype without linke to the POSIX skin library.

-- 
					    Gilles.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Gilles Chanteperdrix]

Steve Deiters wrote:
>> I already answered that: yes, the thread will re-acquire the 
>> mutex. The pthread_cleanup_push/pthread_cleanup_pop services 
>> can be called from a native skin thread, they are part of the 
>> glibc, not of Xenomai posix skin, so you do not need to link 
>> against the Xenomai posix skin library.
>>
>> Anyway, the proper way to fix this, which we use in the POSIX 
>> skin, is to set the cancellation type to asynchronous just 
>> before calling rt_cond_wait, saving the previoys cancellation 
>> type, then restore the cancellation type after rt_cond_wait.
> 
> This is probably the best solution for me.  Can I also use
> pthread_setcanceltype without linking to the POSIX skin or is there some
> method in the native skin to change the cancellation type?  Can I assume
> anything not wrapped in the posix.wrappers is provided by glibc?

Ok. We will also fix this on Xenomai side.

-- 
					    Gilles.

Re: [Xenomai-help] Mixing skin calls (cancellation points)

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Steve Deiters wrote:
>>> I already answered that: yes, the thread will re-acquire the 
>>> mutex. The pthread_cleanup_push/pthread_cleanup_pop services 
>>> can be called from a native skin thread, they are part of the 
>>> glibc, not of Xenomai posix skin, so you do not need to link 
>>> against the Xenomai posix skin library.
>>>
>>> Anyway, the proper way to fix this, which we use in the POSIX 
>>> skin, is to set the cancellation type to asynchronous just 
>>> before calling rt_cond_wait, saving the previoys cancellation 
>>> type, then restore the cancellation type after rt_cond_wait.
>> This is probably the best solution for me.  Can I also use
>> pthread_setcanceltype without linking to the POSIX skin or is there some
>> method in the native skin to change the cancellation type?  Can I assume
>> anything not wrapped in the posix.wrappers is provided by glibc?
> 
> Ok. We will also fix this on Xenomai side.
> 
Should be fixed with commit 7a39eff4ed5f9e4aa8acb111328b1343511e41ae
http://git.xenomai.org/?p=xenomai-2.5.git;a=commit;h=7a39eff4ed5f9e4aa8acb111328b1343511e41ae

-- 
					    Gilles.