[Xenomai-help] rtdm_event_timedwait returns -EINTR

[Xenomai-help] rtdm_event_timedwait returns -EINTR

[Petr Cervenka]

[-- Attachment #1: Type: text/plain, Size: 931 bytes --]

Hello,
I have a problem with rtdm_event_timedwait() function. It sometimes ends prematurely with -EINTR error code. It depends on special circumstances (CPU load, resizing console window in X, ...).

I tried to create simple example with native skin to reproduce the behaviour. But the rt_event_wait() behaves differently.
It, insted of returning -EINTR, resets the timeout. It's like the function is called ones more after last interruption/unblock (with original timeout).

normal run:
------------------
event: res = -110 (Connection timed out)
time - actual 5.00001 s, wanted 5 s

window resizing
-------------------------
event: res = -110 (Connection timed out)
time - actual 14.9161 s, wanted 5 s

used versions:
----------------------
Linux rtc 2.6.26.3-adeos #1 SMP PREEMPT Tue Apr 14 15:03:35 CEST 2009 x86_64 GNU/Linux
Xenomai 2.4.7
adeos-ipipe-2.6.26-x86-2.0-17.patch

Thank you for any help or advice.

Petr Cervenka


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: main.cpp --]
[-- Type: text/x-c++src; name="main.cpp", Size: 1187 bytes --]

#include <sys/mman.h>
#include <native/task.h>
#include <native/event.h>
#include <native/timer.h>

#define MY_EVENT    0x00000001ll
#define MY_PRIORITY 99

static RT_TASK mainTask;

int main(int argc, char** argv) {
    const double ONE_SEC = 1e9;
    const RTIME timeout = (RTIME)(5 * ONE_SEC);
    RT_EVENT event;
    unsigned long mask;
    RTIME start, end;
    int res;

    mlockall(MCL_CURRENT | MCL_FUTURE);

    rt_task_shadow(&mainTask, NULL, MY_PRIORITY, T_FPU);

    rt_task_set_mode(0, T_PRIMARY, NULL);

    rt_event_create(&event, NULL, 0, EV_PRIO);

    start = rt_timer_read();
    res = rt_event_wait(&event, MY_EVENT, &mask, EV_ALL, rt_timer_ns2ticks(timeout));
    end = rt_timer_read();

    switch (res) {
        case -EINVAL:
        case -EIDRM:
        case -EWOULDBLOCK:
        case -EINTR:
        case -ETIMEDOUT:
        case -EPERM:
            printf("event: res = %d (%s)\n", res, strerror(-res));
            break;
        default:
            printf("unknown event: res = %d\n", res);
    }

    printf("time - actual %g s, wanted %g s\n", rt_timer_ticks2ns(end - start)/ONE_SEC, timeout/ONE_SEC);
    
    rt_event_delete(&event);

    return 0;
}

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Thomas Lockhart]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

>  Hello, I have a problem with rtdm_event_timedwait() function. It
>  sometimes ends prematurely with -EINTR error code. It depends on
>  special circumstances (CPU load, resizing console window in X, ...).

In other areas of Xenomai when one sees a EINTR return one should 
reenter the original call. Like this (?):

do {
  err = rt_task_wait_period(...)
} while (err != -EINTR);

hth

                               - Tom

-- 
Thomas Lockhart
Supervisor, Distributed and Real-time Group
Instrument Software and Science Data Systems
Caltech/JPL


[-- Attachment #2: Type: text/html, Size: 1043 bytes --]

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Thu, 2009-04-30 at 07:09 -0700, Thomas Lockhart wrote:
> > Hello, I have a problem with rtdm_event_timedwait() function. It
> > sometimes ends prematurely with -EINTR error code. It depends on
> > special circumstances (CPU load, resizing console window in X, ...).
> 
> In other areas of Xenomai when one sees a EINTR return one should
> reenter the original call. Like this (?):
> 
> do {
>   err = rt_task_wait_period(...)
> } while (err != -EINTR);
> 

Just to clarify a few points more:

Xenomai always tells the kernel that blocking (Xenomai) syscalls
_should_ be restarted when interrupted by a Linux signal (i.e.
-ERESTARTSYS is passed down to the kernel by the Xenomai core in that
case), and the kernel will actually restart that (Xenomai) syscall if no
handler was installed for such signal, or if SA_RESTART is set in the
sigaction() flags for the signal.

In all other cases, the syscall will not be restarted despite the
"should restart" hint passed to the Linux kernel, and -EINTR will be
eventually returned by the native API syscall (or errno == EINTR for the
POSIX skin). The way other skins (e.g. VxWorks) will propagate the
-EINTR condition on interrupted syscalls to the user program depends on
their respective API conventions, but they will propagate that
information in a way or another as well, if the syscall was not
restarted.

Additionally, syscalls may be forcibly unblocked via rt_task_unblock(),
which will cause -EINTR to be returned to the caller as well. This is
totally unrelated to Linux signal management though.

> hth
> 
>                                - Tom
> 
> -- 
> Thomas Lockhart
> Supervisor, Distributed and Real-time Group
> Instrument Software and Science Data Systems
> Caltech/JPL
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Gilles Chanteperdrix]

Philippe Gerum wrote:
> On Thu, 2009-04-30 at 07:09 -0700, Thomas Lockhart wrote:
>>> Hello, I have a problem with rtdm_event_timedwait() function. It
>>> sometimes ends prematurely with -EINTR error code. It depends on
>>> special circumstances (CPU load, resizing console window in X, ...).
>> In other areas of Xenomai when one sees a EINTR return one should
>> reenter the original call. Like this (?):
>>
>> do {
>>   err = rt_task_wait_period(...)
>> } while (err != -EINTR);
>>
> 
> Just to clarify a few points more:
> 
> Xenomai always tells the kernel that blocking (Xenomai) syscalls
> _should_ be restarted when interrupted by a Linux signal (i.e.
> -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
> case), and the kernel will actually restart that (Xenomai) syscall if no
> handler was installed for such signal, or if SA_RESTART is set in the
> sigaction() flags for the signal.
> 
> In all other cases, the syscall will not be restarted despite the
> "should restart" hint passed to the Linux kernel, and -EINTR will be
> eventually returned by the native API syscall (or errno == EINTR for the
> POSIX skin). The way other skins (e.g. VxWorks) will propagate the
> -EINTR condition on interrupted syscalls to the user program depends on
> their respective API conventions, but they will propagate that
> information in a way or another as well, if the syscall was not
> restarted.
> 
> Additionally, syscalls may be forcibly unblocked via rt_task_unblock(),
> which will cause -EINTR to be returned to the caller as well. This is
> totally unrelated to Linux signal management though.

Are not there cases where the kernel/libc "jumps back" to the beginning
of the syscall when asked to restart a syscall ? Is not it what causes
the problem with rt_event_wait ? In fact if it is the problem then it
means that the user-space skins should always convert delays to an
absolute expiry date, and pass this as argument to the syscall, so that
blindly restarting the syscall will work as expected. However, if that
is the case, it means that we have quite a few syscalls to fix...

Also, from your mail, it is not clear to me, what is Petr problem,
rtdm_event_timedwait ? or rt_event_wait.

-- 
                                                 Gilles.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Thu, 2009-04-30 at 17:27 +0200, Gilles Chanteperdrix wrote:
> Philippe Gerum wrote:
> > On Thu, 2009-04-30 at 07:09 -0700, Thomas Lockhart wrote:
> >>> Hello, I have a problem with rtdm_event_timedwait() function. It
> >>> sometimes ends prematurely with -EINTR error code. It depends on
> >>> special circumstances (CPU load, resizing console window in X, ...).
> >> In other areas of Xenomai when one sees a EINTR return one should
> >> reenter the original call. Like this (?):
> >>
> >> do {
> >>   err = rt_task_wait_period(...)
> >> } while (err != -EINTR);
> >>
> > 
> > Just to clarify a few points more:
> > 
> > Xenomai always tells the kernel that blocking (Xenomai) syscalls
> > _should_ be restarted when interrupted by a Linux signal (i.e.
> > -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
> > case), and the kernel will actually restart that (Xenomai) syscall if no
> > handler was installed for such signal, or if SA_RESTART is set in the
> > sigaction() flags for the signal.
> > 
> > In all other cases, the syscall will not be restarted despite the
> > "should restart" hint passed to the Linux kernel, and -EINTR will be
> > eventually returned by the native API syscall (or errno == EINTR for the
> > POSIX skin). The way other skins (e.g. VxWorks) will propagate the
> > -EINTR condition on interrupted syscalls to the user program depends on
> > their respective API conventions, but they will propagate that
> > information in a way or another as well, if the syscall was not
> > restarted.
> > 
> > Additionally, syscalls may be forcibly unblocked via rt_task_unblock(),
> > which will cause -EINTR to be returned to the caller as well. This is
> > totally unrelated to Linux signal management though.
> 
> Are not there cases where the kernel/libc "jumps back" to the beginning
> of the syscall when asked to restart a syscall ?

Yes, that is how a restart is done, simply by moving the instruction
pointer back.

>  Is not it what causes
> the problem with rt_event_wait ?
>  In fact if it is the problem then it
> means that the user-space skins should always convert delays to an
> absolute expiry date, and pass this as argument to the syscall, so that
> blindly restarting the syscall will work as expected. However, if that
> is the case, it means that we have quite a few syscalls to fix...
> 

This is why 2.5 introduces *_until() call forms for most timed syscalls,
to specify absolute delays.

> Also, from your mail, it is not clear to me, what is Petr problem,
> rtdm_event_timedwait ? or rt_event_wait.
> 

Same issue actually; both are blocking syscalls that may be restarted.
In the rtdm_event_timedwait() case there is an additional issue: the
initial timeout value is reloaded each time a restart happens, i.e.
three times in a row in this case, because this is a relative timespec.

-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Thomas Lockhart]

> Xenomai always tells the kernel that blocking (Xenomai) syscalls
> _should_ be restarted when interrupted by a Linux signal (i.e.
> -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
> case), and the kernel will actually restart that (Xenomai) syscall if no
> handler was installed for such signal, or if SA_RESTART is set in the
> sigaction() flags for the signal.

Thanks for the clarification. I was hoping my reply would expose my lack 
of understanding :)

So I'm getting what I think are great results on my system with my 
Xenomai-enabled software (a few usec jitter, ~10usec latency or offset 
without anything special done when building or calibrating Xenomai. The 
system is a fairly modern dual-processor Xeon desktop machine and 
Xenomai is 2.4.7; kernel is 2.6.26 (needed for a third party device 
driver which breaks with 2.6.27).

When running at a very low rate (10Hz) every 30 seconds or so there is a 
latency spike of around 50usec. Not bad (and acceptable for my system 
even at kHz rates), but it certainly stands out from the usual case.

Could this occasional extra latency be due to this signal interrupt and 
transparent restart? Or should I dig around elsewhere?

TIA

                           - Tom

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Thu, 2009-04-30 at 15:40 -0700, Thomas Lockhart wrote:
> > Xenomai always tells the kernel that blocking (Xenomai) syscalls
> > _should_ be restarted when interrupted by a Linux signal (i.e.
> > -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
> > case), and the kernel will actually restart that (Xenomai) syscall if no
> > handler was installed for such signal, or if SA_RESTART is set in the
> > sigaction() flags for the signal.
> 
> Thanks for the clarification. I was hoping my reply would expose my lack 
> of understanding :)
> 
> So I'm getting what I think are great results on my system with my 
> Xenomai-enabled software (a few usec jitter, ~10usec latency or offset 
> without anything special done when building or calibrating Xenomai. The 
> system is a fairly modern dual-processor Xeon desktop machine and 
> Xenomai is 2.4.7; kernel is 2.6.26 (needed for a third party device 
> driver which breaks with 2.6.27).
> 
> When running at a very low rate (10Hz) every 30 seconds or so there is a 
> latency spike of around 50usec. Not bad (and acceptable for my system 
> even at kHz rates), but it certainly stands out from the usual case.
> 
> Could this occasional extra latency be due to this signal interrupt and 
> transparent restart? Or should I dig around elsewhere?
> 

Linux signal handling is fully preemptible by Xenomai like most of what
happens in plain kernel context, so this is most likely not related. I
mean that a task receiving a signal would not delay a real-time task
(unless both tasks are somehow logically bound, but I guess this is not
the case here).

Maybe counter-intuitively, the lower the rate, the higher the latency.
This is due to cache artifacts, since at low rate, the non-RT Linux
activity has even more opportunities to cause cache eviction of RT code
since it may run longer. This is why running a latency test at 10Khz
most often gives better results than running it at 1Khz on any
architecture (e.g. latency -p 100 vs latency -p 1000). However, bumping
from 10 to 50 us seems quite a large penalty for such situation,
especially on high-end x86 architectures.

The fact that such peak seems to occur periodically may point the finger
at an external factor, like SMI-like preemption (even if the penalty
usually observed in this case is more in the 300 us range).

In order to understand what is going on your system, I would first try
to reproduce the issue using the Xenomai latency, running at 10hz, on a
kernel enabling the I-Pipe tracer (CONFIG_IPIPE_TRACE). The latency test
provides an option that takes a snapshot of the kernel execution path at
the time the highest latency was observed (-f switch). i.e.

# echo 1 > /proc/ipipe/trace/verbose
# echo 200 > /proc/ipipe/trace/back_trace_points
# /usr/xenomai/bin/latency -p 100000 -f

Once this latency has been observed, just stop the latency test, then
read /proc/ipipe/trace/frozen to get the trace log, and post it to this
list.

HTH, 

> TIA
> 
>                            - Tom
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Thu, 2009-04-30 at 15:40 -0700, Thomas Lockhart wrote:
> > Xenomai always tells the kernel that blocking (Xenomai) syscalls
> > _should_ be restarted when interrupted by a Linux signal (i.e.
> > -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
> > case), and the kernel will actually restart that (Xenomai) syscall if no
> > handler was installed for such signal, or if SA_RESTART is set in the
> > sigaction() flags for the signal.
> 
> Thanks for the clarification. I was hoping my reply would expose my lack 
> of understanding :)
> 
> So I'm getting what I think are great results on my system with my 
> Xenomai-enabled software (a few usec jitter, ~10usec latency or offset 
> without anything special done when building or calibrating Xenomai. The 
> system is a fairly modern dual-processor Xeon desktop machine and 
> Xenomai is 2.4.7; kernel is 2.6.26 (needed for a third party device 
> driver which breaks with 2.6.27).
> 

Mmm, actually, I would bet that this latency is caused by an inter-CPU
TLB flush request; this is a bug fixed in recent releases of the I-pipe
patch, both for 2.6.28.9 and 2.6.29.1. There is no backport for 2.6.26,
but you should be able to apply this commit manually:

http://git.denx.de/?p=ipipe-2.6.git;a=commit;h=69f7e0c005eca37a04104dd85f30095791e8c2f7

> When running at a very low rate (10Hz) every 30 seconds or so there is a 
> latency spike of around 50usec. Not bad (and acceptable for my system 
> even at kHz rates), but it certainly stands out from the usual case.
> 
> Could this occasional extra latency be due to this signal interrupt and 
> transparent restart? Or should I dig around elsewhere?
> 
> TIA
> 
>                            - Tom
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Thomas Lockhart]

>  Mmm, actually, I would bet that this latency is caused by an
>  inter-CPU TLB flush request; this is a bug fixed in recent releases
>  of the I-pipe patch, both for 2.6.28.9 and 2.6.29.1. There is no
>  backport for 2.6.26, but you should be able to apply this commit
>  manually:
> 
http://git.denx.de/?p=ipipe-2.6.git;a=commit;h=69f7e0c005eca37a04104dd85f30095791e8c2f7

That was easy to patch manually and seems to have helped a lot. I'm 
still seeing an occasional "spike" of about 35usec or so every few 
minutes but the majority of the spikes (hmm, the latency is still pretty 
good so I hate to call them spikes; maybe "bumps"?) have gone away. 
Thanks for the suggestion!

Would it be helpful if I integrated this patch into the 2.6.26 support?

                                      - Tom

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Fri, 2009-05-01 at 14:35 -0700, Thomas Lockhart wrote:
> >  Mmm, actually, I would bet that this latency is caused by an
> >  inter-CPU TLB flush request; this is a bug fixed in recent releases
> >  of the I-pipe patch, both for 2.6.28.9 and 2.6.29.1. There is no
> >  backport for 2.6.26, but you should be able to apply this commit
> >  manually:
> > 
> http://git.denx.de/?p=ipipe-2.6.git;a=commit;h=69f7e0c005eca37a04104dd85f30095791e8c2f7
> 
> That was easy to patch manually and seems to have helped a lot. I'm 
> still seeing an occasional "spike" of about 35usec or so every few 
> minutes but the majority of the spikes (hmm, the latency is still pretty 
> good so I hate to call them spikes; maybe "bumps"?) have gone away. 
> Thanks for the suggestion!
> 
> Would it be helpful if I integrated this patch into the 2.6.26 support?
> 

Yep.

>                                       - Tom
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Thomas Lockhart]

> > Would it be helpful if I integrated this patch into the 2.6.26
> > support?
>  Yep.

OK, I have patches. Will continue the discussion for those on 
xenomai-core unless you prefer otherwise.

                               - Tom

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Gilles Chanteperdrix]

Philippe Gerum wrote:
> On Thu, 2009-04-30 at 17:27 +0200, Gilles Chanteperdrix wrote:
>> Philippe Gerum wrote:
>>> On Thu, 2009-04-30 at 07:09 -0700, Thomas Lockhart wrote:
>>>>> Hello, I have a problem with rtdm_event_timedwait() function. It
>>>>> sometimes ends prematurely with -EINTR error code. It depends on
>>>>> special circumstances (CPU load, resizing console window in X, ...).
>>>> In other areas of Xenomai when one sees a EINTR return one should
>>>> reenter the original call. Like this (?):
>>>>
>>>> do {
>>>>   err = rt_task_wait_period(...)
>>>> } while (err != -EINTR);
>>>>
>>> Just to clarify a few points more:
>>>
>>> Xenomai always tells the kernel that blocking (Xenomai) syscalls
>>> _should_ be restarted when interrupted by a Linux signal (i.e.
>>> -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
>>> case), and the kernel will actually restart that (Xenomai) syscall if no
>>> handler was installed for such signal, or if SA_RESTART is set in the
>>> sigaction() flags for the signal.
>>>
>>> In all other cases, the syscall will not be restarted despite the
>>> "should restart" hint passed to the Linux kernel, and -EINTR will be
>>> eventually returned by the native API syscall (or errno == EINTR for the
>>> POSIX skin). The way other skins (e.g. VxWorks) will propagate the
>>> -EINTR condition on interrupted syscalls to the user program depends on
>>> their respective API conventions, but they will propagate that
>>> information in a way or another as well, if the syscall was not
>>> restarted.
>>>
>>> Additionally, syscalls may be forcibly unblocked via rt_task_unblock(),
>>> which will cause -EINTR to be returned to the caller as well. This is
>>> totally unrelated to Linux signal management though.
>> Are not there cases where the kernel/libc "jumps back" to the beginning
>> of the syscall when asked to restart a syscall ?
> 
> Yes, that is how a restart is done, simply by moving the instruction
> pointer back.
> 
>>  Is not it what causes
>> the problem with rt_event_wait ?
>>  In fact if it is the problem then it
>> means that the user-space skins should always convert delays to an
>> absolute expiry date, and pass this as argument to the syscall, so that
>> blindly restarting the syscall will work as expected. However, if that
>> is the case, it means that we have quite a few syscalls to fix...
>>
> 
> This is why 2.5 introduces *_until() call forms for most timed syscalls,
> to specify absolute delays.

Ok. But the one with fixed relative delays can be fixed too, doing like
nanosleep: in case of -EINTR, copy back to user-space the next delay in
case the syscall is restarted.

-- 
					    Gilles.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Sat, 2009-05-02 at 18:27 +0200, Gilles Chanteperdrix wrote:
> Philippe Gerum wrote:
> > On Thu, 2009-04-30 at 17:27 +0200, Gilles Chanteperdrix wrote:
> >> Philippe Gerum wrote:
> >>> On Thu, 2009-04-30 at 07:09 -0700, Thomas Lockhart wrote:
> >>>>> Hello, I have a problem with rtdm_event_timedwait() function. It
> >>>>> sometimes ends prematurely with -EINTR error code. It depends on
> >>>>> special circumstances (CPU load, resizing console window in X, ...).
> >>>> In other areas of Xenomai when one sees a EINTR return one should
> >>>> reenter the original call. Like this (?):
> >>>>
> >>>> do {
> >>>>   err = rt_task_wait_period(...)
> >>>> } while (err != -EINTR);
> >>>>
> >>> Just to clarify a few points more:
> >>>
> >>> Xenomai always tells the kernel that blocking (Xenomai) syscalls
> >>> _should_ be restarted when interrupted by a Linux signal (i.e.
> >>> -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
> >>> case), and the kernel will actually restart that (Xenomai) syscall if no
> >>> handler was installed for such signal, or if SA_RESTART is set in the
> >>> sigaction() flags for the signal.
> >>>
> >>> In all other cases, the syscall will not be restarted despite the
> >>> "should restart" hint passed to the Linux kernel, and -EINTR will be
> >>> eventually returned by the native API syscall (or errno == EINTR for the
> >>> POSIX skin). The way other skins (e.g. VxWorks) will propagate the
> >>> -EINTR condition on interrupted syscalls to the user program depends on
> >>> their respective API conventions, but they will propagate that
> >>> information in a way or another as well, if the syscall was not
> >>> restarted.
> >>>
> >>> Additionally, syscalls may be forcibly unblocked via rt_task_unblock(),
> >>> which will cause -EINTR to be returned to the caller as well. This is
> >>> totally unrelated to Linux signal management though.
> >> Are not there cases where the kernel/libc "jumps back" to the beginning
> >> of the syscall when asked to restart a syscall ?
> > 
> > Yes, that is how a restart is done, simply by moving the instruction
> > pointer back.
> > 
> >>  Is not it what causes
> >> the problem with rt_event_wait ?
> >>  In fact if it is the problem then it
> >> means that the user-space skins should always convert delays to an
> >> absolute expiry date, and pass this as argument to the syscall, so that
> >> blindly restarting the syscall will work as expected. However, if that
> >> is the case, it means that we have quite a few syscalls to fix...
> >>
> > 
> > This is why 2.5 introduces *_until() call forms for most timed syscalls,
> > to specify absolute delays.
> 
> Ok. But the one with fixed relative delays can be fixed too, doing like
> nanosleep: in case of -EINTR, copy back to user-space the next delay in
> case the syscall is restarted.
> 

I'm not saying this can't be fixed, I'm just saying that this is not
mandatory to fix them to have proper behavior based on the new *_until()
forms.

-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Petr Cervenka]

Gilles Chanteperdrix wrote:
>Philippe Gerum wrote:
>> On Thu, 2009-04-30 at 07:09 -0700, Thomas Lockhart wrote:
>>>> Hello, I have a problem with rtdm_event_timedwait() function. It
>>>> sometimes ends prematurely with -EINTR error code. It depends on
>>>> special circumstances (CPU load, resizing console window in X, ...).

>> 
>> Just to clarify a few points more:
>> 
>> Xenomai always tells the kernel that blocking (Xenomai) syscalls
>> _should_ be restarted when interrupted by a Linux signal (i.e.
>> -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
>> case), and the kernel will actually restart that (Xenomai) syscall if no
>> handler was installed for such signal, or if SA_RESTART is set in the
>> sigaction() flags for the signal.
>> 

>> 
>> Additionally, syscalls may be forcibly unblocked via rt_task_unblock(),
>> which will cause -EINTR to be returned to the caller as well. This is
>> totally unrelated to Linux signal management though.
 the case, it means that we have quite a few syscalls to fix...
>

It seems that rt_task_unblock is not called, because the rt_event_wait is restarted.

>Also, from your mail, it is not clear to me, what is Petr problem,
>rtdm_event_timedwait ? or rt_event_wait.
>

I have ported a device driver fragment to rtdm. I have a problem with rtdm_event_timedwait() function that sometimes returns -EINTR.
In that driver (in IOCTL handling), when a signal is received the -ERESTARTSYS is returned. So the syscall is restarted and takes more time.
Similar situation is in reading from device (using DMA). But when the signal is received there (instead of waiting for interrupt), error happens.
So I wanted to find the source of the -EINTR return values. I created a simple example with alternative function (rt_event_wait()) to examine it's behavior.

I can perhaps fix it by calling rtdm_event_timedwait() again when a signal is received. But do you have any advice how to prevent the signals to happen or what is causing them?

Petr

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Mon, 2009-05-04 at 10:24 +0200, Petr Cervenka wrote:
> Gilles Chanteperdrix wrote:
> >Philippe Gerum wrote:
> >> On Thu, 2009-04-30 at 07:09 -0700, Thomas Lockhart wrote:
> >>>> Hello, I have a problem with rtdm_event_timedwait() function. It
> >>>> sometimes ends prematurely with -EINTR error code. It depends on
> >>>> special circumstances (CPU load, resizing console window in X, ...).
> 
> >> 
> >> Just to clarify a few points more:
> >> 
> >> Xenomai always tells the kernel that blocking (Xenomai) syscalls
> >> _should_ be restarted when interrupted by a Linux signal (i.e.
> >> -ERESTARTSYS is passed down to the kernel by the Xenomai core in that
> >> case), and the kernel will actually restart that (Xenomai) syscall if no
> >> handler was installed for such signal, or if SA_RESTART is set in the
> >> sigaction() flags for the signal.
> >> 
> 
> >> 
> >> Additionally, syscalls may be forcibly unblocked via rt_task_unblock(),
> >> which will cause -EINTR to be returned to the caller as well. This is
> >> totally unrelated to Linux signal management though.
>  the case, it means that we have quite a few syscalls to fix...
> >
> 
> It seems that rt_task_unblock is not called, because the rt_event_wait is restarted.
> 
> >Also, from your mail, it is not clear to me, what is Petr problem,
> >rtdm_event_timedwait ? or rt_event_wait.
> >
> 
> I have ported a device driver fragment to rtdm. I have a problem with rtdm_event_timedwait() function that sometimes returns -EINTR.
> In that driver (in IOCTL handling), when a signal is received the -ERESTARTSYS is returned. So the syscall is restarted and takes more time.
> Similar situation is in reading from device (using DMA). But when the signal is received there (instead of waiting for interrupt), error happens.
> So I wanted to find the source of the -EINTR return values. I created a simple example with alternative function (rt_event_wait()) to examine it's behavior.
> 
> I can perhaps fix it by calling rtdm_event_timedwait() again when a signal is received. But do you have any advice how to prevent the signals to happen or what is causing them?
> 

Assuming you are using the native API (the POSIX interface would have to
be adapted the same way), does the following patch help?

diff --git a/include/asm-generic/bits/bind.h b/include/asm-generic/bits/bind.h
index 50db3d3..7a05e81 100644
--- a/include/asm-generic/bits/bind.h
+++ b/include/asm-generic/bits/bind.h
@@ -76,7 +76,7 @@ xeno_bind_skin(unsigned skin_magic, const char *skin, const char *module)
 
 	sa.sa_handler = &xeno_handle_mlock_alert;
 	sigemptyset(&sa.sa_mask);
-	sa.sa_flags = 0;
+	sa.sa_flags = SA_RESTART;
 	sigaction(SIGXCPU, &sa, NULL);
 
 	return muxid;
diff --git a/src/skins/native/task.c b/src/skins/native/task.c
index 869a264..368c883 100644
--- a/src/skins/native/task.c
+++ b/src/skins/native/task.c
@@ -58,6 +58,7 @@ static void *rt_task_trampoline(void *cookie)
 	struct rt_task_iargs *iargs = (struct rt_task_iargs *)cookie;
 	void (*entry) (void *cookie);
 	struct sched_param param;
+	struct sigaction sa, osa;
 	struct rt_arg_bulk bulk;
 	long err;
 
@@ -74,7 +75,11 @@ static void *rt_task_trampoline(void *cookie)
 	/* rt_task_delete requires asynchronous cancellation */
 	pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
 
-	old_sigharden_handler = signal(SIGHARDEN, &rt_task_sigharden);
+	sa.sa_handler = &rt_task_sigharden;
+	sigemptyset(&sa.sa_mask);
+	sa.sa_flags = SA_RESTART;
+	sigaction(SIGHARDEN, &sa, &osa);
+	old_sigharden_handler = osa.sa_handler;
 
 	bulk.a1 = (u_long)iargs->task;
 	bulk.a2 = (u_long)iargs->name;
@@ -168,11 +173,16 @@ int rt_task_shadow(RT_TASK *task, const char *name, int prio, int mode)
 {
 	struct sched_param param;
 	struct rt_arg_bulk bulk;
+	struct sigaction sa, osa;
 
 	/* rt_task_delete requires asynchronous cancellation */
 	pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
 
-	old_sigharden_handler = signal(SIGHARDEN, &rt_task_sigharden);
+	sa.sa_handler = &rt_task_sigharden;
+	sigemptyset(&sa.sa_mask);
+	sa.sa_flags = SA_RESTART;
+	sigaction(SIGHARDEN, &sa, &osa);
+	old_sigharden_handler = osa.sa_handler;
 
 	if (prio > 0) {
 		/* Make sure the POSIX library caches the right priority. */

> Petr
> 
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Petr Cervenka]

>Assuming you are using the native API (the POSIX interface would have to
>be adapted the same way), does the following patch help?
>

The patch doesn't make any difference. It could be because I use mostly rtdm skin in the driver, not native. And the native skin example already resets syscall (but with wrong timeout).

Meanwhile I have found out the type of the signal, which interrupts my syscalls. It's SIGWINCH (or perhaps SIGHARDEN). What is exactly this signal used for, should it be interrupting syscalls?

Petr

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Mon, 2009-05-04 at 16:20 +0200, Petr Cervenka wrote:
> >Assuming you are using the native API (the POSIX interface would have to
> >be adapted the same way), does the following patch help?
> >
> 
> The patch doesn't make any difference. It could be because I use mostly rtdm skin in the driver, not native. And the native skin example already resets syscall (but with wrong timeout).
> 
> Meanwhile I have found out the type of the signal, which interrupts my syscalls. It's SIGWINCH (or perhaps SIGHARDEN). What is exactly this signal used for, should it be interrupting syscalls?

This is a signal internally used by skins to force a thread in primary
mode; it should not do interrupt syscalls, provided we set the sigaction
flags properly.

> 
> Petr
> 
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Mon, 2009-05-04 at 16:22 +0200, Philippe Gerum wrote:
> On Mon, 2009-05-04 at 16:20 +0200, Petr Cervenka wrote:
> > >Assuming you are using the native API (the POSIX interface would have to
> > >be adapted the same way), does the following patch help?
> > >
> > 
> > The patch doesn't make any difference. It could be because I use mostly rtdm skin in the driver, not native. And the native skin example already resets syscall (but with wrong timeout).
> > 
> > Meanwhile I have found out the type of the signal, which interrupts my syscalls. It's SIGWINCH (or perhaps SIGHARDEN). What is exactly this signal used for, should it be interrupting syscalls?
> 
> This is a signal internally used by skins to force a thread in primary
> mode; it should not do interrupt syscalls, provided we set the sigaction
> flags properly.
> 

Read: it must interrupt Xenomai syscalls internally but the latter
should always be silently restarted, by properly setting the SA_RESTART
bit in the sigaction flags for that signal.

Not allowing Linux to interrupt blocking Xenomai syscalls to process
pending signals would basically kill GDB support, and beyond this create
a very fragile situation wrt signal handling and Xenomai, which is the
last thing we would want to do.

> > 
> > Petr
> > 
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Petr Cervenka]

>
>Read: it must interrupt Xenomai syscalls internally but the latter
>should always be silently restarted, by properly setting the SA_RESTART
>bit in the sigaction flags for that signal.
>
>Not allowing Linux to interrupt blocking Xenomai syscalls to process
>pending signals would basically kill GDB support, and beyond this create
>a very fragile situation wrt signal handling and Xenomai, which is the
>last thing we would want to do.
>
>Philippe.
>

I have found following problems / errors:
--------------------------------------------------------------
1) rtdm_event_timedwait() in ioctl handler is not automatically internally restarted (driver: rtdm, app. tasks: native)

2) computer hangs, when I call in ioctl handler something like:
do {
  res = rtdm_event_timedwait(&event, timeout, NULL);
} while (res == -EINTR);
this could have perhaps some relevance: https://mail.gna.org/public/xenomai-help/2008-11/msg00025.html

3) rt_event_wait() in application is restarted always with the same relative timeout. it could run forever.

problems with version 2.5-rc1:
-----------------------------------------------
1) function rt_task_shadow always returns -EFAULT

2) when resizing window with latency test running, it prints out "Not SIGSHADOW !". But it should be SIGSHADOW (in my opinion).

Petr

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Gilles Chanteperdrix]

Petr Cervenka wrote:
>> Read: it must interrupt Xenomai syscalls internally but the latter 
>> should always be silently restarted, by properly setting the
>> SA_RESTART bit in the sigaction flags for that signal.
>> 
>> Not allowing Linux to interrupt blocking Xenomai syscalls to
>> process pending signals would basically kill GDB support, and
>> beyond this create a very fragile situation wrt signal handling and
>> Xenomai, which is the last thing we would want to do.
>> 
>> Philippe.
>> 
> 
> I have found following problems / errors: 
> -------------------------------------------------------------- 1)
> rtdm_event_timedwait() in ioctl handler is not automatically
> internally restarted (driver: rtdm, app. tasks: native)

That is because rtdm_event_timedwait is not a syscall. It is a
kernel-space call. Your job is to get ioctl to return -EINTR when
rtdm_event_timedwait returns -EINTR, so that ioctl will be restarted in
user-space.

This is undocumented, because you should already know about that if you
have done a little driver work under linux.

> 
> 2) computer hangs, when I call in ioctl handler something like: do { 
> res = rtdm_event_timedwait(&event, timeout, NULL); } while (res ==
> -EINTR); this could have perhaps some relevance:
> https://mail.gna.org/public/xenomai-help/2008-11/msg00025.html

That is because you should return to user-space to get the signal handled.

> 
> 3) rt_event_wait() in application is restarted always with the same
> relative timeout. it could run forever.

Yes, we know that, and we know a fix. But another fix is to use the new
*_until calls.

> 
> problems with version 2.5-rc1: 
> ----------------------------------------------- 1) function
> rt_task_shadow always returns -EFAULT

Could you provide us with a small example which has this problem ?

> 
> 2) when resizing window with latency test running, it prints out "Not
> SIGSHADOW !". But it should be SIGSHADOW (in my opinion).

"Not SIGSHADOW!" is printed when the SIGWINCH signal handler identifies
a SIGWINCH not originating from Xenomai. And when you resize the window,
a SIGWINCH is sent to the application which originates from the Linux
kernel, not from Xenomai. So, the "Not SIGSHADOW!" is correct. However,
it is essentially a debug message

> 
> Petr
> 
> 
> _______________________________________________ Xenomai-help mailing
> list Xenomai-help@domain.hid https://mail.gna.org/listinfo/xenomai-help


-- 
                                                 Gilles.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Philippe Gerum]

On Mon, 2009-05-04 at 18:11 +0200, Petr Cervenka wrote:
> >
> >Read: it must interrupt Xenomai syscalls internally but the latter
> >should always be silently restarted, by properly setting the SA_RESTART
> >bit in the sigaction flags for that signal.
> >
> >Not allowing Linux to interrupt blocking Xenomai syscalls to process
> >pending signals would basically kill GDB support, and beyond this create
> >a very fragile situation wrt signal handling and Xenomai, which is the
> >last thing we would want to do.
> >
> >Philippe.
> >
> 
> I have found following problems / errors:
> --------------------------------------------------------------
> 1) rtdm_event_timedwait() in ioctl handler is not automatically internally restarted (driver: rtdm, app. tasks: native)
> 
> 2) computer hangs, when I call in ioctl handler something like:
> do {
>   res = rtdm_event_timedwait(&event, timeout, NULL);
> } while (res == -EINTR);

>From any kernel space API called from userland syscalls:

   res = whatever_xenomai_skin_blocking_call(...);
   if (res == -EINTR)
	return -EINTR;

> this could have perhaps some relevance: https://mail.gna.org/public/xenomai-help/2008-11/msg00025.html
> 
> 3) rt_event_wait() in application is restarted always with the same relative timeout. it could run forever.
> 

As discussed earlier, using rt_event_wait_until() with an absolute
timeout spec would fix this (2.5 only though).

> problems with version 2.5-rc1:
> -----------------------------------------------
> 1) function rt_task_shadow always returns -EFAULT
> 

Are you passing a NULL task pointer?

> 2) when resizing window with latency test running, it prints out "Not SIGSHADOW !". But it should be SIGSHADOW (in my opinion).
> 
> Petr
> 
-- 
Philippe.

Re: [Xenomai-help] rtdm_event_timedwait returns -EINTR

[Petr Cervenka]

[-- Attachment #1: Type: text/plain, Size: 2436 bytes --]

From: gilles.chanteperdrix@xenomai.org
>Petr Cervenka wrote:

>> I have found following problems / errors: 
>> -------------------------------------------------------------- 1)
>> rtdm_event_timedwait() in ioctl handler is not automatically
>> internally restarted (driver: rtdm, app. tasks: native)
>
>That is because rtdm_event_timedwait is not a syscall. It is a
>kernel-space call. Your job is to get ioctl to return -EINTR when
>rtdm_event_timedwait returns -EINTR, so that ioctl will be restarted in
>user-space.
>
>This is undocumented, because you should already know about that if you
>have done a little driver work under linux.
>

Finally I start to understand it. Sorry, but I wrote only couple of drivers. With 
knowledge of hardware but not linux.

>> 
>> 2) computer hangs, when I call in ioctl handler something like: do { 
>> res = rtdm_event_timedwait(&event, timeout, NULL); } while (res ==
>> -EINTR); this could have perhaps some relevance:
>> https://mail.gna.org/public/xenomai-help/2008-11/msg00025.html
>
>That is because you should return to user-space to get the signal handled.
>

So it is better to divide longer operation with device (for example initialization) 
into two ioctls. First one for registry settings and second one for waiting for 
ready state. Or am I wrong?

>> 
>> 3) rt_event_wait() in application is restarted always with the same
>> relative timeout. it could run forever.
>
>Yes, we know that, and we know a fix. But another fix is to use the new
>*_until calls.
>
>> 
>> problems with version 2.5-rc1: 
>> ----------------------------------------------- 1) function
>> rt_task_shadow always returns -EFAULT
>
>Could you provide us with a small example which has this problem ?
>

Example is in the attachment.

>> 
>> 2) when resizing window with latency test running, it prints out "Not
>> SIGSHADOW !". But it should be SIGSHADOW (in my opinion).
>
>"Not SIGSHADOW!" is printed when the SIGWINCH signal handler identifies
>a SIGWINCH not originating from Xenomai. And when you resize the window,
>a SIGWINCH is sent to the application which originates from the Linux
>kernel, not from Xenomai. So, the "Not SIGSHADOW!" is correct. However,
>it is essentially a debug message
>

Oh, it my fault. When I identified these signals as SIGWINCH, I automatically 
assumed that it's SIGHARDEN. I completely forgot to look at the basic mean 
of SIGWINCH signal. I feel little ashamed.

Petr


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: main.cpp --]
[-- Type: text/x-c++src; name="main.cpp", Size: 1390 bytes --]

#include <sys/mman.h>
#include <native/task.h>
#include <native/event.h>
#include <native/timer.h>

#define MY_EVENT    0x00000001ll
#define MY_PRIORITY 99

static RT_TASK mainTask;

int main(int argc, char** argv) {
    const double ONE_SEC = 1e9;
    const RTIME timeout = (RTIME)(5 * ONE_SEC);
    RT_EVENT event;
    unsigned long mask;
    RTIME start, end;
    int res;

    mlockall(MCL_CURRENT | MCL_FUTURE);

    res = rt_task_shadow(&mainTask, "main_task", MY_PRIORITY, 0);
    if (res < 0) {
        printf("rt_task_shadow(): %d (%s)\n", res, strerror(-res));
    }

    res = rt_task_set_mode(0, T_PRIMARY, NULL);
    if (res < 0) {
        printf("rt_task_set_mode(): %d (%s)\n", res, strerror(-res));
    }

    res = rt_event_create(&event, NULL, 0, EV_PRIO);
    if (res < 0) {
        printf("rt_event_create(): %d (%s)\n", res, strerror(-res));
    }

    start = rt_timer_read();
    res = rt_event_wait(&event, MY_EVENT, &mask, EV_ALL, rt_timer_ns2ticks(timeout));
    end = rt_timer_read();
    
    if (res < 0 && res != -ETIMEDOUT) {
        printf("rt_event_wait(): %d (%s)\n", res, strerror(-res));
    }

    printf("time - measured %g s, wanted %g s\n", rt_timer_ticks2ns(end - start)/ONE_SEC, timeout/ONE_SEC);
    
    res = rt_event_delete(&event);
    if (res < 0) {
        printf("rt_event_delete(): %d (%s)\n", res, strerror(-res));
    }

    return 0;
}