[Xenomai-help] Hang at glibc time() function call

[Xenomai-help] Hang at glibc time() function call

[Petr Cervenka]

Hello,
we have a problems with hanging of our real-time application. In fact, the computer stops to respond totally.
I succeeded to find the exact hang position in couple of our tasks (but not all of them) by setting of some digital outputs of HW device:
1) The most real-time task with the highest priority is still running as it should be (100us period).
2) Some of the tasks are in "sleep" state. They are waiting on some event or timeout.
3) One low priority task seems to be blocked at glibc time() function call.

But I would like to know if it is the source of the problem or only a consequent.
I will try to replace the time() function call with rt_timer_read(), but couldn't be the problem elsewhere?

Configuration:
Athlon XP 4600+
linux-2.6.26.3
xenomai-2.4.5 + patch for proper queue deletion

Thanks for any advice.
Petr Cervenka

Re: [Xenomai-help] Hang at glibc time() function call

[Gilles Chanteperdrix]

Petr Cervenka wrote:
> Hello, we have a problems with hanging of our real-time application.
> In fact, the computer stops to respond totally. I succeeded to find
> the exact hang position in couple of our tasks (but not all of them)
> by setting of some digital outputs of HW device: 1) The most
> real-time task with the highest priority is still running as it
> should be (100us period). 2) Some of the tasks are in "sleep" state.
> They are waiting on some event or timeout. 3) One low priority task
> seems to be blocked at glibc time() function call.
> 
> But I would like to know if it is the source of the problem or only a
> consequent. I will try to replace the time() function call with
> rt_timer_read(), but couldn't be the problem elsewhere?
> 
> Configuration: Athlon XP 4600+ linux-2.6.26.3 xenomai-2.4.5 + patch
> for proper queue deletion

Could not it be that the highest priority task is using all the CPU?

-- 
                                                 Gilles.

Re: [Xenomai-help] Hang at glibc time() function call

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

Petr Cervenka wrote:
> Hello,
> we have a problems with hanging of our real-time application. In fact, the computer stops to respond totally.
> I succeeded to find the exact hang position in couple of our tasks (but not all of them) by setting of some digital outputs of HW device:
> 1) The most real-time task with the highest priority is still running as it should be (100us period).
> 2) Some of the tasks are in "sleep" state. They are waiting on some event or timeout.
> 3) One low priority task seems to be blocked at glibc time() function call.

Is the low priority task also mapped as Xenomai task? In that case
calling standard Linux time services can be lethal:

If the Linux clocksource is TSC or HPET, glibc will use a syscall-less
variant for time of day reading. That variant has two properties:

A) It doesn't switch Xenomai task into secondary mode like ordinary
   syscalls do.

B) It can deadlock with the Linux kernel, leaving the caller spinning
   forever so that Linux is no longer executed.

The testsuite's clocktest used to have this issue as well (luckily, I
was able to analyze it over KVM with a source level debugger :) ).

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] Hang at glibc time() function call

[Petr Cervenka]

>Could not it be that the highest priority task is using all the CPU?

No, it's highly unprobable. The task runs at is should.

>Is the low priority task also mapped as Xenomai task? In that case
>calling standard Linux time services can be lethal:

Yes, the task is mapped as Xenomai task, because it waits on real-time queue with some timeout.

>
>If the Linux clocksource is TSC or HPET, glibc will use a syscall-less
>variant for time of day reading. That variant has two properties:
>
>A) It doesn't switch Xenomai task into secondary mode like ordinary
>   syscalls do.
>
>B) It can deadlock with the Linux kernel, leaving the caller spinning
>   forever so that Linux is no longer executed.

Hmm, probably my cause.

Meanwhile I replaced the glibc time() function call with rt_timer_read and it seems to work fine now (for 3 days).

Thank you for you support.
Petr