Re: [Xenomai] Xenomai-forge: thread using 100% cpu load

[Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>]

On 03/01/2013 09:30 AM, Philippe Gerum wrote:

> On 03/01/2013 09:26 AM, Gilles Chanteperdrix wrote:
>> On 03/01/2013 09:22 AM, Philippe Gerum wrote:
>>
>>> On 02/28/2013 09:22 PM, Thomas De Schampheleire wrote:
>>>> On Thu, Feb 28, 2013 at 9:10 PM, Gilles Chanteperdrix
>>>> <gilles.chanteperdrix@xenomai.org> wrote:
>>>>> On 02/28/2013 08:19 PM, Ronny Meeus wrote:
>>>>>
>>>>>> Hello
>>>>>>
>>>>>> we are using the PSOS interface of Xenomai forge, running completely
>>>>>> in user-space using the mercury code.
>>>>>> We deploy our application on different processors, one product is
>>>>>> running on PPC multicore (P4040, P4080, P4034) and another one on
>>>>>> Cavium (8 core device).
>>>>>> The Linux version we use is 2.6.32 but I would assume that this is not
>>>>>> so relevant.
>>>>>>
>>>>>> Our Xenomai application is running on one of the cores (affinity is
>>>>>> set), while the other cores are running other code.
>>>>>>
>>>>>> On both architectures we recently start to see issues that one thread
>>>>>> is consuming 100% of the core on which the application is pinned.
>>>>>> The thread that monopolizes the core is the thread internally used to
>>>>>> manage the timers, running at the highest priority.
>>>>>> The trigger for running into this behavior is currently unclear.
>>>>>> If we only start a part of the application (platform management only),
>>>>>> the issue is not observed.
>>>>>> We see this on both an old version of Xenomai and a very recent one
>>>>>> (pulled from the git repo yesterday).
>>>>>>
>>>>>> I will continue to debug this issue in the coming days and try isolate
>>>>>> the code that is triggering it, but I can use hints from the
>>>>>> community.
>>>>>> Debugging is complex since once the load starts, the debugger is not
>>>>>> reacting anymore.
>>>>>> If I put breakpoints in the functions that are called when the timer
>>>>>> expires (both oneshot and periodic), the process starts to clone
>>>>>> itself and I endup with tens of them.
>>>>>>
>>>>>> Has anybody seen an issue like this before or does somebody has some
>>>>>> hints on how to debug this problem?
>>>>>
>>>>>
>>>>> First enable the watchdog. It will send a signal to the application when
>>>>> detecting a problem, then you can use the watchdog to trigger an I-pipe
>>>>> tracer trace when the bug happens. You will probably have to increase
>>>>> the watchdog polling frequency, in order to have a meaningful trace.
>>>>>
>>>>
>>>> I don't think an I-pipe tracer will be possible when using the Mercury
>>>> core, right (xenomai-forge) ?
>>>>
>>>
>>> Correct.
>>
>>
>> I do not think so. The way I see it, you can enable the I-pipe tracer
>> without CONFIG_XENOMAI.
>>
> 
> Mercury has NO pipeline in the kernel.
> 

You mean mercury can not run with an I-pipe kernel?

-- 
                                                                Gilles.