Re: [Xenomai] ipipe: fix domain migration helper

Re: [Xenomai] ipipe: fix domain migration helper

[Jan Kiszka]

Hi Philippe,

a first heads-up, maybe you have an idea before I start debugging: That
patch causes troubles when our application shuts down. The main thread
hangs like this, unkillable:

# cat /proc/xenomai/sched
CPU  PID    CLASS  PRI      TIMEOUT   TIMEBASE   STAT       NAME
  0  0      idle    -1      -         master     R          ROOT/0
  1  0      idle    -1      -         master     R          ROOT/1
  2  0      idle    -1      -         master     R          ROOT/2
  3  0      idle    -1      -         master     R          ROOT/3
  4  0      idle    -1      -         master     R          ROOT/4
  5  0      idle    -1      -         master     R          ROOT/5
  6  0      idle    -1      -         master     R          ROOT/6
  7  0      idle    -1      -         master     R          ROOT/7
  8  0      idle    -1      -         master     R          ROOT/8
  9  0      idle    -1      -         master     R          ROOT/9
 10  0      idle    -1      -         master     R          ROOT/10
 11  0      idle    -1      -         master     R          ROOT/11
  0  2435   rt       0      -         master     X          ShadowTask_2435
# cat /proc/2435/stack
[<ffffffff810b5b94>] xnpod_suspend_thread+0x434/0x570
[<ffffffff810beb30>] xnshadow_relax+0xe0/0x240
[<ffffffff810c08e3>] losyscall_event+0x1c3/0x280
[<ffffffff8109e73a>] ipipe_syscall_hook+0x5a/0x90
[<ffffffff8109c311>] __ipipe_notify_syscall+0x131/0x2e0
[<ffffffff8101e841>] __ipipe_syscall_root+0x41/0x160
[<ffffffff8126a41a>] __ipipe_syscall_root_thunk+0x35/0x67
[<ffffffffffffffff>] 0xffffffffffffffff

This was with your current core-3.5 branch.

I'm currently updating my virtual debug environment. The issue may also
be reproducible with standard Xenomai tests.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] ipipe: fix domain migration helper

[Philippe Gerum]

On 01/16/2013 04:14 PM, Jan Kiszka wrote:
> Hi Philippe,
>
> a first heads-up, maybe you have an idea before I start debugging: That
> patch causes troubles when our application shuts down. The main thread
> hangs like this, unkillable:
>

It looks like the LO_WAKEUP_REQ virtual interrupt does not reach the 
handler.

> # cat /proc/xenomai/sched
> CPU  PID    CLASS  PRI      TIMEOUT   TIMEBASE   STAT       NAME
>    0  0      idle    -1      -         master     R          ROOT/0
>    1  0      idle    -1      -         master     R          ROOT/1
>    2  0      idle    -1      -         master     R          ROOT/2
>    3  0      idle    -1      -         master     R          ROOT/3
>    4  0      idle    -1      -         master     R          ROOT/4
>    5  0      idle    -1      -         master     R          ROOT/5
>    6  0      idle    -1      -         master     R          ROOT/6
>    7  0      idle    -1      -         master     R          ROOT/7
>    8  0      idle    -1      -         master     R          ROOT/8
>    9  0      idle    -1      -         master     R          ROOT/9
>   10  0      idle    -1      -         master     R          ROOT/10
>   11  0      idle    -1      -         master     R          ROOT/11
>    0  2435   rt       0      -         master     X          ShadowTask_2435
> # cat /proc/2435/stack
> [<ffffffff810b5b94>] xnpod_suspend_thread+0x434/0x570
> [<ffffffff810beb30>] xnshadow_relax+0xe0/0x240
> [<ffffffff810c08e3>] losyscall_event+0x1c3/0x280
> [<ffffffff8109e73a>] ipipe_syscall_hook+0x5a/0x90
> [<ffffffff8109c311>] __ipipe_notify_syscall+0x131/0x2e0
> [<ffffffff8101e841>] __ipipe_syscall_root+0x41/0x160
> [<ffffffff8126a41a>] __ipipe_syscall_root_thunk+0x35/0x67
> [<ffffffffffffffff>] 0xffffffffffffffff
>
> This was with your current core-3.5 branch.
>
> I'm currently updating my virtual debug environment. The issue may also
> be reproducible with standard Xenomai tests.
>
> Jan
>


-- 
Philippe.

Re: [Xenomai] ipipe: fix domain migration helper

[Jan Kiszka]

On 2013-01-16 16:25, Philippe Gerum wrote:
> On 01/16/2013 04:14 PM, Jan Kiszka wrote:
>> Hi Philippe,
>>
>> a first heads-up, maybe you have an idea before I start debugging: That
>> patch causes troubles when our application shuts down. The main thread
>> hangs like this, unkillable:
>>
> 
> It looks like the LO_WAKEUP_REQ virtual interrupt does not reach the 
> handler.

Hmm, lostage queue overflow? We used to carry some detection patch (to
make the existing detector unconditional), not sure what happened to it.
Will dig in this direction first.

That said, the system runs fine with the patch removed.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] ipipe: fix domain migration helper

[Jan Kiszka]

On 2013-01-16 16:30, Jan Kiszka wrote:
> On 2013-01-16 16:25, Philippe Gerum wrote:
>> On 01/16/2013 04:14 PM, Jan Kiszka wrote:
>>> Hi Philippe,
>>>
>>> a first heads-up, maybe you have an idea before I start debugging: That
>>> patch causes troubles when our application shuts down. The main thread
>>> hangs like this, unkillable:
>>>
>>
>> It looks like the LO_WAKEUP_REQ virtual interrupt does not reach the 
>> handler.
> 
> Hmm, lostage queue overflow? We used to carry some detection patch (to
> make the existing detector unconditional), not sure what happened to it.
> Will dig in this direction first.

No queue overflow. And you can immediately trigger it on x86-64 with
latency e.g. I have this now in my VM, but a different bug on real HW is
calling which has higher prio. Will come back to this ASAP.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] ipipe: fix domain migration helper

[Jan Kiszka]

On 2013-01-16 16:52, Jan Kiszka wrote:
> @@ -1095,12 +1120,8 @@ int __ipipe_switch_tail(void)
>  	hard_local_irq_disable();
>  #endif
>  	x = __ipipe_root_p;
> -
> -#ifdef CONFIG_IPIPE_LEGACY
> -	current->state &= ~TASK_HARDENING;
> -#else
> -	complete_domain_migration();
> -#endif	/* !CONFIG_IPIPE_LEGACY */
> +	if (x)
> +		complete_domain_migration();

This seems wrong, at least for legacy: Now we no longer clear
TASK_HARDENING after the switch back from the gatekeeper because the
current domain is Xenomai then. Does the new migration scheme work
differently in this regard? Then we will have to keep the #ifdef.

Do I get this correctly that the patch did not target an issue with
legacy usage? Or is there still a problem that could also affect Xenomai
2.6?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux