[Xenomai-core] [PowerPC] Registers Corruption at Context Switch

[Xenomai-core] [PowerPC] Registers Corruption at Context Switch

[Benjamin ZORES]

Hi,

I'm facing a problem with the PowerPC version of Xenomai/Adeos that I 
have difficulties
to identify the exact source.

I'm running a Xenomai RT kernel thread that use to crash sometimes due 
to potential register corruption.
Problem occurs after a context switch and, in some cases, if the task 
gets interrupted and reschedule,
its registers values are not the same as they used to be before context 
switch.

The code is a bit complex and so, makes use of register that is 
generally rarely used (GPR r26 to be accurate).
Driver is compiled with -02 and compiling with -O0 (so disabling 
optimizations and so, not using r26) works fine
but is not what I'm looking for.

Can someone tell me where exactly in Adeos/Xenomai is context switching 
actually performed and where are the registers save/restore functions ? 
I've seen there is specific code for FPU registers handling but can't 
find the equivalent for GPR.

FYI, I'm running on PowerPC 603e core with Linux 2.6.23, Adeos 2.0-09 
(latest) and Xenomai 2.3.4 (latest).
I've seen there are adeos updates (but for updated kernels) but is there 
some ChangeLog of Adeos changes ?
Maybe this is a known bug that has been fixed in updated Adeos release ?

Thx to anyone who can help me on this,

Ben

Re: [Xenomai-core] [PowerPC] Registers Corruption at Context Switch

[Philippe Gerum]

Benjamin ZORES wrote:
> Hi,
> 
> I'm facing a problem with the PowerPC version of Xenomai/Adeos that I 
> have difficulties
> to identify the exact source.
> 
> I'm running a Xenomai RT kernel thread that use to crash sometimes due 
> to potential register corruption.
> Problem occurs after a context switch and, in some cases, if the task 
> gets interrupted and reschedule,
> its registers values are not the same as they used to be before context 
> switch.
> 
> The code is a bit complex and so, makes use of register that is 
> generally rarely used (GPR r26 to be accurate).
> Driver is compiled with -02 and compiling with -O0 (so disabling 
> optimizations and so, not using r26) works fine
> but is not what I'm looking for.
> 
> Can someone tell me where exactly in Adeos/Xenomai is context switching 
> actually performed and where are the registers save/restore functions ? 
> I've seen there is specific code for FPU registers handling but can't 
> find the equivalent for GPR.
> 
> FYI, I'm running on PowerPC 603e core with Linux 2.6.23, Adeos 2.0-09 
> (latest) and Xenomai 2.3.4 (latest).
> I've seen there are adeos updates (but for updated kernels) but is there 
> some ChangeLog of Adeos changes ?
> Maybe this is a known bug that has been fixed in updated Adeos release ?
> 
> Thx to anyone who can help me on this,
> 

See arch/powerpc/switch_32.S, rthal_switch_threads(), for the part that does the
actual stack switching.

Note that this code is obfuscated by the fact that we have to handle so-called
"hybrid" switching, between Xenomai kernel threads (which do not rely on a
task_struct), and Linux tasks (Xenomai userland, Linux kthreads, or regular
userland Linux). Fortunately, what is saved on the stack in any case is easy to
find out.


> Ben
> 
> 
> _______________________________________________
> Xenomai-core mailing list
> Xenomai-core@domain.hid
> https://mail.gna.org/listinfo/xenomai-core
> 


-- 
Philippe.

Re: [Xenomai-core] [PowerPC] Registers Corruption at Context Switch

[Benjamin ZORES]

Philippe Gerum a écrit :
> See arch/powerpc/switch_32.S, rthal_switch_threads(), for the part that does the
> actual stack switching.
>
> Note that this code is obfuscated by the fact that we have to handle so-called
> "hybrid" switching, between Xenomai kernel threads (which do not rely on a
> task_struct), and Linux tasks (Xenomai userland, Linux kthreads, or regular
> userland Linux). Fortunately, what is saved on the stack in any case is easy to
> find out.
>   
Thx for the info.
Can you tell me why GPR registers would be saved there and FPU ones in 
another function ?

Ben

Re: [Xenomai-core] [PowerPC] Registers Corruption at Context Switch

[Benjamin ZORES]

Philippe Gerum a écrit :
>> FYI, I'm running on PowerPC 603e core with Linux 2.6.23, Adeos 2.0-09 
>> (latest) and Xenomai 2.3.4 (latest).
>>     
read Xenomai 2.4.4 here, of course ...
>
> See arch/powerpc/switch_32.S, rthal_switch_threads(), for the part that does the
> actual stack switching.
>
> Note that this code is obfuscated by the fact that we have to handle so-called
> "hybrid" switching, between Xenomai kernel threads (which do not rely on a
> task_struct), and Linux tasks (Xenomai userland, Linux kthreads, or regular
> userland Linux). Fortunately, what is saved on the stack in any case is easy to
> find out.
>   
Ok, I've dig a bit more at sources and found out something strange.

In xenomai arch/powerpc/xenomai/switch_32.S in rthal_thread_switch() we 
have:

********
#ifdef CONFIG_SMP
    sync
#endif /* CONFIG_SMP */

    lwz    r1,KSP(r4)    /* Load new stack pointer */

    mr    r3,r2
    lwz    r0,PGDIR(r4)
        cmpwi   r0, 0
        beq-    same_current

    tophys(r0,r4)
    CLR_TOP32(r0)
    mtspr    SPRN_SPRG3,r0    /* Update current THREAD phys addr */
    addi    r2,r4,-THREAD    /* Update current */

same_current:
**********

While, in arch/powerpc/kernel/entry_32.S in _switch() we have:

**********
#ifdef CONFIG_SMP
    /* We need a sync somewhere here to make sure that if the
     * previous task gets rescheduled on another CPU, it sees all
     * stores it has performed on this one.
     */
    sync
#endif /* CONFIG_SMP */

    tophys(r0,r4)
    CLR_TOP32(r0)
    mtspr    SPRN_SPRG3,r0    /* Update current THREAD phys addr */
    lwz    r1,KSP(r4)    /* Load new stack pointer */

    /* save the old current 'last' for return value */
    mr    r3,r2
    addi    r2,r4,-THREAD    /* Update current */
************

As we can see, the code differs from kernel, as

    tophys(r0,r4)
    CLR_TOP32(r0)
    mtspr    SPRN_SPRG3,r0    /* Update current THREAD phys addr */

is done _before_ loading new stack pointer in kernel and _after_ doing 
so in xenomai.

Is there a good reason for that or is this unintended ??

Ben

Re: [Xenomai-core] [PowerPC] Registers Corruption at Context Switch

[Philippe Gerum]

Benjamin ZORES wrote:
> Philippe Gerum a écrit :
>>> FYI, I'm running on PowerPC 603e core with Linux 2.6.23, Adeos 2.0-09
>>> (latest) and Xenomai 2.3.4 (latest).
>>>     
> read Xenomai 2.4.4 here, of course ...
>>
>> See arch/powerpc/switch_32.S, rthal_switch_threads(), for the part
>> that does the
>> actual stack switching.
>>
>> Note that this code is obfuscated by the fact that we have to handle
>> so-called
>> "hybrid" switching, between Xenomai kernel threads (which do not rely
>> on a
>> task_struct), and Linux tasks (Xenomai userland, Linux kthreads, or
>> regular
>> userland Linux). Fortunately, what is saved on the stack in any case
>> is easy to
>> find out.
>>   
> Ok, I've dig a bit more at sources and found out something strange.
> 
> In xenomai arch/powerpc/xenomai/switch_32.S in rthal_thread_switch() we
> have:
> 
> ********
> #ifdef CONFIG_SMP
>    sync
> #endif /* CONFIG_SMP */
> 
>    lwz    r1,KSP(r4)    /* Load new stack pointer */
> 
>    mr    r3,r2
>    lwz    r0,PGDIR(r4)
>        cmpwi   r0, 0
>        beq-    same_current
> 
>    tophys(r0,r4)
>    CLR_TOP32(r0)
>    mtspr    SPRN_SPRG3,r0    /* Update current THREAD phys addr */
>    addi    r2,r4,-THREAD    /* Update current */
> 
> same_current:
> **********
> 
> While, in arch/powerpc/kernel/entry_32.S in _switch() we have:
> 
> **********
> #ifdef CONFIG_SMP
>    /* We need a sync somewhere here to make sure that if the
>     * previous task gets rescheduled on another CPU, it sees all
>     * stores it has performed on this one.
>     */
>    sync
> #endif /* CONFIG_SMP */
> 
>    tophys(r0,r4)
>    CLR_TOP32(r0)
>    mtspr    SPRN_SPRG3,r0    /* Update current THREAD phys addr */
>    lwz    r1,KSP(r4)    /* Load new stack pointer */
> 
>    /* save the old current 'last' for return value */
>    mr    r3,r2
>    addi    r2,r4,-THREAD    /* Update current */
> ************
> 
> As we can see, the code differs from kernel, as
> 
>    tophys(r0,r4)
>    CLR_TOP32(r0)
>    mtspr    SPRN_SPRG3,r0    /* Update current THREAD phys addr */
> 
> is done _before_ loading new stack pointer in kernel and _after_ doing
> so in xenomai.
> 
> Is there a good reason for that or is this unintended ??
> 

It's just code placement to avoid additional branches depending on whether we
want to update SPRG3 upon switch or not (when switching to a Xenomai kernel
thread, we don't). I see no dependency from that code on the stack pointer and
conversely. Do you see any?

-- 
Philippe.

Re: [Xenomai-core] [PowerPC] Registers Corruption at Context Switch

[Philippe Gerum]

Benjamin ZORES wrote:
> Philippe Gerum a écrit :
>> See arch/powerpc/switch_32.S, rthal_switch_threads(), for the part that does the
>> actual stack switching.
>>
>> Note that this code is obfuscated by the fact that we have to handle so-called
>> "hybrid" switching, between Xenomai kernel threads (which do not rely on a
>> task_struct), and Linux tasks (Xenomai userland, Linux kthreads, or regular
>> userland Linux). Fortunately, what is saved on the stack in any case is easy to
>> find out.
>>   
> Thx for the info.
> Can you tell me why GPR registers would be saved there and FPU ones in 
> another function ?
> 

Because FPU management with Xenomai involves additional handling, e.g. FPU state
fixup during primary/secondary mode switch, Linux to Xenomai real-time
transitions. That support has to be provided independently from the pure task
switching code.

-- 
Philippe.