[PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Sebastian Siewior]

From: Thiemo Seufer <ths@linutronix.de>

those two are requried on my fresh gcc 4.3.1

Signed-off-by: Thiemo Seufer <ths@linutronix.de>
Signed-off-by: Sebastian Siewior <bigeasy@linutronix.de>
---
Not sure if this is intendent or a gcc bug but with -mno-spe
the spe opcodes were not used floating point anymore but
for 64bit save/restore for instance.

 arch/powerpc/Makefile |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
index b7d4c4c..3727e4f 100644
--- a/arch/powerpc/Makefile
+++ b/arch/powerpc/Makefile
@@ -108,7 +108,10 @@ endif
 KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
 
 # No SPE instruction when building kernel
+# (We use all available options to help semi-broken compilers)
 KBUILD_CFLAGS += $(call cc-option,-mno-spe)
+KBUILD_CFLAGS += $(call cc-option,-mspe=no)
+KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)
 
 # Enable unit-at-a-time mode when possible. It shrinks the
 # kernel considerably.
-- 
1.5.6.5

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Nate Case]

On Mon, 2008-09-01 at 16:23 +0200, Sebastian Siewior wrote:
> those two are requried on my fresh gcc 4.3.1
> 
> Signed-off-by: Thiemo Seufer <ths@linutronix.de>
> Signed-off-by: Sebastian Siewior <bigeasy@linutronix.de>
> ---
> Not sure if this is intendent or a gcc bug but with -mno-spe
> the spe opcodes were not used floating point anymore but
> for 64bit save/restore for instance.

I wouldn't say this is due to a broken compiler. As I understand it,
-mabi=no-spe and -mspe=no serve two different purposes.  One is for
disabling the SPE instructions and the other controls the ABI (which
would make those 64-bit save/restores I'm guessing).  I don't know why
you'd ever want to use the SPE ABI without -mspe=yes, but gcc does
provide that flexibility.

	-mno-spe: Deprecated way to say "no SPE instructions"
	-mspe=no: New way to do -mno-spe
	-mabi=no-spe: Disable SPE ABI

Some compilers may enable "-mabi=spe" and/or "-mspe=yes" by default, so
explicitly disabling both is necessary.  I recently built a SPE
toolchain which enabled both by default, so I ran into the "SPE used in
kernel" problem when the kernel only passed "-mno-spe".
	
- Nate Case <ncase@xes-inc.com>

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Kumar Gala]

On Sep 1, 2008, at 9:23 AM, Sebastian Siewior wrote:

> From: Thiemo Seufer <ths@linutronix.de>
>
> those two are requried on my fresh gcc 4.3.1
>
> Signed-off-by: Thiemo Seufer <ths@linutronix.de>
> Signed-off-by: Sebastian Siewior <bigeasy@linutronix.de>
> ---
> Not sure if this is intendent or a gcc bug but with -mno-spe
> the spe opcodes were not used floating point anymore but
> for 64bit save/restore for instance.

what code is getting generated for you that is causing issue?

> arch/powerpc/Makefile |    3 +++
> 1 files changed, 3 insertions(+), 0 deletions(-)
>
> diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
> index b7d4c4c..3727e4f 100644
> --- a/arch/powerpc/Makefile
> +++ b/arch/powerpc/Makefile
> @@ -108,7 +108,10 @@ endif
> KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
>
> # No SPE instruction when building kernel
> +# (We use all available options to help semi-broken compilers)
> KBUILD_CFLAGS += $(call cc-option,-mno-spe)
> +KBUILD_CFLAGS += $(call cc-option,-mspe=no)

Why does -mno-spe work?

 From my gcc-4.3 info pages:

`-mspe=YES/NO'
      This option has been deprecated.  Use `-mspe' and `-mno-spe'
      instead.

> +KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)

is the -mabi=no-spe really needed?

- k

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Nate Case]

On Tue, 2008-10-14 at 14:25 -0500, Nate Case wrote:
> 
>         -mno-spe: Deprecated way to say "no SPE instructions"
>         -mspe=no: New way to do -mno-spe

Sorry, I got this backwards (as Kumar pointed out in his other reply).
-mspe=no is actually deprecated.

- Nate Case <ncase@xes-inc.com>

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Nate Case]

On Tue, 2008-10-14 at 16:23 -0500, Kumar Gala wrote:
> Why does -mno-spe work?
> 
>  From my gcc-4.3 info pages:
> 
> `-mspe=YES/NO'
>       This option has been deprecated.  Use `-mspe' and `-mno-spe'
>       instead.
> 
> > +KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)
> 
> is the -mabi=no-spe really needed?

My guess is that the -mabi=no-spe was the real key of what made it work
for him.  I went through the same thing with my toolchain.

You do need -mabi=no-spe if your toolchain defaults to -mabi=spe like
mine does.  I know that the more generic toolchains out there
(CodeSourcery, ELDK) default to -mabi=no-spe, so in that case it would
not be necessary.

I don't know what generated instructions are actually to blame, but I do
know that if you compile certain programs with "-mno-spe -mabi=spe" vs.
"-mno-spe -mabi=no-spe", the results will differ.  In the case of the
kernel, you'll get a bunch of "SPE used in kernel" messages with the
"-mno-spe -mabi=spe" combination.

- Nate Case <ncase@xes-inc.com>

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Sebastian Andrzej Siewior]

* Kumar Gala | 2008-10-14 16:23:05 [-0500]:

>>Not sure if this is intendent or a gcc bug but with -mno-spe
>>the spe opcodes were not used floating point anymore but
>>for 64bit save/restore for instance.
>
>what code is getting generated for you that is causing issue?
Without the patch do_syslog() looks like this:

|c0025840 <do_syslog>:
|c0025840:       94 21 ff 50     stwu    r1,-176(r1)
|c0025844:       7c 08 02 a6     mflr    r0
|c0025848:       13 21 73 21     evstdd  r25,112(r1)
|c002584c:       13 41 7b 21     evstdd  r26,120(r1)
|c0025850:       7c 9a 23 78     mr      r26,r4
|c0025854:       13 61 83 21     evstdd  r27,128(r1)
|c0025858:       7c bb 2b 78     mr      r27,r5
|c002585c:       13 e1 a3 21     evstdd  r31,160(r1)
|c0025860:       7c 7f 1b 78     mr      r31,r3
|c0025864:       90 01 00 b4     stw     r0,180(r1)
|c0025868:       12 21 33 21     evstdd  r17,48(r1)
|c002586c:       12 41 3b 21     evstdd  r18,56(r1)
|c0025870:       12 61 43 21     evstdd  r19,64(r1)
|c0025874:       12 81 4b 21     evstdd  r20,72(r1)
|c0025878:       12 a1 53 21     evstdd  r21,80(r1)
|c002587c:       12 c1 5b 21     evstdd  r22,88(r1)
|c0025880:       12 e1 63 21     evstdd  r23,96(r1)
|c0025884:       13 01 6b 21     evstdd  r24,104(r1)
|c0025888:       13 81 8b 21     evstdd  r28,136(r1)
|c002588c:       13 a1 93 21     evstdd  r29,144(r1)
|c0025890:       13 c1 9b 21     evstdd  r30,152(r1)
|c0025894:       48 0f 00 b5     bl      c0115948 <cap_syslog>
|c0025898:       7c 79 1b 79     mr.     r25,r3
|c002589c:       40 82 04 54     bne-    c0025cf0 <do_syslog+0x4b0>
|c00258a0:       2b 9f 00 0a     cmplwi  cr7,r31,10
|c00258a4:       41 9d 04 40     bgt-    cr7,c0025ce4 <do_syslog+0x4a4>
|c00258a8:       3d 20 c0 27     lis     r9,-16345
|c00258ac:       57 e0 10 3a     rlwinm  r0,r31,2,0,29
|c00258b0:       39 29 a6 e4     addi    r9,r9,-22812
|c00258b4:       7c 09 00 2e     lwzx    r0,r9,r0
|c00258b8:       7c 00 4a 14     add     r0,r0,r9
|c00258bc:       7c 09 03 a6     mtctr   r0
|c00258c0:       4e 80 04 20     bctr
|c00258c4:       3b e0 00 00     li      r31,0
....

do_syslog() is not the only one.

>>1 files changed, 3 insertions(+), 0 deletions(-)
>>
>>diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile
>>index b7d4c4c..3727e4f 100644
>>--- a/arch/powerpc/Makefile
>>+++ b/arch/powerpc/Makefile
>>@@ -108,7 +108,10 @@ endif
>>KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
>>
>># No SPE instruction when building kernel
>>+# (We use all available options to help semi-broken compilers)
>>KBUILD_CFLAGS += $(call cc-option,-mno-spe)
>>+KBUILD_CFLAGS += $(call cc-option,-mspe=no)
>
>Why does -mno-spe work?
Good question. Without this option and only with -mabi=no-spe the
do_syslog() looks like the following:

|c0025270 <do_syslog>:
|c0025270:       94 21 ff 90     stwu    r1,-112(r1)
|c0025274:       7c 08 02 a6     mflr    r0
|c0025278:       be 21 00 34     stmw    r17,52(r1)
|c002527c:       7c 9a 23 78     mr      r26,r4
|c0025280:       90 01 00 74     stw     r0,116(r1)
|c0025284:       7c bb 2b 78     mr      r27,r5
|c0025288:       7c 7f 1b 78     mr      r31,r3
|c002528c:       48 0e ce c5     bl      c0112150 <cap_syslog>
|c0025290:       7c 79 1b 79     mr.     r25,r3
|c0025294:       40 82 04 54     bne-    c00256e8 <do_syslog+0x478>
|c0025298:       2b 9f 00 0a     cmplwi  cr7,r31,10
|c002529c:       41 9d 04 40     bgt-    cr7,c00256dc <do_syslog+0x46c>
|c00252a0:       3d 20 c0 26     lis     r9,-16346
|c00252a4:       57 e0 10 3a     rlwinm  r0,r31,2,0,29
|c00252a8:       39 29 36 e4     addi    r9,r9,14052
|c00252ac:       7c 09 00 2e     lwzx    r0,r9,r0
|c00252b0:       7c 00 4a 14     add     r0,r0,r9
|c00252b4:       7c 09 03 a6     mtctr   r0
|c00252b8:       4e 80 04 20     bctr
|c00252bc:       3b e0 00 00     li      r31,0
|c00252c0:       48 00 01 c4     b       c0025484 <do_syslog+0x214>
|c00252c4:       2f 9a 00 00     cmpwi   cr7,r26,0
|c00252c8:       41 9e 04 14     beq-    cr7,c00256dc <do_syslog+0x46c>
|c00252cc:       2f 9b 00 00     cmpwi   cr7,r27,0
|c00252d0:       41 9c 04 0c     blt-    cr7,c00256dc <do_syslog+0x46c>
|c00252d4:       41 9e 04 14     beq-    cr7,c00256e8 <do_syslog+0x478>
|c00252d8:       80 02 03 3c     lwz     r0,828(r2)
|c00252dc:       7f 9a 00 40     cmplw   cr7,r26,r0
|c00252e0:       41 9d 04 04     bgt-    cr7,c00256e4 <do_syslog+0x474>
|c00252e4:       7c 1a 00 50     subf    r0,r26,r0
|c00252e8:       39 3b ff ff     addi    r9,r27,-1
|c00252ec:       7f 89 00 40     cmplw   cr7,r9,r0
|c00252f0:       41 9d 03 f4     bgt-    cr7,c00256e4 <do_syslog+0x474>
|c00252f4:       3d 20 c0 39     lis     r9,-16327
|c00252f8:       39 09 8e f8     addi    r8,r9,-28936
|c00252fc:       81 29 8e f8     lwz     r9,-28936(r9)
|c0025300:       80 08 00 04     lwz     r0,4(r8)
|c0025304:       7f 80 48 00     cmpw    cr7,r0,r9
|c0025308:       40 be 00 a4     bne+    cr7,c00253ac <do_syslog+0x13c>
|c002530c:       3d 20 c0 04     lis     r9,-16380
|c0025310:       39 61 00 14     addi    r11,r1,20
|c0025314:       39 29 ae 78     addi    r9,r9,-20872
|c0025318:       10 00 02 16     evxor   r0,r0,r0
|c002531c:       10 01 0b 21     evstdd  r0,8(r1)
|c0025320:       7d 1f 43 78     mr      r31,r8

boom.

>
>From my gcc-4.3 info pages:
>
>`-mspe=YES/NO'
>     This option has been deprecated.  Use `-mspe' and `-mno-spe'
>     instead.
>
so it could be gcc bug then and I should open a bugzilla. I didn't know
that it is deprecated.

>>+KBUILD_CFLAGS += $(call cc-option,-mabi=no-spe)
>
>is the -mabi=no-spe really needed?
It seems to work the other way around (without -mabi=no-spe but with
-mabi=no-spe) alteast I did not find anything in do_syslog() or while
browsing through the dissasm. I do a boot check tomorrow.

>
>- k

Sebastian

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Sebastian Andrzej Siewior]

* Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:

>>is the -mabi=no-spe really needed?
>It seems to work the other way around (without -mabi=no-spe but with
>-mabi=no-spe) alteast I did not find anything in do_syslog() or while
>browsing through the dissasm. I do a boot check tomorrow.

Okay. I recompiled with -mspe=no and was able to boot without trouble.
It didn't work with -mabi=no-spe.

>>
>>- k

Sebastian

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Kumar Gala]

On Oct 15, 2008, at 3:59 AM, Sebastian Andrzej Siewior wrote:

> * Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:
>
>>> is the -mabi=no-spe really needed?
>> It seems to work the other way around (without -mabi=no-spe but with
>> -mabi=no-spe) alteast I did not find anything in do_syslog() or while
>> browsing through the dissasm. I do a boot check tomorrow.
>
> Okay. I recompiled with -mspe=no and was able to boot without trouble.
> It didn't work with -mabi=no-spe.

I'm confused.. what did you change from your patch?

- k

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Sebastian Andrzej Siewior]

* Kumar Gala | 2008-10-15 08:25:54 [-0500]:

>
> On Oct 15, 2008, at 3:59 AM, Sebastian Andrzej Siewior wrote:
>
>> * Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:
>>
>>>> is the -mabi=no-spe really needed?
>>> It seems to work the other way around (without -mabi=no-spe but with
>>> -mabi=no-spe) alteast I did not find anything in do_syslog() or while
>>> browsing through the dissasm. I do a boot check tomorrow.
>>
>> Okay. I recompiled with -mspe=no and was able to boot without trouble.
>> It didn't work with -mabi=no-spe.
>
> I'm confused.. what did you change from your patch?

This is the patch it right now:
--- a/arch/powerpc/Makefile
+++ b/arch/powerpc/Makefile
@@ -103,6 +103,7 @@ KBUILD_CFLAGS += $(call cc-option,-mno-altivec)

 # No SPE instruction when building kernel
 KBUILD_CFLAGS += $(call cc-option,-mno-spe)
+KBUILD_CFLAGS += $(call cc-option,-mspe=no)

 # Enable unit-at-a-time mode when possible. It shrinks the
 # kernel considerably.
-- 

With this patch it compiles and boots fine.
The option -mabi=no-spe is not required.

> - k
Sebastian

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Kumar Gala]

On Oct 15, 2008, at 9:43 AM, Sebastian Andrzej Siewior wrote:

> * Kumar Gala | 2008-10-15 08:25:54 [-0500]:
>
>>
>> On Oct 15, 2008, at 3:59 AM, Sebastian Andrzej Siewior wrote:
>>
>>> * Sebastian Andrzej Siewior | 2008-10-15 00:49:46 [+0200]:
>>>
>>>>> is the -mabi=no-spe really needed?
>>>> It seems to work the other way around (without -mabi=no-spe but  
>>>> with
>>>> -mabi=no-spe) alteast I did not find anything in do_syslog() or  
>>>> while
>>>> browsing through the dissasm. I do a boot check tomorrow.
>>>
>>> Okay. I recompiled with -mspe=no and was able to boot without  
>>> trouble.
>>> It didn't work with -mabi=no-spe.
>>
>> I'm confused.. what did you change from your patch?
>
> This is the patch it right now:
> --- a/arch/powerpc/Makefile
> +++ b/arch/powerpc/Makefile
> @@ -103,6 +103,7 @@ KBUILD_CFLAGS += $(call cc-option,-mno-altivec)
>
> # No SPE instruction when building kernel
> KBUILD_CFLAGS += $(call cc-option,-mno-spe)
> +KBUILD_CFLAGS += $(call cc-option,-mspe=no)
>
> # Enable unit-at-a-time mode when possible. It shrinks the
> # kernel considerably.
> -- 
>
> With this patch it compiles and boots fine.
> The option -mabi=no-spe is not required.

ok.  can post a clean patch w/just this change.

- k

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Nate Case]

On Wed, 2008-10-15 at 16:43 +0200, Sebastian Andrzej Siewior wrote:
> With this patch it compiles and boots fine.
> The option -mabi=no-spe is not required.

Please don't accept this patch yet.  My past testing showed that
"-mabi=no-spe" was required for my toolchain.  I'll go back and double
check though.

- Nate Case <ncase@xes-inc.com>

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Sebastian Andrzej Siewior]

* Nate Case | 2008-10-17 09:02:11 [-0500]:

>On Wed, 2008-10-15 at 16:43 +0200, Sebastian Andrzej Siewior wrote:
>> With this patch it compiles and boots fine.
>> The option -mabi=no-spe is not required.
>
>Please don't accept this patch yet.  My past testing showed that
>"-mabi=no-spe" was required for my toolchain.  I'll go back and double
>check though.
Okay. Here my compiler details:

| powerpc-linux-gnuspe-gcc -v 
| Using built-in specs.
| Target: powerpc-linux-gnuspe
| Configured with: ../src/configure -v --with-pkgversion='Debian 4.3.1-9'
| --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
| --enable-languages=c,c++,obj-c++ --prefix=/usr --enable-shared
| --with-system-zlib --libexecdir=/usr/lib --without-included-gettext
| --enable-threads=posix --enable-nls
| --with-gxx-include-dir=/usr/powerpc-linux-gnuspe/include/c++/4.3.1
| --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug
| --disable-multilib --disable-multilib --with-cpu=8540
| --enable-e500_double --with-long-double-128 --enable-checking=release
| --program-prefix=powerpc-linux-gnuspe-
| --includedir=/usr/powerpc-linux-gnuspe/include --build=i486-linux-gnu
| --host=i486-linux-gnu --target=powerpc-linux-gnuspe
| Thread model: posix
| gcc version 4.3.1 (Debian 4.3.1-9) 
 
ths told me that we need all three options in this order because of how
they interact with gcc's internal flags.

>- Nate Case <ncase@xes-inc.com>

Sebastian

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Nate Case]

On Fri, 2008-10-17 at 09:02 -0500, Nate Case wrote:
> > With this patch it compiles and boots fine.
> > The option -mabi=no-spe is not required.
> 
> Please don't accept this patch yet.  My past testing showed that
> "-mabi=no-spe" was required for my toolchain.  I'll go back and double
> check though.

OK, I went back and re-tested.

Kernel: 2.6.27
CPU: MPC8572
Toolchain:
    Cross-compiler built using crosstool-ng
    gcc 4.3.1, default target CFLAGS include '-mabi=spe -mspe'
    binutils 2.18.90 snapshot (built with --enable-spe=yes)

Kbuild flags                    Result
------------                    ------
-mno-spe (*)                    FAILED
-mno-spe -mabi=no-spe           FAILED
-mno-spe -mspe=no               OK
-mspe=no                        OK

(*) 2.6.27 default

In the failure case, the kernel would repeatedly dump out "SPE used in
kernel (task=xxxxxxxx, pc=xxxxxxxx)".

I think I was fooled before because I added _both_ "-mspe=no" and
"-mabi=no-spe" to my KBUILD_CFLAGS and saw the problem go away.  Since I
trusted the documentation that -mspe=no and -mno-spe were the same, I
assumed that -mabi=no-spe was the key.

So, I've changed my mind.  I now agree with Sebastian that
"-mabi=no-spe" is not required.  "-mno-spe -mspe=no" is probably the
safe way to go.

-- 
Nate Case <ncase@xes-inc.com>

Re: [PATCH] powerpc: enforce a non-spe kernel build even on broken compilers

[Kumar Gala]

On Oct 24, 2008, at 6:51 PM, Nate Case wrote:

> On Fri, 2008-10-17 at 09:02 -0500, Nate Case wrote:
>>> With this patch it compiles and boots fine.
>>> The option -mabi=no-spe is not required.
>>
>> Please don't accept this patch yet.  My past testing showed that
>> "-mabi=no-spe" was required for my toolchain.  I'll go back and  
>> double
>> check though.
>
> OK, I went back and re-tested.
>
> Kernel: 2.6.27
> CPU: MPC8572
> Toolchain:
>    Cross-compiler built using crosstool-ng
>    gcc 4.3.1, default target CFLAGS include '-mabi=spe -mspe'
>    binutils 2.18.90 snapshot (built with --enable-spe=yes)
>
> Kbuild flags                    Result
> ------------                    ------
> -mno-spe (*)                    FAILED
> -mno-spe -mabi=no-spe           FAILED
> -mno-spe -mspe=no               OK
> -mspe=no                        OK
>
> (*) 2.6.27 default
>
> In the failure case, the kernel would repeatedly dump out "SPE used in
> kernel (task=xxxxxxxx, pc=xxxxxxxx)".
>
> I think I was fooled before because I added _both_ "-mspe=no" and
> "-mabi=no-spe" to my KBUILD_CFLAGS and saw the problem go away.   
> Since I
> trusted the documentation that -mspe=no and -mno-spe were the same, I
> assumed that -mabi=no-spe was the key.
>
> So, I've changed my mind.  I now agree with Sebastian that
> "-mabi=no-spe" is not required.  "-mno-spe -mspe=no" is probably the
> safe way to go.
>
> -- 
> Nate Case <ncase@xes-inc.com>

thanks for testing this all out.

I'll submit a patch to remove the -mabi=.*spe.* foo.

- k