Re: mpc8xx and ld.so problem

Re: mpc8xx and ld.so problem

[Marcelo Tosatti]

Hi Anton,


(moving to ppc-embedded since it might be of interesting for other=20
8xx users)

On Thu, Jun 30, 2005 at 04:13:30PM +0200, Anton W=F6llert wrote:
> Hello Marcelo
>=20
> I suggest you should find out why binaries hang and where.
> >=20
> > You can see where processes are sleeping with:
> >=20
> > ps -axeo "command nwchan"
> >=20
>=20
> thank you for that tip. but i found out (what i should have had to do=20
> before), that the application doesn't hang in kernel-mode. so wchan doe=
sn't=20
> say anything. but with gdb i saw the problem, the application hangs in =
the=20
> function memset at the instruction dcbz, this should be a instruction, =
that=20
> loops until it something is zero or so ( sorry, that i didn't looked up=
 it=20
> yet, i will do that ). and because of the bug of these dcbx instruction=
s on=20
> 8xx i think, that this is the cause. here my gdb-session, i hope you ma=
y=20
> find it helpful or give me an advise how to fix that :
>=20
> awoeller@zwiebel
> :~/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/busybox-1.00$po=
werpc-linux-gdb
> GNU gdb 6.3
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and yo=
u are
> welcome to change it and/or distribute copies of it under certain=20
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for detai=
ls.
> This GDB was configured as "--host=3Di686-linux --target=3Dpowerpc-linu=
x".
> (gdb) set solib-absolute-prefix /tmp/fakelibc=20
> (gdb) file busybox
> Reading symbols from=20
> /home/awoeller/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/bus=
ybox-
> 1.00/busybox...done.
> (gdb) b main
> Breakpoint 1 at 0x1000398c: file=20
> /home/awoeller/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/bus=
ybox-
> 1.00/applets/busybox.c, line 75.
> (gdb) target remote tqm850l:123
> Remote debugging using tqm850l:123
> 0x300103f4 in ?? ()
> (gdb) cont
> Continuing.
> # here i interrupt, because it hangs
> Program received signal SIGINT, Interrupt.
> 0x30013e58 in ?? ()
> (gdb) return
> Make selected stack frame return now? (y or n) y
> #0 0x3000e374 in ?? ()
> (gdb) cont
> Continuing.
>=20
> Breakpoint 1, main (argc=3D1, argv=3D0x7ffffeb4) at=20
> /home/awoeller/ToolChains/new.usr.chain/powerpc-linux-toolchain/src/bus=
ybox-
> 1.00/applets/busybox.c:75
> 75 bb_applet_name =3D argv[0];
>=20
> (gdb) disas 0x30013e58
> Dump of assembler code for function memset:
> 0x30013ba4 <memset+0>: cmplwi cr1,r5,4
> 0x30013ba8 <memset+4>: andi. r7,r3,3
> 0x30013bac <memset+8>: mr r6,r3
> 0x30013bb0 <memset+12>: ble- cr1,0x30013d40 <memset+412>
> 0x30013bb4 <memset+16>: cmplwi cr5,r5,31
> 0x30013bb8 <memset+20>: rlwimi r4,r4,8,16,23
> 0x30013bbc <memset+24>: beq+ 0x30013be0 <memset+60>
> 0x30013bc0 <memset+28>: mtcrf 1,r3
> 0x30013bc4 <memset+32>: subfic r7,r7,4
> 0x30013bc8 <memset+36>: add r6,r6,r7
> 0x30013bcc <memset+40>: subf r5,r7,r5
> 0x30013bd0 <memset+44>: bns+ cr7,0x30013bdc <memset+56>
> 0x30013bd4 <memset+48>: stb r4,0(r3)
> 0x30013bd8 <memset+52>: beq- cr7,0x30013be0 <memset+60>
> 0x30013bdc <memset+56>: sth r4,-2(r6)
> 0x30013be0 <memset+60>: mtcrf 1,r5
> 0x30013be4 <memset+64>: rlwimi r4,r4,16,0,15
> 0x30013be8 <memset+68>: ble- cr5,0x30013d80 <memset+476>
> 0x30013bec <memset+72>: andi. r7,r6,28
> 0x30013bf0 <memset+76>: subfic r7,r7,32
> 0x30013bf4 <memset+80>: beq- 0x30013c34 <memset+144>
> 0x30013bf8 <memset+84>: mtcrf 1,r7
> 0x30013bfc <memset+88>: add r6,r6,r7
> 0x30013c00 <memset+92>: subf r5,r7,r5
> 0x30013c04 <memset+96>: cmplwi cr1,r7,16
> 0x30013c08 <memset+100>: mr r8,r6
> 0x30013c0c <memset+104>: bge- cr7,0x30013c18 <memset+116>
> 0x30013c10 <memset+108>: stw r4,-4(r8)
> 0x30013c14 <memset+112>: stwu r4,-8(r8)
> 0x30013c18 <memset+116>: blt- cr1,0x30013c2c <memset+136>
> 0x30013c1c <memset+120>: stw r4,-4(r8)
> 0x30013c20 <memset+124>: stw r4,-8(r8)
> 0x30013c24 <memset+128>: stw r4,-12(r8)
> 0x30013c28 <memset+132>: stwu r4,-16(r8)
> 0x30013c2c <memset+136>: ble- cr7,0x30013c34 <memset+144>
> 0x30013c30 <memset+140>: stw r4,-4(r8)
> 0x30013c34 <memset+144>: cmplwi cr1,r4,0
> 0x30013c38 <memset+148>: rlwinm. r7,r5,0,0,26
> 0x30013c3c <memset+152>: mtcrf 1,r5
> 0x30013c40 <memset+156>: beq- cr1,0x30013de0 <memset+572>
> 0x30013c44 <memset+160>: rlwinm r0,r7,27,5,31
> 0x30013c48 <memset+164>: mtctr r0
> 0x30013c4c <memset+168>: beq- 0x30013d80 <memset+476>
> 0x30013c50 <memset+172>: clrlwi. r5,r5,27
> 0x30013c54 <memset+176>: add r6,r6,r7
> 0x30013c58 <memset+180>: li r8,-64
> 0x30013c5c <memset+184>: bdz- 0x30013c90 <memset+236>
> 0x30013c60 <memset+188>: dcbtst r8,r6
> 0x30013c64 <memset+192>: stw r4,-4(r6)
> 0x30013c68 <memset+196>: stw r4,-8(r6)
> 0x30013c6c <memset+200>: stw r4,-12(r6)
> 0x30013c70 <memset+204>: stw r4,-16(r6)
> 0x30013c74 <memset+208>: nop
> 0x30013c78 <memset+212>: stw r4,-20(r6)
> 0x30013c7c <memset+216>: stw r4,-24(r6)
> 0x30013c80 <memset+220>: nop
> 0x30013c84 <memset+224>: stw r4,-28(r6)
> 0x30013c88 <memset+228>: stwu r4,-32(r6)
> 0x30013c8c <memset+232>: bdnz+ 0x30013c60 <memset+188>
> 0x30013c90 <memset+236>: stw r4,-4(r6)
> 0x30013c94 <memset+240>: stw r4,-8(r6)
> 0x30013c98 <memset+244>: stw r4,-12(r6)
> 0x30013c9c <memset+248>: stw r4,-16(r6)
> 0x30013ca0 <memset+252>: stw r4,-20(r6)
> 0x30013ca4 <memset+256>: cmplwi cr1,r5,16
> 0x30013ca8 <memset+260>: stw r4,-24(r6)
> 0x30013cac <memset+264>: stw r4,-28(r6)
> 0x30013cb0 <memset+268>: stwu r4,-32(r6)
> 0x30013cb4 <memset+272>: beqlr=20
> 0x30013cb8 <memset+276>: add r6,r6,r7
> 0x30013cbc <memset+280>: b 0x30013d84 <memset+480>
> 0x30013cc0 <memset+284>: nop
> 0x30013cc4 <memset+288>: clrlwi r5,r5,27
> 0x30013cc8 <memset+292>: mtcrf 2,r7
> 0x30013ccc <memset+296>: rlwinm. r0,r7,25,7,31
> ---Type <return> to continue, or q <return> to quit---=20
> 0x30013cd0 <memset+300>: mtctr r0
> 0x30013cd4 <memset+304>: li r7,32
> 0x30013cd8 <memset+308>: li r8,-64
> 0x30013cdc <memset+312>: cmplwi cr1,r5,16
> 0x30013ce0 <memset+316>: bne- cr6,0x30013cec <memset+328>
> 0x30013ce4 <memset+320>: dcbz r0,r6
> 0x30013ce8 <memset+324>: addi r6,r6,32
> 0x30013cec <memset+328>: li r9,-32
> 0x30013cf0 <memset+332>: ble- cr6,0x30013d00 <memset+348>
> 0x30013cf4 <memset+336>: dcbz r0,r6
> 0x30013cf8 <memset+340>: dcbz r7,r6
> 0x30013cfc <memset+344>: addi r6,r6,64
> 0x30013d00 <memset+348>: cmplwi cr5,r5,0
> 0x30013d04 <memset+352>: beq- 0x30013d80 <memset+476>
> 0x30013d08 <memset+356>: dcbz r0,r6
> 0x30013d0c <memset+360>: dcbz r7,r6
> 0x30013d10 <memset+364>: addi r6,r6,128
> 0x30013d14 <memset+368>: dcbz r8,r6
> 0x30013d18 <memset+372>: dcbz r9,r6
> 0x30013d1c <memset+376>: bdnz+ 0x30013d08 <memset+356>
> 0x30013d20 <memset+380>: beqlr cr5
> 0x30013d24 <memset+384>: b 0x30013d84 <memset+480>
> 0x30013d28 <memset+388>: nop
> 0x30013d2c <memset+392>: nop
> 0x30013d30 <memset+396>: nop
> 0x30013d34 <memset+400>: nop
> 0x30013d38 <memset+404>: nop
> 0x30013d3c <memset+408>: nop
> 0x30013d40 <memset+412>: cmplwi cr5,r5,1
> 0x30013d44 <memset+416>: cmplwi cr1,r5,3
> 0x30013d48 <memset+420>: bltlr cr5
> 0x30013d4c <memset+424>: stb r4,0(r6)
> 0x30013d50 <memset+428>: beqlr cr5
> 0x30013d54 <memset+432>: nop
> 0x30013d58 <memset+436>: stb r4,1(r6)
> 0x30013d5c <memset+440>: bltlr cr1
> 0x30013d60 <memset+444>: stb r4,2(r6)
> 0x30013d64 <memset+448>: beqlr cr1
> 0x30013d68 <memset+452>: nop
> 0x30013d6c <memset+456>: stb r4,3(r6)
> 0x30013d70 <memset+460>: blr
> 0x30013d74 <memset+464>: nop
> 0x30013d78 <memset+468>: nop
> 0x30013d7c <memset+472>: nop
> 0x30013d80 <memset+476>: cmplwi cr1,r5,16
> 0x30013d84 <memset+480>: add r6,r6,r5
> 0x30013d88 <memset+484>: bso- cr7,0x30013da8 <memset+516>
> 0x30013d8c <memset+488>: beq- cr7,0x30013db0 <memset+524>
> 0x30013d90 <memset+492>: bgt- cr7,0x30013db8 <memset+532>
> 0x30013d94 <memset+496>: bge- cr1,0x30013dc0 <memset+540>
> 0x30013d98 <memset+500>: bgelr cr7
> 0x30013d9c <memset+504>: stw r4,-4(r6)
> 0x30013da0 <memset+508>: stw r4,-8(r6)
> 0x30013da4 <memset+512>: blr
> 0x30013da8 <memset+516>: stbu r4,-1(r6)
> 0x30013dac <memset+520>: bne- cr7,0x30013d90 <memset+492>
> 0x30013db0 <memset+524>: sthu r4,-2(r6)
> 0x30013db4 <memset+528>: ble- cr7,0x30013d94 <memset+496>
> 0x30013db8 <memset+532>: stwu r4,-4(r6)
> 0x30013dbc <memset+536>: blt- cr1,0x30013dd0 <memset+556>
> 0x30013dc0 <memset+540>: stw r4,-4(r6)
> 0x30013dc4 <memset+544>: stw r4,-8(r6)
> 0x30013dc8 <memset+548>: stw r4,-12(r6)
> 0x30013dcc <memset+552>: stwu r4,-16(r6)
> 0x30013dd0 <memset+556>: bgelr cr7
> 0x30013dd4 <memset+560>: stw r4,-4(r6)
> 0x30013dd8 <memset+564>: stw r4,-8(r6)
> 0x30013ddc <memset+568>: blr
> 0x30013de0 <memset+572>: mflr r0
> 0x30013de4 <memset+576>: beq+ 0x30013d80 <memset+476>
> 0x30013de8 <memset+580>: bl 0x30029000 <_dl_auxv+180>
> 0x30013dec <memset+584>: mflr r9
> 0x30013df0 <memset+588>: lwz r9,1832(r9)
> 0x30013df4 <memset+592>: lwz r8,0(r9)
> 0x30013df8 <memset+596>: mtlr r0
> 0x30013dfc <memset+600>: cmplwi cr1,r8,0
> ---Type <return> to continue, or q <return> to quit---
> 0x30013e00 <memset+604>: beq+ cr1,0x30013c44 <memset+160>
> 0x30013e04 <memset+608>: cmplwi cr1,r8,32
> 0x30013e08 <memset+612>: beq+ cr1,0x30013cc4 <memset+288>
> 0x30013e0c <memset+616>: dcbtst r0,r6
> 0x30013e10 <memset+620>: addi r9,r8,-1
> 0x30013e14 <memset+624>: cmplwi cr1,r5,32
> 0x30013e18 <memset+628>: and. r0,r9,r6
> 0x30013e1c <memset+632>: blt- cr1,0x30013e68 <memset+708>
> 0x30013e20 <memset+636>: beq- 0x30013e50 <memset+684>
> 0x30013e24 <memset+640>: addi r6,r6,32
> 0x30013e28 <memset+644>: addi r5,r5,-32
> 0x30013e2c <memset+648>: stw r4,-32(r6)
> 0x30013e30 <memset+652>: stw r4,-28(r6)
> 0x30013e34 <memset+656>: stw r4,-24(r6)
> 0x30013e38 <memset+660>: stw r4,-20(r6)
> 0x30013e3c <memset+664>: stw r4,-16(r6)
> 0x30013e40 <memset+668>: stw r4,-12(r6)
> 0x30013e44 <memset+672>: stw r4,-8(r6)
> 0x30013e48 <memset+676>: stw r4,-4(r6)
> 0x30013e4c <memset+680>: b 0x30013e14 <memset+624>
> 0x30013e50 <memset+684>: cmplw cr1,r5,r8
> 0x30013e54 <memset+688>: blt- cr1,0x30013e68 <memset+708>
>=20
>=20
> 0x30013e58 <memset+692>: dcbz r0,r6 #<--- the problem
>=20
>=20
> 0x30013e5c <memset+696>: subf r5,r8,r5
> 0x30013e60 <memset+700>: add r6,r6,r8
> 0x30013e64 <memset+704>: b 0x30013e50 <memset+684>
> 0x30013e68 <memset+708>: rlwinm. r7,r5,0,0,26
> 0x30013e6c <memset+712>: b 0x30013c44 <memset+160>
> 0x30013e70 <memset+716>: nop
> 0x30013e74 <memset+720>: nop
> 0x30013e78 <memset+724>: nop
> 0x30013e7c <memset+728>: nop

Can you please examine in more detail what is going on here?=20

What are the contents of the r0 and r6 registers, where exactly is the
task stopped.

I _guess_ it might be some form of "dcbz" misbehaviour, in such case I=20
imagine that either the task would loop forever or execute an invalid
exception/instruction.

Re: mpc8xx and ld.so problem

[Jason McMullan]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

On Fri, 2005-07-01 at 06:44 -0300, Marcelo Tosatti wrote:
> Hi Anton,
> 
> 
> (moving to ppc-embedded since it might be of interesting for other 
> 8xx users)
> 

Apply this patch to glibc, and recompile:

rm -f glibc/sysdeps/powerpc/powerpc32/memset.S


The PPC32 dbcz semantics don't seem to work properly on 8xx
in all cases. Removing the '.S' file makes glibc fall back on
the .c implementation.

-- 
Jason McMullan <jason.mcmullan@gmail.com>
"Sure, send me the latest Knoppix DVD as an attachment..."

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: mpc8xx and ld.so problem

[Marcelo Tosatti]

On Fri, Jul 01, 2005 at 10:55:16AM -0400, Jason McMullan wrote:
> On Fri, 2005-07-01 at 06:44 -0300, Marcelo Tosatti wrote:
> > Hi Anton,
> > 
> > 
> > (moving to ppc-embedded since it might be of interesting for other 
> > 8xx users)
> > 
> 
> Apply this patch to glibc, and recompile:
> 
> rm -f glibc/sysdeps/powerpc/powerpc32/memset.S
> 
> 
> The PPC32 dbcz semantics don't seem to work properly on 8xx
> in all cases. Removing the '.S' file makes glibc fall back on
> the .c implementation. 

Hi Jason, 

That was a quick response - thanks.

Two questions:

- Do you happen to know about details of dcbz's (mis)behaviour on 8xx? 

I ask that mainly because I worry about in-kernel dcbz users.

- Shouldnt upstream glibc have that fixed for 8xx by now?

Re: mpc8xx and ld.so problem

[Jason McMullan]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

On Fri, 2005-07-01 at 07:17 -0300, Marcelo Tosatti wrote:
> That was a quick response - thanks.
> 
> Two questions:
> 
> - Do you happen to know about details of dcbz's (mis)behaviour on 8xx? 
> 
> I ask that mainly because I worry about in-kernel dcbz users.

    IIRC, it isn't used in any 8xx code paths.

> - Shouldnt upstream glibc have that fixed for 8xx by now?

    Ha. Funny. The glibc powerpc maintainer doesn't want any embedded
fixes in the mainline. Last I checked, that was for 'the tools
vendors' to fix.

    "We won't work around processor bugs" is their philosophy.

    I went through a similar (unsuccessful) battle with the
amcc 440ep's "blrl" errata and gcc/glibc.

    It would be nice if the politics there have changed
(maybe they just didn't like me personally), but I don't have
much hope.

-- 
Jason McMullan <jason.mcmullan@gmail.com>
"Sure, send me the latest Knoppix DVD as an attachment..."

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: mpc8xx and ld.so problem

[Marcelo Tosatti]

On Fri, Jul 01, 2005 at 02:56:30PM -0400, Jason McMullan wrote:
> On Fri, 2005-07-01 at 07:17 -0300, Marcelo Tosatti wrote:
> > That was a quick response - thanks.
> > 
> > Two questions:
> > 
> > - Do you happen to know about details of dcbz's (mis)behaviour on 8xx? 
> > 
> > I ask that mainly because I worry about in-kernel dcbz users.
> 
>     IIRC, it isn't used in any 8xx code paths.

OK!

> > - Shouldnt upstream glibc have that fixed for 8xx by now?
> 
>     Ha. Funny. The glibc powerpc maintainer doesn't want any embedded
> fixes in the mainline. 

Who is that?

> Last I checked, that was for 'the tools vendors' to fix.

Silly - so embedded developers are supposed rely on "embedded vendors"
and not on the mainstream software distributions? 

>     "We won't work around processor bugs" is their philosophy.
>
>     I went through a similar (unsuccessful) battle with the
> amcc 440ep's "blrl" errata and gcc/glibc.
> 
>     It would be nice if the politics there have changed
> (maybe they just didn't like me personally), but I don't have
> much hope.

If enough people complain they will, hopefully, listen.

Re: mpc8xx and ld.so problem

[Yuli Barcohen]

>>>>> Jason McMullan writes:

[...deleted...]

    Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any
    Jason> embedded fixes in the mainline. Last I checked, that was for
    Jason> 'the tools vendors' to fix.

    Jason> "We won't work around processor bugs" is their philosophy.

[...deleted...]

I investigated the problem a bit when I had trouble with a self-compiled
glibc a year or so ago. IIRC, I found bug in the memset code, not in the
chip. The code was just wrong for cache line sizes not equal to 32. So
memset.S is good for 60x series (PQII included) but for 8xx it fails. We
use dcbX instructions in some kernel drivers and since we never had any
problems with those drivers I'm a bit surprised to hear that all 8xx
chips have got that bug.

-- 
========================================================================
 Yuli Barcohen       | Phone +972-9-765-1788 |  Software Project Leader
 yuli@arabellasw.com | Fax   +972-9-765-7494 | Arabella Software, Israel
========================================================================

Re: mpc8xx and ld.so problem

[Tom Rini]

On Mon, Jul 04, 2005 at 11:22:14AM +0300, Yuli Barcohen wrote:
> >>>>> Jason McMullan writes:
> 
> [...deleted...]
> 
>     Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any
>     Jason> embedded fixes in the mainline. Last I checked, that was for
>     Jason> 'the tools vendors' to fix.
> 
>     Jason> "We won't work around processor bugs" is their philosophy.
> 
> [...deleted...]
> 
> I investigated the problem a bit when I had trouble with a self-compiled
> glibc a year or so ago. IIRC, I found bug in the memset code, not in the
> chip. The code was just wrong for cache line sizes not equal to 32. So
> memset.S is good for 60x series (PQII included) but for 8xx it fails. We
> use dcbX instructions in some kernel drivers and since we never had any
> problems with those drivers I'm a bit surprised to hear that all 8xx
> chips have got that bug.

It's also OK on a multiple of 32, iirc, but not smaller.  And using the
information the kernel does export would be too slow.  Or at least no
one figured out a good way to do it, userspace side.

-- 
Tom Rini
http://gate.crashing.org/~trini/

Re: mpc8xx and ld.so problem

[Yuli Barcohen]

>>>>> Tom Rini writes:

    Yuli> [...deleted...]

    Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any
    Jason> embedded fixes in the mainline. Last I checked, that was for
    Jason> 'the tools vendors' to fix.

    Jason> "We won't work around processor bugs" is their philosophy.

    Yuli> [...deleted...]

    Yuli> I investigated the problem a bit when I had trouble with a
    Yuli> self-compiled glibc a year or so ago. IIRC, I found bug in the
    Yuli> memset code, not in the chip. The code was just wrong for
    Yuli> cache line sizes not equal to 32. So memset.S is good for 60x
    Yuli> series (PQII included) but for 8xx it fails. We use dcbX
    Yuli> instructions in some kernel drivers and since we never had any
    Yuli> problems with those drivers I'm a bit surprised to hear that
    Yuli> all 8xx chips have got that bug.

    Tom> It's also OK on a multiple of 32, iirc, but not smaller.  And
    Tom> using the information the kernel does export would be too slow.
    Tom> Or at least no one figured out a good way to do it, userspace
    Tom> side.

IMHO the cache line size from cputable is not used in the kernel but
only passed to the ELF interpreter. I did not want to re-build glibc so
I changed the entry for mpc8xx to pass 0 instead of 16. This disabled
the assembler implementation of memset and since then all our
mpc8xx-based boards work properly. In any case, since it looks like a
coding bug and not CPU errata, maybe the glibc maintainer would be
willing to fix it?

-- 
========================================================================
 Yuli Barcohen       | Phone +972-9-765-1788 |  Software Project Leader
 yuli@arabellasw.com | Fax   +972-9-765-7494 | Arabella Software, Israel
========================================================================

Re: mpc8xx and ld.so problem

[Marcelo Tosatti]

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]


Hi Yuli, 

On Mon, Jul 04, 2005 at 11:22:14AM +0300, Yuli Barcohen wrote:
> >>>>> Jason McMullan writes:
> 
> [...deleted...]
> 
>     Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any
>     Jason> embedded fixes in the mainline. Last I checked, that was for
>     Jason> 'the tools vendors' to fix.
> 
>     Jason> "We won't work around processor bugs" is their philosophy.
> 
> [...deleted...]
> 
> I investigated the problem a bit when I had trouble with a self-compiled
> glibc a year or so ago. IIRC, I found bug in the memset code, not in the
> chip. The code was just wrong for cache line sizes not equal to 32. So
> memset.S is good for 60x series (PQII included) but for 8xx it fails.

I suppose you didnt actually use dcbz for userspace memset on 8xx? 

> We use dcbX instructions in some kernel drivers and since we never had any
> problems with those drivers I'm a bit surprised to hear that all 8xx
> chips have got that bug.

The problem is that the DAR register is correctly unset (it comes as NULL IIRC) 
on pagefaults for the dcbz instruction. The dcbz instructions you issue are probably
always works on kernel addresses whose pagetables are present?

Joakim has developed a workaround for the problem... although I promised him
several times to test it I never managed to get dcbz to work on the kernel 
copying functions. :(

He has posted it here in the past... here it is attached anyway.
 



[-- Attachment #2: dcbx.patch5 --]
[-- Type: text/plain, Size: 8750 bytes --]

===== head_8xx.S 1.21 vs edited =====
--- 1.21/arch/ppc/kernel/head_8xx.S	2005-03-29 00:21:20 +02:00
+++ edited/head_8xx.S	2005-04-06 17:01:51 +02:00
@@ -32,6 +32,8 @@
 #include <asm/ppc_asm.h>
 #include <asm/offsets.h>
 
+#define CONFIG_8xx_DCBxFIXED
+
 /* Macro to make the code more readable. */
 #ifdef CONFIG_8xx_CPU6
 #define DO_8xx_CPU6(val, reg)	\
@@ -41,6 +43,20 @@
 #else
 #define DO_8xx_CPU6(val, reg)
 #endif
+
+#ifdef CONFIG_8xx_DCBxFIXED
+/* These macros are used to tag DAR with a known value so that the
+ * DataTLBError can recognize a buggy dcbx instruction and workaround
+ * the problem.
+ */
+#define TAG_VAL 0x00f0	/*  -1 may also be used */
+#define TAG_DAR_R10 	\
+	li	r10, TAG_VAL;\
+	mtspr	SPRN_DAR, r10;
+#else
+#define TAG_DAR_R10
+#endif
+
 	.text
 	.globl	_stext
 _stext:
@@ -174,6 +190,7 @@
 	xfer(n, hdlr)
 
 #define EXC_XFER_TEMPLATE(n, hdlr, trap, copyee, tfer, ret)	\
+	TAG_DAR_R10;						\
 	li	r10,trap;					\
 	stw	r10,TRAP(r11);					\
 	li	r10,MSR_KERNEL;					\
@@ -214,6 +231,7 @@
 	mfspr r5,SPRN_DSISR
 	stw r5,_DSISR(r11)
 	addi r3,r1,STACK_FRAME_OVERHEAD
+	TAG_DAR_R10
 	EXC_XFER_STD(0x200, MachineCheckException)
 
 /* Data access exception.
@@ -227,6 +245,7 @@
 	stw	r10,_DSISR(r11)
 	mr	r5,r10
 	mfspr	r4,SPRN_DAR
+	TAG_DAR_R10
 	EXC_XFER_EE_LITE(0x300, handle_page_fault)
 
 /* Instruction access exception.
@@ -252,6 +271,7 @@
 	mfspr	r5,SPRN_DSISR
 	stw	r5,_DSISR(r11)
 	addi	r3,r1,STACK_FRAME_OVERHEAD
+	TAG_DAR_R10
 	EXC_XFER_EE(0x600, AlignmentException)
 
 /* Program check exception */
@@ -414,7 +434,13 @@
 	rlwimi	r10, r11, 0, 24, 28	/* Set 24-27, clear 28 */
 	DO_8xx_CPU6(0x3d80, r3)
 	mtspr	SPRN_MD_RPN, r10	/* Update TLB entry */
-
+#ifdef CONFIG_8xx_DCBxFIXED
+ #if TAG_VAL == 0x00f0 /* Save 1 instr. by reusing the val loaded in r11 above */
+ 	mtspr	SPRN_DAR, r11
+ #else
+ 	TAG_DAR_R10
+ #endif
+#endif
 	mfspr	r10, SPRN_M_TW	/* Restore registers */
 	lwz	r11, 0(r0)
 	mtcr	r11
@@ -450,11 +476,20 @@
 	mfcr	r10
 	stw	r10, 0(r0)
 	stw	r11, 4(r0)
+ 	mfspr	r10, SPRN_DAR
+#ifdef  CONFIG_8xx_DCBxFIXED
+	/* If DAR contains TAG_VAL implies a buggy dcbx instruction
+	 * that did not set DAR.
+	 */
+	cmpwi	cr0, r10, TAG_VAL
+	beq-	100f	/* Branch if TAG_VAL to dcbx workaround procedure */
+101:	/* return from dcbx instruction bug workaround, r10 holds value of DAR */	
+#endif
 
 	/* First, make sure this was a store operation.
 	*/
-	mfspr	r10, SPRN_DSISR
-	andis.	r11, r10, 0x0200	/* If set, indicates store op */
+	mfspr	r11, SPRN_DSISR
+	andis.	r11, r11, 0x0200	/* If set, indicates store op */
 	beq	2f
 
 	/* The EA of a data TLB miss is automatically stored in the MD_EPN
@@ -473,7 +508,7 @@
 	 * are initialized in mapin_ram().  This will avoid the problem,
 	 * assuming we only use the dcbi instruction on kernel addresses.
 	 */
-	mfspr	r10, SPRN_DAR
+	/* DAR is in r10 already */
 	rlwinm	r11, r10, 0, 0, 19
 	ori	r11, r11, MD_EVALID
 	mfspr	r10, SPRN_M_CASID
@@ -523,7 +558,13 @@
 	rlwimi	r10, r11, 0, 24, 28	/* Set 24-27, clear 28 */
 	DO_8xx_CPU6(0x3d80, r3)
 	mtspr	SPRN_MD_RPN, r10	/* Update TLB entry */
-
+#ifdef CONFIG_8xx_DCBxFIXED
+ #if TAG_VAL == 0x00f0 /* Save 1 instr. by reusing the val loaded in r11 above */
+	mtspr	SPRN_DAR, r11
+ #else
+	TAG_DAR_R10
+ #endif
+#endif
 	mfspr	r10, SPRN_M_TW	/* Restore registers */
 	lwz	r11, 0(r0)
 	mtcr	r11
@@ -561,6 +602,185 @@
 
 	. = 0x2000
 
+#ifdef CONFIG_8xx_DCBxFIXED
+/* This is the workaround procedure to calculate the data EA for buggy dcbx,dcbi instructions
+ * by decoding the registers used by the dcbx instruction and adding them.
+ * DAR is set to the calculated address and r10 also holds the EA on exit.
+ */
+//#define INSTR_CHECK /* define to verify if it is a dcbx instr. Should not be needed. */
+//#define NO_SELF_MODIFYING_CODE /* define if you don't want to use self modifying code */
+//#define DEBUG_DCBX_INSTRUCTIONS /* for debugging only. Needs INSTR_CHECK defined as well. */
+//#define KERNEL_SPACE_ONLY /* define if user space do NOT contain dcbx instructions. */
+
+#ifndef KERNEL_SPACE_ONLY
+	nop	/* A few nops to make the modified_instr: space below cache line aligned */
+	nop
+139:	/* fetch instruction from userspace memory */
+	DO_8xx_CPU6(0x3780, r3)
+	mtspr	SPRN_MD_EPN, r10
+	mfspr	r11, SPRN_M_TWB	/* Get level 1 table entry address */
+	lwz	r11, 0(r11)	/* Get the level 1 entry */
+	DO_8xx_CPU6(0x3b80, r3)
+	mtspr	SPRN_MD_TWC, r11	/* Load pte table base address */
+	mfspr	r11, SPRN_MD_TWC	/* ....and get the pte address */
+	lwz	r11, 0(r11)	/* Get the pte */
+	/* concat physical page address(r11) and page offset(r10) */
+	rlwimi	r11, r10, 0, 20, 31
+	b	140f
+#endif
+100:	/* Entry point for dcbx workaround. */
+	/* fetch instruction from memory. */
+	mfspr	r10,SPRN_SRR0
+#ifndef KERNEL_SPACE_ONLY
+	andis.	r11, r10, 0x8000
+	tophys  (r11, r10)
+	beq-	139b		/* Branch if user space address */
+#else
+	tophys  (r11, r10)
+#endif
+140:	lwz	r11,0(r11)
+#ifdef INSTR_CHECK
+/* Check if it really is a dcbx instruction. This is not needed as far as I can tell */
+/* dcbt and dcbtst does not generate DTLB Misses/Errors, no need to include them here */
+	rlwinm	r10, r11, 0, 21, 30
+	cmpwi	cr0, r10, 2028	/* Is dcbz? */
+	beq+	142f
+	cmpwi	cr0, r10, 940	/* Is dcbi? */
+	beq+	142f
+	cmpwi	cr0, r10, 108	/* Is dcbst? */
+	beq+	142f
+	cmpwi	cr0, r10, 172	/* Is dcbf? */
+	beq+	142f
+	cmpwi	cr0, r10, 1964	/* Is icbi? */
+	beq+	142f
+#ifdef DEBUG_DCBX_INSTRUCTIONS
+141:	b 141b /* Stop here if no dcbx instruction */
+#endif
+	mfspr	r10, SPRN_DAR	/* r10 must hold DAR at exit */
+	b 101b			/* None of the above, go back to normal TLB processing */
+142:	/* continue, it was a dcbx instruction. */
+#endif
+#ifdef CONFIG_8xx_CPU6
+	lwz	r3, 8(r0)		/* restore r3 from memory */
+#endif
+#ifndef NO_SELF_MODIFYING_CODE
+	andis.	r10,r11,0x1f	/* test if reg RA is r0 */
+	li	r10,modified_instr@l
+	dcbtst	r0,r10		/* touch for store */
+	rlwinm	r11,r11,0,0,20	/* Zero lower 10 bits */
+	oris	r11,r11,640	/* Transform instr. to a "add r10,RA,RB" */
+	ori	r11,r11,532
+	stw	r11,0(r10)	/* store add/and instruction */
+	dcbf	0,r10		/* flush new instr. to memory. */
+	icbi	0,r10		/* invalidate instr. cache line */
+	lwz	r11, 4(r0)	/* restore r11 from memory */
+	mfspr	r10, SPRN_M_TW	/* restore r10 from M_TW */
+	isync			/* Wait until new instr is loaded from memory */
+modified_instr:
+	.space	4		/* this is where the add/and instr. is stored */
+#ifdef DEBUG_DCBX_INSTRUCTIONS
+	/* fill with some garbage */ 
+	li	r11,0xffff
+	stw	r11,0(r11)
+#endif
+	bne+	143f
+	subf	r10,r0,r10		/* r10=r10-r0, only if reg RA is r0 */
+143:	mtdar	r10			/* store faulting EA in DAR */
+	b	101b			/* Go back to normal TLB handling */
+#else
+	mfctr	r10
+	mtdar	r10			/* save ctr reg in DAR */
+	rlwinm	r10, r11, 24, 24, 28	/* offset into jump table for reg RB */
+	addi	r10, r10, 150f@l	/* add start of table */
+	mtctr	r10			/* load ctr with jump address */
+	xor	r10, r10, r10		/* sum starts at zero */
+	bctr				/* jump into table */
+150:
+	add	r10, r10, r0
+	b	151f
+	add	r10, r10, r1
+	b	151f
+	add	r10, r10, r2
+	b	151f
+	add	r10, r10, r3
+	b	151f
+	add	r10, r10, r4
+	b	151f
+	add	r10, r10, r5
+	b	151f
+	add	r10, r10, r6
+	b	151f
+	add	r10, r10, r7
+	b	151f
+	add	r10, r10, r8
+	b	151f
+	add	r10, r10, r9
+	b	151f
+	mtctr	r11	/* reg 10 needs special handling */
+	b	154f
+	mtctr	r11	/* reg 11 needs special handling */
+	b	153f
+	add	r10, r10, r12
+	b	151f
+	add	r10, r10, r13
+	b	151f
+	add	r10, r10, r14
+	b	151f
+	add	r10, r10, r15
+	b	151f
+	add	r10, r10, r16
+	b	151f
+	add	r10, r10, r17
+	b	151f
+	add	r10, r10, r18
+	b	151f
+	add	r10, r10, r19
+	b	151f
+	add	r10, r10, r20
+	b	151f
+	add	r10, r10, r21
+	b	151f
+	add	r10, r10, r22
+	b	151f
+	add	r10, r10, r23
+	b	151f
+	add	r10, r10, r24
+	b	151f
+	add	r10, r10, r25
+	b	151f
+	add	r10, r10, r25
+	b	151f
+	add	r10, r10, r27
+	b	151f
+	add	r10, r10, r28
+	b	151f
+	add	r10, r10, r29
+	b	151f
+	add	r10, r10, r30
+	b	151f
+	add	r10, r10, r31
+151:
+	rlwinm. r11,r11,19,24,28	/* offset into jump table for reg RA */
+	beq	152f			/* if reg RA is zero, don't add it */ 
+	addi	r11, r11, 150b@l	/* add start of table */
+	mtctr	r11			/* load ctr with jump address */
+	rlwinm	r11,r11,0,16,10		/* make sure we don't execute this more than once */
+	bctr				/* jump into table */
+152:
+	mfdar	r11
+	mtctr	r11			/* restore ctr reg from DAR */
+	mtdar	r10			/* save fault EA to DAR */
+	b	101b			/* Go back to normal TLB handling */
+
+	/* special handling for r10,r11 since these are modified already */
+153:	lwz	r11, 4(r0)	/* load r11 from memory */
+	b	155f
+154:	mfspr	r11, SPRN_M_TW	/* load r10 from M_TW */
+155:	add	r10, r10, r11	/* add it */
+	mfctr	r11		/* restore r11 */
+	b	151b
+#endif
+#endif
 	.globl	giveup_fpu
 giveup_fpu:
 	blr

Re: mpc8xx and ld.so problem

[Yuli Barcohen]

>>>>> Marcelo Tosatti writes:

    Yuli> [...deleted...]

    Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any
    Jason> embedded fixes in the mainline. Last I checked, that was for
    Jason> 'the tools vendors' to fix.

    Jason> "We won't work around processor bugs" is their philosophy.

    Yuli> [...deleted...]

    Yuli> I investigated the problem a bit when I had trouble with a
    Yuli> self-compiled glibc a year or so ago. IIRC, I found bug in the
    Yuli> memset code, not in the chip. The code was just wrong for
    Yuli> cache line sizes not equal to 32. So memset.S is good for 60x
    Yuli> series (PQII included) but for 8xx it fails.

    Marcelo> I suppose you didnt actually use dcbz for userspace memset
    Marcelo> on 8xx?

Standard glibc did. After the fix, it doesn't do it any more on our
systems.

    Yuli> We use dcbX instructions in some kernel drivers and since we
    Yuli> never had any problems with those drivers I'm a bit surprised
    Yuli> to hear that all 8xx chips have got that bug.

    Marcelo> The problem is that the DAR register is correctly unset (it
    Marcelo> comes as NULL IIRC) on pagefaults for the dcbz
    Marcelo> instruction. The dcbz instructions you issue are probably
    Marcelo> always works on kernel addresses whose pagetables are
    Marcelo> present?

It's not dcbz, it's dcbi/dcbf. And yes, they work on kernel addresses. I
never investigated if the page tables are present or not because there
were no problems.

    Marcelo> Joakim has developed a workaround for the
    Marcelo> problem... although I promised him several times to test it
    Marcelo> I never managed to get dcbz to work on the kernel copying
    Marcelo> functions. :(

[...patch deleted...]

Well, if I manage to find time, I'll try it. No timetables though. I'm
not sure if using dcbz in user-space memset is such a great
optimisation. It well can be an example of over-engineering.

-- 
========================================================================
 Yuli Barcohen       | Phone +972-9-765-1788 |  Software Project Leader
 yuli@arabellasw.com | Fax   +972-9-765-7494 | Arabella Software, Israel
========================================================================

Re: mpc8xx and ld.so problem

[Theo Gjaltema]

Hi,

It appeard that on my mpc862 target I had also this problem. I am using
ELDK3.1.1 which includes a 2.4.25 kernel.
The symptoms however were:
    Large application (20MB c++, mostly shared-libs) crashes (sigseg) at
startup BEFORE main() is called.
    Attempts to debug the application also fail: the debugger crashes
also (sigseg).
I didn't change the C-lib, I took the 2.4.30/arch/ppc/kernel/head_8xx.S
file and copied this over the 2.4.25 head_8xx.S file.
In the 2.4.30 version there is a piece of code mentioning the dcbX
instruction problems.

Has anyone an idea why only this large application failed? busybox_1.0
and more applications work fine.
system is a: mcp862/32Mb SDRAM started from u-boot 0.4.1,
kernel enhanced with atm/utopia driver.

This driver causes the CPM sometimes to hang when performing a memset,
can this be caused by the same problem?
(CPM stops responding, console buffers are not flused anymore and
kernel stops waiting for buffers)

Greetings,
   Theo Gjaltema



Yuli Barcohen schreef:

>>>>>>Marcelo Tosatti writes:
>>>>>>            
>>>>>>
>
>    Yuli> [...deleted...]
>
>    Jason> Ha. Funny. The glibc powerpc maintainer doesn't want any
>    Jason> embedded fixes in the mainline. Last I checked, that was for
>    Jason> 'the tools vendors' to fix.
>
>    Jason> "We won't work around processor bugs" is their philosophy.
>
>    Yuli> [...deleted...]
>
>    Yuli> I investigated the problem a bit when I had trouble with a
>    Yuli> self-compiled glibc a year or so ago. IIRC, I found bug in the
>    Yuli> memset code, not in the chip. The code was just wrong for
>    Yuli> cache line sizes not equal to 32. So memset.S is good for 60x
>    Yuli> series (PQII included) but for 8xx it fails.
>
>    Marcelo> I suppose you didnt actually use dcbz for userspace memset
>    Marcelo> on 8xx?
>
>Standard glibc did. After the fix, it doesn't do it any more on our
>systems.
>
>    Yuli> We use dcbX instructions in some kernel drivers and since we
>    Yuli> never had any problems with those drivers I'm a bit surprised
>    Yuli> to hear that all 8xx chips have got that bug.
>
>    Marcelo> The problem is that the DAR register is correctly unset (it
>    Marcelo> comes as NULL IIRC) on pagefaults for the dcbz
>    Marcelo> instruction. The dcbz instructions you issue are probably
>    Marcelo> always works on kernel addresses whose pagetables are
>    Marcelo> present?
>
>It's not dcbz, it's dcbi/dcbf. And yes, they work on kernel addresses. I
>never investigated if the page tables are present or not because there
>were no problems.
>
>    Marcelo> Joakim has developed a workaround for the
>    Marcelo> problem... although I promised him several times to test it
>    Marcelo> I never managed to get dcbz to work on the kernel copying
>    Marcelo> functions. :(
>
>[...patch deleted...]
>
>Well, if I manage to find time, I'll try it. No timetables though. I'm
>not sure if using dcbz in user-space memset is such a great
>optimisation. It well can be an example of over-engineering.
>
>  
>

Re: mpc8xx and ld.so problem

[Wolfgang Denk]

In message <42D5361B.2070402@chello.nl> you wrote:
> 
> It appeard that on my mpc862 target I had also this problem. I am using
> ELDK3.1.1 which includes a 2.4.25 kernel.

No, most definitely not.

>     Large application (20MB c++, mostly shared-libs) crashes (sigseg) at
> startup BEFORE main() is called.
>     Attempts to debug the application also fail: the debugger crashes
> also (sigseg).

I bet that this is a FAQ. Your SDRAM initialization is broken.

> I didn't change the C-lib, I took the 2.4.30/arch/ppc/kernel/head_8xx.S
> file and copied this over the 2.4.25 head_8xx.S file.
> In the 2.4.30 version there is a piece of code mentioning the dcbX
> instruction problems.

And did this fix the problem completely? I would be a bit surprised.

> This driver causes the CPM sometimes to hang when performing a memset,
> can this be caused by the same problem?

Yes. When your SDRAM init is broken, strange things will happen.


Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Work 8 hours, sleep 8 hours; but not the same 8 hours.

Re: mpc8xx and ld.so problem

[Theo Gjaltema]

Wolfgang Denk schreef:

>In message <42D5361B.2070402@chello.nl> you wrote:
>  
>
>>It appeard that on my mpc862 target I had also this problem. I am using
>>ELDK3.1.1 which includes a 2.4.25 kernel.
>>    
>>
>
>No, most definitely not.
>
>  
>
I don't understand this, on
/pub/Linux/distributions/eldk/3.1.1/ppc-linux-x86/distribution/common
there are the sources of a 2.4.25 kernel.

>>    Large application (20MB c++, mostly shared-libs) crashes (sigseg) at
>>startup BEFORE main() is called.
>>    Attempts to debug the application also fail: the debugger crashes
>>also (sigseg).
>>    
>>
>
>I bet that this is a FAQ. Your SDRAM initialization is broken.
>  
>
This was one of my guesses also. Since this is beyond my knowledge, I 
had the parametes initialisation parameters compared to an other 
platform which did not had the problems (rtos based), the same 
parameters were used. I did not investigate it any further.

>  
>
>>I didn't change the C-lib, I took the 2.4.30/arch/ppc/kernel/head_8xx.S
>>file and copied this over the 2.4.25 head_8xx.S file.
>>In the 2.4.30 version there is a piece of code mentioning the dcbX
>>instruction problems.
>>    
>>
>
>And did this fix the problem completely? I would be a bit surprised.
>
>  
>
No problem since.

>>This driver causes the CPM sometimes to hang when performing a memset,
>>can this be caused by the same problem?
>>    
>>
>
>Yes. When your SDRAM init is broken, strange things will happen.
>
>  
>
I'll try the kernel with the original atm/utopia to see what happens.

>Best regards,
>
>Wolfgang Denk
>
>  
>
Thanks,
    Theo Gjaltema.

Re: mpc8xx and ld.so problem

[Wolfgang Denk]

In message <42D58861.5040900@chello.nl> you wrote:
> 
> >>It appeard that on my mpc862 target I had also this problem. I am using
> >>ELDK3.1.1 which includes a 2.4.25 kernel.
> >
> >No, most definitely not.
> >
> I don't understand this, on
> /pub/Linux/distributions/eldk/3.1.1/ppc-linux-x86/distribution/common
> there are the sources of a 2.4.25 kernel.

Yes, sure. But most definitely you did not have *this* problem.
You have a *different* one.

> >I bet that this is a FAQ. Your SDRAM initialization is broken.
> >
> This was one of my guesses also. Since this is beyond my knowledge, I 
> had the parametes initialisation parameters compared to an other 
> platform which did not had the problems (rtos based), the same 
> parameters were used. I did not investigate it any further.

This *is* a FAQ. Please go and read the documentation.
See http://www.denx.de/twiki/bin/view/DULG/UBootCrashAfterRelocation

Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
"Obviously, a major malfunction has occurred."
              -- Steve Nesbitt, voice of Mission Control, January 28,
                 1986, as the shuttle Challenger exploded within view
                 of the grandstands.

Re: mpc8xx and ld.so problem

[Anton Wöllert]

Theo Gjaltema wrote:

 > Has anyone an idea why only this large application failed? busybox_1.0
 > and more applications work fine.
 > system is a: mcp862/32Mb SDRAM started from u-boot 0.4.1,
 > kernel enhanced with atm/utopia driver.
 >
 > This driver causes the CPM sometimes to hang when performing a memset,
 > can this be caused by the same problem?
 > (CPM stops responding, console buffers are not flused anymore and
 > kernel stops waiting for buffers)
 >
 > Greetings,
 >   Theo Gjaltema
 >

the problem ( so i think ) is, that larger applications will need more 
space, which in turn is satisfied by malloc (or mmap with fd = -1). 
these allocated pages are not real initialized after malloc ( they will 
be initialized completely, if a write on it is done, which will cause 
the page fault handler, that does the hardware mmu-stuff, so that a 
write can be done). so they will cause a page fault on a first access. 
no problem so long, but when the memset.S routine in libc tries to set 
all to zero, it uses the dcbz instructions. this instruction then causes 
a page fault or better a TLB miss (if i'm right). after the exception is 
raised, the address where the write to was done is looked up in the DAR 
(data address register), which could be bogus on the dcb* instruction 
(on 8xx, where is the errata for that?).
to sum up, the chance, that a memset( .., '\0', ..) with dcbz is done on 
a page (that has just registered structures in the kernel, and not 
cached in the tlb or even marked as writeable) is bigger :)

that's my opinion. if someone has more knowledge about that, or if i'm 
wrong - please! correct me.

anton

ptrace on linux 2.6.12 causes oops

[Anton Wöllert]

[-- Attachment #1: Type: text/plain, Size: 3721 bytes --]

Hello

when i try to run strace or gdbserver on a program, the following comes:

Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: 0300 Not 
tainted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK = c0ea8430[761] 'gdbserver' THREAD: c0f34000
Last syscall: 26 
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF 00F58000 
00000001 
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 01000800 
00000001 
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 C0839300 
C01E0000 
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 C02ACB00 
C02ACB00 
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30
Call trace:
[c000b154] update_mmu_cache+0x7c/0xa4
[c005ae98] do_wp_page+0x460/0x5ec
[c005c8a0] handle_mm_fault+0x7cc/0x91c
[c005ccec] get_user_pages+0x2fc/0x65c
[c0027104] access_process_vm+0x9c/0x1d4
[c00076e0] sys_ptrace+0x240/0x4a4
[c0002bd0] ret_from_syscall+0x0/0x44
mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by 
mm/memory.c/1306

and strace or gdbserver of course says segmentation fault. with gdbserver, 
this happens every time. with strace, the first time it works nearly all 
time, but when i strace a second time program again, it segfaults. i think 
the access_process_vm is accessed trough PEEKDATA and PEEKTEXT in 
sys_ptrace. so here some more debug :) :

DEBUG: peekdata @ 1006d4ec
DEBUG: peektext @ 1006d4f0
DEBUG: peekdata @ 1006d4f0
DEBUG: peektext @ 1006d4f4
DEBUG: peekdata @ 1006d4f4
DEBUG: peektext @ 1006d4f8
DEBUG: peekdata @ 1006d4f8
DEBUG: peektext @ 1006d4fc
DEBUG: peekdata @ 1006d4fc
DEBUG: peektext @ 1006d500
DEBUG: peekdata @ 1006d500
DEBUG: peektext @ 1006d504
DEBUG: peekdata @ 1006d504
DEBUG: peektext @ 1006d508
DEBUG: peekdata @ 1006d508
DEBUG: peektext @ 1006d50c
DEBUG: peekdata @ 1006d50c
DEBUG: peektext @ 1006d510
DEBUG: peekdata @ 1006d510
DEBUG: peektext @ 1006d514
DEBUG: peekdata @ 1006d514
DEBUG: peektext @ 1006d518
DEBUG: peekdata @ 1006d518
DEBUG: peektext @ 1006d51c
DEBUG: peekdata @ 1006d51c
DEBUG: peektext @ 1006d520
DEBUG: peekdata @ 1006d520
DEBUG: peektext @ 1006d524
DEBUG: peekdata @ 1006d524
DEBUG: peektext @ 1006d528
DEBUG: peekdata @ 1006d528
DEBUG: peektext @ 1006d52c
DEBUG: peekdata @ 1006d52c
DEBUG: peektext @ 00000000
DEBUG: peekdata @ 00000000
DEBUG: peektext @ 3000ac18
DEBUG: peekdata @ 3000ac18
DEBUG: peektext @ 3000ac18
DEBUG: peekdata @ 3000ac18
DEBUG: flush_dcache_icache_page
Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: 0300 Not 
tainted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK = c0ea8430[761] 'gdbserver' THREAD: c0f34000
Last syscall: 26 
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF 00F58000 
00000001 
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 01000800 
00000001 
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 C0839300 
C01E0000 
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 C02ACB00 
C02ACB00 
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30
Call trace:
[c000b154] update_mmu_cache+0x7c/0xa4
[c005ae98] do_wp_page+0x460/0x5ec
[c005c8a0] handle_mm_fault+0x7cc/0x91c
[c005ccec] get_user_pages+0x2fc/0x65c
[c0027104] access_process_vm+0x9c/0x1d4
[c00076e0] sys_ptrace+0x240/0x4a4
[c0002bd0] ret_from_syscall+0x0/0x44
mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by 
mm/memory.c/1306

[-- Attachment #2: Type: text/html, Size: 4221 bytes --]

Re: ptrace on linux 2.6.12 causes oops

[Kumar Gala]

What system is this on?

- kumar

On Jul 14, 2005, at 3:23 AM, Anton W=F6llert wrote:

> Hello
>
> when i try to run strace or gdbserver on a program, the following =20
> comes:
>
> Oops: kernel access of bad area, sig: 11 [#2]
> NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: =20
> 0300    Not tainted
> MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
> DAR: 00000010, DSISR: C2000000
> TASK =3D c0ea8430[761] 'gdbserver' THREAD: c0f34000
> Last syscall: 26
> GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF =20
> 00F58000 00000001
> GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 =20
> 01000800 00000001
> GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 =20
> C0839300 C01E0000
> GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 =20
> C02ACB00 C02ACB00
> NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
> LR [c000b060] flush_dcache_icache_page+0x20/0x30
> Call trace:
>  [c000b154] update_mmu_cache+0x7c/0xa4
>  [c005ae98] do_wp_page+0x460/0x5ec
>  [c005c8a0] handle_mm_fault+0x7cc/0x91c
>  [c005ccec] get_user_pages+0x2fc/0x65c
>  [c0027104] access_process_vm+0x9c/0x1d4
>  [c00076e0] sys_ptrace+0x240/0x4a4
>  [c0002bd0] ret_from_syscall+0x0/0x44
> mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked =20
> by mm/memory.c/1306
>
> and strace or gdbserver of course says segmentation fault. with =20
> gdbserver, this happens every time. with strace, the first time it =20
> works nearly all time, but when i strace a second time program =20
> again, it segfaults. i think the access_process_vm is accessed =20
> trough PEEKDATA and PEEKTEXT in sys_ptrace. so here some more =20
> debug :) :
>
> DEBUG: peekdata @ 1006d4ec
> DEBUG: peektext @ 1006d4f0
> DEBUG: peekdata @ 1006d4f0
> DEBUG: peektext @ 1006d4f4
> DEBUG: peekdata @ 1006d4f4
> DEBUG: peektext @ 1006d4f8
> DEBUG: peekdata @ 1006d4f8
> DEBUG: peektext @ 1006d4fc
> DEBUG: peekdata @ 1006d4fc
> DEBUG: peektext @ 1006d500
> DEBUG: peekdata @ 1006d500
> DEBUG: peektext @ 1006d504
> DEBUG: peekdata @ 1006d504
> DEBUG: peektext @ 1006d508
> DEBUG: peekdata @ 1006d508
> DEBUG: peektext @ 1006d50c
> DEBUG: peekdata @ 1006d50c
> DEBUG: peektext @ 1006d510
> DEBUG: peekdata @ 1006d510
> DEBUG: peektext @ 1006d514
> DEBUG: peekdata @ 1006d514
> DEBUG: peektext @ 1006d518
> DEBUG: peekdata @ 1006d518
> DEBUG: peektext @ 1006d51c
> DEBUG: peekdata @ 1006d51c
> DEBUG: peektext @ 1006d520
> DEBUG: peekdata @ 1006d520
> DEBUG: peektext @ 1006d524
> DEBUG: peekdata @ 1006d524
> DEBUG: peektext @ 1006d528
> DEBUG: peekdata @ 1006d528
> DEBUG: peektext @ 1006d52c
> DEBUG: peekdata @ 1006d52c
> DEBUG: peektext @ 00000000
> DEBUG: peekdata @ 00000000
> DEBUG: peektext @ 3000ac18
> DEBUG: peekdata @ 3000ac18
> DEBUG: peektext @ 3000ac18
> DEBUG: peekdata @ 3000ac18
> DEBUG: flush_dcache_icache_page
> Oops: kernel access of bad area, sig: 11 [#2]
> NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: =20
> 0300    Not tainted
> MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
> DAR: 00000010, DSISR: C2000000
> TASK =3D c0ea8430[761] 'gdbserver' THREAD: c0f34000
> Last syscall: 26
> GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF =20
> 00F58000 00000001
> GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 =20
> 01000800 00000001
> GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 =20
> C0839300 C01E0000
> GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 =20
> C02ACB00 C02ACB00
> NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
> LR [c000b060] flush_dcache_icache_page+0x20/0x30
> Call trace:
>  [c000b154] update_mmu_cache+0x7c/0xa4
>  [c005ae98] do_wp_page+0x460/0x5ec
>  [c005c8a0] handle_mm_fault+0x7cc/0x91c
>  [c005ccec] get_user_pages+0x2fc/0x65c
>  [c0027104] access_process_vm+0x9c/0x1d4
>  [c00076e0] sys_ptrace+0x240/0x4a4
>  [c0002bd0] ret_from_syscall+0x0/0x44
> mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked =20
> by mm/memory.c/1306
>
>
>
>
> <ATT11632593.txt>
>

Re: ptrace on linux 2.6.12 causes oops

[Marcelo Tosatti]

On Thu, Jul 14, 2005 at 08:31:50AM -0500, Kumar Gala wrote:
> What system is this on?

Thats 8xx (Anton told me on IRC earlier)...

ptrace on linux 2.6.12 causes oops

[Anton Wöllert]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

 it's on a tqm850l with mpc850. i've done the following:

in update_mmu_cache:

if (!PageReserved(page)
&& !test_bit(PG_arch_1, &page->flags)) {
// if (vma->vm_mm == current->active_mm){
// _tlbie(address);
// __flush_dcache_icache((void *) address);
// } else
__flush_dcache_icache(page _address(page));
// flush_dcache_icache_page(page);

set_bit(PG_arch_1, &page->flags);
}

like it is in ppc64. now it works. the flush_dcache_icache_page calls 
__flush_dcache_icache_phys, which temporary turns off the mmu for 
data-addressing. a bit strange i think. but unfortunately i have too less 
knowledge about kernel-internal ppc-stuff :(

anton

[-- Attachment #2: Type: text/html, Size: 2401 bytes --]

Re: ptrace on linux 2.6.12 causes oops

[aris]

> when i try to run strace or gdbserver on a program, the following comes:
> 
> NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
> LR [c000b060] flush_dcache_icache_page+0x20/0x30
> Call trace:
> [c000b154] update_mmu_cache+0x7c/0xa4
> [c005ae98] do_wp_page+0x460/0x5ec
> [c005c8a0] handle_mm_fault+0x7cc/0x91c
> [c005ccec] get_user_pages+0x2fc/0x65c
> [c0027104] access_process_vm+0x9c/0x1d4
> [c00076e0] sys_ptrace+0x240/0x4a4
> [c0002bd0] ret_from_syscall+0x0/0x44
> mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by 
> mm/memory.c/1306
please try to reproduce with 2.6.13-rc2. seems much like the problem Marcelo
fixed recently (dcbst misbehaviour with unpopulated TLB entries)

-- 
Aristeu

Re: ptrace on linux 2.6.12 causes oops

[Marcelo Tosatti]

On Thu, Jul 14, 2005 at 05:27:15PM -0300, aris@conectiva.com.br wrote:
> > when i try to run strace or gdbserver on a program, the following comes:
> > 
> > NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
> > LR [c000b060] flush_dcache_icache_page+0x20/0x30
> > Call trace:
> > [c000b154] update_mmu_cache+0x7c/0xa4
> > [c005ae98] do_wp_page+0x460/0x5ec
> > [c005c8a0] handle_mm_fault+0x7cc/0x91c
> > [c005ccec] get_user_pages+0x2fc/0x65c
> > [c0027104] access_process_vm+0x9c/0x1d4
> > [c00076e0] sys_ptrace+0x240/0x4a4
> > [c0002bd0] ret_from_syscall+0x0/0x44
> > mm/memory.c:2054: spin_lock(kernel/fork.c:c0ea1618) already locked by 
> > mm/memory.c/1306
> please try to reproduce with 2.6.13-rc2. seems much like the problem Marcelo
> fixed recently (dcbst misbehaviour with unpopulated TLB entries)

Yep, just that now its the ptraceing process which is faulting in the page,
instead of the (ptraced) process itself. 

So Anton, can you move the _tlbie() call up to 

                    && !test_bit(PG_arch_1, &page->flags)) {
								<---------- HERE
                        if (vma->vm_mm == current->active_mm)
                                __flush_dcache_icache((void *) address);
                        else
                                flush_dcache_icache_page(page);
                        set_bit(PG_arch_1, &page->flags);

So that it covers both cases instead of just (vma->vm_mm == current->active_mm) ?

Its safe to do it because the address space ID is ignored by tlbie accordingly 
to the manual page:

The ASID value in the entry is ignored for the purpose of
matching an invalidate address, thus multiple entries can be invalidated 
if they have the same effective address and different ASID values.

Re: ptrace on linux 2.6.12 causes oops

[Anton Wöllert]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

> 
> Yep, just that now its the ptraceing process which is faulting in the 
> page,
> instead of the (ptraced) process itself.
> 
> So Anton, can you move the _tlbie() call up to
> 
> && !test_bit(PG_arch_1, &page->flags)) {
> <---------- HERE
> if (vma->vm_mm == current->active_mm)
> __flush_dcache_icache((void *) address);
> else
> flush_dcache_icache_page(page);
> set_bit(PG_arch_1, &page->flags);
> 
> So that it covers both cases instead of just (vma->vm_mm == 
> current->active_mm) ?
> 
> Its safe to do it because the address space ID is ignored by tlbie 
> accordingly
> to the manual page:
> 
> The ASID value in the entry is ignored for the purpose of
> matching an invalidate address, thus multiple entries can be invalidated
> if they have the same effective address and different ASID values.



Well, unfortunately, that doesn't work :(. If i'm right, the 
__flush_dcache_icache((void *) address) should avoid that the cache says 
faulting address again.
The flush_dcache_icache_page(page) should flush the cache, where stands, 
page not mapped. but the flush_dcache_icache_page(page) oopses on my system. 
but instead of this call, the call __flush_dcache_icache(page_address(page)) 
works. for me, that also makes more sence. and also, the 
flush_dcache_icache_page(page) calls the flush_dcache_icache_phys, which 
turns off the data virtual address mapping. i found that a bit strange. any 
comments?

[-- Attachment #2: Type: text/html, Size: 2985 bytes --]

Re: ptrace on linux 2.6.12 causes oops

[Marcelo Tosatti]

Hi Anton,

On Fri, Jul 15, 2005 at 11:42:27AM +0200, Anton W=F6llert wrote:
> >=20
> > Yep, just that now its the ptraceing process which is faulting in the=
=20
> > page,
> > instead of the (ptraced) process itself.
> >=20
> > So Anton, can you move the _tlbie() call up to
> >=20
> > && !test_bit(PG_arch_1, &page->flags)) {
> > <---------- HERE
> > if (vma->vm_mm =3D=3D current->active_mm)
> > __flush_dcache_icache((void *) address);
> > else
> > flush_dcache_icache_page(page);
> > set_bit(PG_arch_1, &page->flags);
> >=20
> > So that it covers both cases instead of just (vma->vm_mm =3D=3D=20
> > current->active_mm) ?
> >=20
> > Its safe to do it because the address space ID is ignored by tlbie=20
> > accordingly
> > to the manual page:
> >=20
> > The ASID value in the entry is ignored for the purpose of
> > matching an invalidate address, thus multiple entries can be invalida=
ted
> > if they have the same effective address and different ASID values.
>=20
>=20
>=20
> Well, unfortunately, that doesn't work :(.=20

Oh doh, on the case where the process faulting in the page is not the own=
er of the
vma (which is the case with ptrace here), the flushing is done via the ph=
ysical address.

So its expected that the _tlbie() on the process virtual address will not=
 change=20
a thing, since flush_dcache_icache_page() is working on the physical addr=
ess.

> If i'm right, the=20
> __flush_dcache_icache((void *) address) should avoid that the cache say=
s=20
> faulting address again.
> The flush_dcache_icache_page(page) should flush the cache, where stands=
,=20
> page not mapped. but the flush_dcache_icache_page(page) oopses on my sy=
stem.=20
> but instead of this call, the call __flush_dcache_icache(page_address(p=
age))=20
> works. for me, that also makes more sence.=20

Well, you end up flushing the page through its kernel virtual address map=
ping (returned
by page_address()).=20

It will be necessary for the kernel virtual map to be faulted in the TLB,=
 which is=20
certainly slower than doing the direct physical address flush (requires u=
pdating=20
the MSR twice (+isync) to turn the MMU on/off).

But other than that I see no problem with your suggestion really, as to w=
orkaround the=20
oops.

Still, we should try to understand what is going on...

> and also, the flush_dcache_icache_page(page) calls the flush_dcache_ica=
che_phys, which=20
> turns off the data virtual address mapping. i found that a bit strange.=
 any=20
> comments?=20

It seems that the instruction at "__flush_dcache_icache_phys+0x38" is icb=
i - can you send=20
the disassembly of __flush_dcache_icache_phys? =20

We should figure out what is causing the oops.=20

What is causing DAR to be set to "00000010"? ie why the hell is it trying=
 to access=20
the 00000010 address.

Oops: kernel access of bad area, sig: 11 [#2]
NIP: C000543C LR: C000B060 SP: C0F35DF0 REGS: c0f35d40 TRAP: 0300 Not tai=
nted
MSR: 00009022 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 10
DAR: 00000010, DSISR: C2000000
TASK =3D c0ea8430[761] 'gdbserver' THREAD: c0f34000
Last syscall: 26
GPR00: 00009022 C0F35DF0 C0EA8430 00F59000 00000100 FFFFFFFF 00F58000
00000001
GPR08: C021DAEF C0270000 00009032 C0270000 22044024 10025428 01000800
00000001
GPR16: 007FFF3F 00000001 00000000 7FBC6AC0 00F61022 00000001 C0839300
C01E0000
GPR24: 00CD0889 C082F568 3000AC18 C02A7A00 C0EA15C8 00F588A9 C02ACB00
C02ACB00
NIP [c000543c] __flush_dcache_icache_phys+0x38/0x54
LR [c000b060] flush_dcache_icache_page+0x20/0x30

Re: mpc8xx and ld.so problem

[Anton Wöllert]

Marcelo Tosatti wrote:
> Hi Anton,
> 
> 
> (moving to ppc-embedded since it might be of interesting for other 
> 8xx users)
> 
> On Thu, Jun 30, 2005 at 04:13:30PM +0200, Anton Wöllert wrote:
> 
> 
> Can you please examine in more detail what is going on here? 
> 
> What are the contents of the r0 and r6 registers, where exactly is the
> task stopped.

with backtrace in gdb i found out, that memset() (which is exactly the 
routine at sysdeps/powerpc/powerpc32/memset.S) called by 
_dl_allocate_tls_storage (sysdeps/generic/dl-tls.c). btw. memset is 
called 3 times before and there it works fine.
well, _dl_allocate_tls_storage invokes __libc_memalign 
(elf/dl-minimal.c). __libc_memalign should return an aligned area of 
memory for the tls-stuff (which later then will be memset'd with zero).
__libc_memalign now allocates an aligned memory block, but first looks 
if there is unused space in the previous page of its data-segment. if 
there is some, it will use that. if more than this is needed, it will 
call mmap with MAP_ANON|MAP_PRIVATE and PROT_READ|PROT_WRITE. it then 
looks, if the returned address lies at the end of the old page, if not, 
the old one will not be used (because then it wouldn't return an 
continuegous area of memory). will gdb, i saw, that mmap returns a 
address right after the previous page and so the block continuegous.
this block is now returned and given to memset.
memset now zeros out a part of the returned block. as long, as dcbz 
zeros out the memory, that lies in the old page (before 0x30001000) it 
works, but when it steps over that address (the page boundary), stepi in 
gdb just hangs and doesn't return anything ( except i press ctrl-c ).
i think, dcbz should cause a page fault, because the new page was 
allocated with PROT_READ|PROT_WRITE. i expect, that there is no handling 
for dcbz's in the page fault handler (which should be do_page_fault in 
arch/ppc/kernel/fault.c if i'm right).
other possibilities, like false cache_line_size etc should explain that, 
  because they are all set right (with gdb i inspected nearly 
everything, and memset should work as expected, and it does so in other 
scenarios).

sorry for not delivering the registers and so on, but actually i have no 
access to the board for one week :). i will post them later, if really 
needed, but throug my gdb-sessions, they seemed all to be right.

> 
> I _guess_ it might be some form of "dcbz" misbehaviour, in such case I 
> imagine that either the task would loop forever or execute an invalid
> exception/instruction.
> 

so what should be the real solution, deleting memset.S because the use 
of dcbz is not allowed or eventually fix the page-fault handler, because 
he didn't recognize dcbz's (if i'm right).

thanks for your help, anton