Erratum 51 bugfix?

Erratum 51 bugfix?

[David Gibson]

According to the ppc405gp errata, the workaround for erratum 51 is
incorrect.  The errata suggest that vector 0x1700 should branch to the
data tlb miss exception, whereas the current code branches to the
instruction tlb miss exception.  The patch below corrects this.

diff -urN ../linuxppc_2_4_devel/arch/ppc/kernel/head_4xx.S linux-bungo/arch/ppc/kernel/head_4xx.S
--- ../linuxppc_2_4_devel/arch/ppc/kernel/head_4xx.S	Mon Sep 17 10:32:05 2001
+++ linux-bungo/arch/ppc/kernel/head_4xx.S	Mon Sep 17 15:19:57 2001
@@ -591,7 +594,7 @@
 #ifdef CONFIG_IBM405_ERR51
 	/* 405GP errata 51 */
 	START_EXCEPTION(0x1700, Trap_17)
-	b ITLBMiss
+	b	DTLBMiss
 #else
 	STND_EXCEPTION(0x1700,	Trap_17,		UnknownException)
 #endif

--
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.  -- H.L. Mencken
http://www.ozlabs.org/people/dgibson


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Erratum 51 bugfix?

[Dan Malek]

David Gibson wrote:
>
> According to the ppc405gp errata, the workaround for erratum 51 is
> incorrect.

Got it, thanks.  I actually went to read the 4xx errata this time instead
of just blindly accepting the patch as I did last time.  There are some
other nasty things we should consider fixing.  The worst is #67, and now
they have told us about #77 so we can fix up the ATOMIC_SYNC_FIX....

Of all the errata, #77 is the only one we have seen with Linux.  I guess
you could force #51 to occur, but whether you call ITLBmiss or DTLBmiss
probably doesn't matter because they end up at the same place.  We are
somewhat lucky with #67 because I don't think we have mapping situations
where this can occur.  Although, some days I read it and think otherwise.....


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[David Gibson]

On Mon, Sep 17, 2001 at 12:32:03PM -0400, Dan Malek wrote:
>
> David Gibson wrote:
> >
> > According to the ppc405gp errata, the workaround for erratum 51 is
> > incorrect.
>
> Got it, thanks.  I actually went to read the 4xx errata this time instead
> of just blindly accepting the patch as I did last time.  There are some
> other nasty things we should consider fixing.  The worst is #67, and now
> they have told us about #77 so we can fix up the ATOMIC_SYNC_FIX....

> Of all the errata, #77 is the only one we have seen with Linux.  I guess
> you could force #51 to occur, but whether you call ITLBmiss or DTLBmiss
> probably doesn't matter because they end up at the same place.  We are
> somewhat lucky with #67 because I don't think we have mapping situations
> where this can occur.  Although, some days I read it and think otherwise.....

Yes, #67 is very nasty, and although it doesn't occur under normal
circumstances, I'm pretty sure it can be triggered by an unprivileged
process.  Userland doesn't generally have the page at 0x0 mapped, but
it can be mapped in with mmap(...MAP_FIXED...) (I checked).  Place
some code at 0x500 or 0x1100 and user programs can execute an
arbitrary instruction or two in privileged mode.  Nasty.

The sanest fix I can think of is to simply ban MAP_FIXED at address
0x0 on 4xx.

I suspect #77 is the cause of the problems I'm seeing on the walnut
now - it mostly works, but every so often a process will freeze up
immune to signals.  Then, anything that looks too hard at it (ps, cat
/proc/##/cmdline) will also freeze up.

--
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.  -- H.L. Mencken
http://www.ozlabs.org/people/dgibson


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Dan Malek]

David Gibson wrote:

> I suspect #77 is the cause of the problems I'm seeing on the walnut
> now - it mostly works, but every so often a process will freeze up
> immune to signals.

Hmmm...We did tons of debugging on this one and pointed it out to
IBM a while back.  The ATOMIC_SYNC_FIX configuration option was the
solution to cure it.  As you can tell, it tries to address the
pipeline issues surrounding the stwcx.  I kind of hope it is the
problem, and a better silicon bug fix will solve it.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[David Gibson]

On Tue, Sep 18, 2001 at 02:52:59PM -0400, Dan Malek wrote:
>
> David Gibson wrote:
>
> > I suspect #77 is the cause of the problems I'm seeing on the walnut
> > now - it mostly works, but every so often a process will freeze up
> > immune to signals.
>
> Hmmm...We did tons of debugging on this one and pointed it out to
> IBM a while back.  The ATOMIC_SYNC_FIX configuration option was the
> solution to cure it.  As you can tell, it tries to address the
> pipeline issues surrounding the stwcx.  I kind of hope it is the
> problem, and a better silicon bug fix will solve it.

Ah, yes, I discovered ATOMIC_SYNC_FIX after I sent that, and have now
turned it on.  That should certainly fix the atomic ops, however there
are quite a number of other places where the kernel uses stwcx., which
ATOMIX_SYNC_FIX doesn't fix - notably arch/ppc/kernel/bitops.c and
include/asm-ppc/bitops.h.  As well as activating ATOMIX_SYNC_FIX I
tried inserting a sync before every other stwcx. that I could find,
and I haven't managed to get a process to lock up yet.

Is there a reason we don't need the sync (or dcbt) everywhere, or
should I send you the patch (once I've cleaned it up).

--
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.  -- H.L. Mencken
http://www.ozlabs.org/people/dgibson


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Mark Hatle]

David Gibson wrote:
>
> Ah, yes, I discovered ATOMIC_SYNC_FIX after I sent that, and have now
> turned it on.  That should certainly fix the atomic ops, however there
> are quite a number of other places where the kernel uses stwcx., which
> ATOMIX_SYNC_FIX doesn't fix - notably arch/ppc/kernel/bitops.c and
> include/asm-ppc/bitops.h.  As well as activating ATOMIX_SYNC_FIX I
> tried inserting a sync before every other stwcx. that I could find,
> and I haven't managed to get a process to lock up yet.

Just as an FYI, we also do this in glibc to be safe.  We have never been
able to pin down a problem in userspace due to this bug, but we thought
it would be better safe then sorry until we can get definative proof
that the bug will not happen in userspace.

The following two files in glibc should be patched:
linuxthreads/sysdeps/powerpc/pt-machine.h
sysdeps/powerpc/atomicity.h

--Mark

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Dan Malek]

Mark Hatle wrote:

> Just as an FYI, we also do this in glibc to be safe.

We use stwcx. in user space?  Hmmm....Hmmmm....


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Ralph Blach]

The fix must be used for each stwcx.

Chip
IBM Microelectronics.

Mark Hatle wrote:
>
> David Gibson wrote:
> >
> > Ah, yes, I discovered ATOMIC_SYNC_FIX after I sent that, and have now
> > turned it on.  That should certainly fix the atomic ops, however there
> > are quite a number of other places where the kernel uses stwcx., which
> > ATOMIX_SYNC_FIX doesn't fix - notably arch/ppc/kernel/bitops.c and
> > include/asm-ppc/bitops.h.  As well as activating ATOMIX_SYNC_FIX I
> > tried inserting a sync before every other stwcx. that I could find,
> > and I haven't managed to get a process to lock up yet.
>
> Just as an FYI, we also do this in glibc to be safe.  We have never been
> able to pin down a problem in userspace due to this bug, but we thought
> it would be better safe then sorry until we can get definative proof
> that the bug will not happen in userspace.
>
> The following two files in glibc should be patched:
> linuxthreads/sysdeps/powerpc/pt-machine.h
> sysdeps/powerpc/atomicity.h
>
> --Mark
>

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Dan Malek]

David Gibson wrote:

> Ah, yes, I discovered ATOMIC_SYNC_FIX after I sent that,....

I thought it was automatically enabled on the 405GP.....I guess not.

> .....  That should certainly fix the atomic ops, however there
> are quite a number of other places where the kernel uses stwcx., which
> ATOMIX_SYNC_FIX doesn't fix

I took a quick look at the original patch and saw that too, I just never
got around to fixing the rest of it.  It solved the problem at the time,
so it didn't seem urgent.  If you would like to finish it up, that would
be great.

Thanks.

	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[David Gibson]

On Wed, Sep 19, 2001 at 02:39:25AM -0400, Dan Malek wrote:
>
> David Gibson wrote:
>
> > Ah, yes, I discovered ATOMIC_SYNC_FIX after I sent that,....
>
> I thought it was automatically enabled on the 405GP.....I guess not.
>
> > .....  That should certainly fix the atomic ops, however there
> > are quite a number of other places where the kernel uses stwcx., which
> > ATOMIX_SYNC_FIX doesn't fix
>
> I took a quick look at the original patch and saw that too, I just never
> got around to fixing the rest of it.  It solved the problem at the time,
> so it didn't seem urgent.  If you would like to finish it up, that would
> be great.

Try the below...

--
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.  -- H.L. Mencken
http://www.ozlabs.org/people/dgibson

diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/config.in linux-bungo/arch/ppc/config.in
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/config.in	Wed Sep 19 13:35:58 2001
+++ linux-bungo/arch/ppc/config.in	Thu Sep 20 14:48:10 2001
@@ -199,10 +199,12 @@
     define_bool CONFIG_405GP y
     define_bool CONFIG_BIOS_FIXUP y
     define_bool CONFIG_TREEBOOT y
+    define_bool CONFIG_IBM405_ERR77 y
   fi
   if [ "$CONFIG_EP405" = "y" ]; then
     define_bool CONFIG_405GP y
     define_bool CONFIG_EMBEDDEDBOOT y
+    define_bool CONFIG_IBM405_ERR77 y
   fi
   if [ "$CONFIG_OAK" = "y" -o "$CONFIG_TIVO" = "y" ]; then
     define_bool CONFIG_403GCX y
@@ -210,7 +212,6 @@
   fi

   bool 'Blue Logic DMA' CONFIG_405_DMA
-  bool 'Workarounds for Atomic Operations' CONFIG_ATOMIC_SYNC_FIX

   define_bool CONFIG_IBM405_ERR51 y
   define_bool CONFIG_NOT_COHERENT_CACHE y
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/bitops.c linux-bungo/arch/ppc/kernel/bitops.c
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/bitops.c	Tue Jun  5 21:22:02 2001
+++ linux-bungo/arch/ppc/kernel/bitops.c	Thu Sep 20 15:54:46 2001
@@ -21,8 +21,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%3 \n\
-	or	%0,%0,%2 \n\
-	stwcx.	%0,0,%3 \n\
+	or	%0,%0,%2 \n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=m" (*p)
@@ -38,8 +39,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%3 \n\
-	andc	%0,%0,%2 \n\
-	stwcx.	%0,0,%3 \n\
+	andc	%0,%0,%2 \n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=m" (*p)
@@ -55,8 +57,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%3 \n\
-	xor	%0,%0,%2 \n\
-	stwcx.	%0,0,%3 \n\
+	xor	%0,%0,%2 \n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=m" (*p)
@@ -72,8 +75,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%4 \n\
-	or	%1,%0,%3 \n\
-	stwcx.	%1,0,%4 \n\
+	or	%1,%0,%3 \n"
+	PPC405_ERR77(0,%4)
+"	stwcx.	%1,0,%4 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=&r" (t), "=m" (*p)
@@ -91,8 +95,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%4 \n\
-	andc	%1,%0,%3 \n\
-	stwcx.	%1,0,%4 \n\
+	andc	%1,%0,%3 \n"
+	PPC405_ERR77(0,%4)
+"	stwcx.	%1,0,%4 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=&r" (t), "=m" (*p)
@@ -110,8 +115,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%4 \n\
-	xor	%1,%0,%3 \n\
-	stwcx.	%1,0,%4 \n\
+	xor	%1,%0,%3 \n"
+	PPC405_ERR77(0,%4)
+"	stwcx.	%1,0,%4 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=&r" (t), "=m" (*p)
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/entry.S linux-bungo/arch/ppc/kernel/entry.S
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/entry.S	Sat Sep  8 21:03:46 2001
+++ linux-bungo/arch/ppc/kernel/entry.S	Thu Sep 20 16:50:26 2001
@@ -321,6 +321,7 @@
 	SYNC			/* Some chip revs have problems here... */
 	mtmsr	r0		/* Update machine state */

+	PPC405_ERR77(0,r1)
 	stwcx.	r0,0,r1		/* to clear the reservation */

 	/* if returning to user mode, set new sprg2 and save kernel SP */
@@ -383,6 +384,7 @@
 	lwz	r4,GPR4(r1)
 	lwz	r1,GPR1(r1)
 	SYNC
+	PPC405_ERR77_SYNC
 	RFI


diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/head_4xx.S linux-bungo/arch/ppc/kernel/head_4xx.S
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/head_4xx.S	Wed Sep 19 13:35:58 2001
+++ linux-bungo/arch/ppc/kernel/head_4xx.S	Thu Sep 20 17:00:28 2001
@@ -292,6 +295,7 @@
 	mfspr	r22, SPRG4
 	mfspr	r21, SPRG1
 	mfspr	r20, SPRG0
+	PPC405_ERR77_SYNC
 	rfi			/* Should sync shadow TLBs */

 2:
@@ -568,6 +572,7 @@
 	mfspr	r22, SPRG4
 	mfspr	r21, SPRG1
 	mfspr	r20, SPRG0
+	PPC405_ERR77_SYNC
 	rfi			/* Should sync shadow TLBs */

 2:
@@ -720,6 +725,7 @@
 	mfspr	r22, SPRG4
 	mfspr	r21, SPRG1
 	mfspr	r20, SPRG0
+	PPC405_ERR77_SYNC
 	rfi			/* Should sync shadow TLBs */

 /* This code finishes saving the registers to the exception frame
@@ -743,6 +749,8 @@
 	andi.	r24,r23,0x3f00		/* Get vector offset */
 	stw	r24,TRAP(r21)
 	li	r22,RESULT
+	/* No need to put an erratum #77 workaround here
+		because interrupts are currently disabled */
 	stwcx.	r22,r22,r21		/* Clear the reservation */
 	li	r22,0
 	stw	r22,RESULT(r21)
@@ -771,6 +779,8 @@
 	mtspr	SPRN_SRR1,r20		/* Set up the machine state register */
 	mtlr	r23			/* Set up the return pointer */
 	SYNC
+	/* We shouldn't need a 405 erratum #77 workaround here, because we're not
+	 * actually returning to the interrupted instruction yet. */
 	rfi

 	/* Critical exception jump path
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/idle.c linux-bungo/arch/ppc/kernel/idle.c
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/idle.c	Tue Sep 11 14:26:51 2001
+++ linux-bungo/arch/ppc/kernel/idle.c	Fri Sep 21 13:26:09 2001
@@ -134,6 +134,7 @@
 		register unsigned long tmp;
 		asm (	"101:lwarx  %1,0,%3\n"  /* reserve zero_cache */
 			"    lwz    %0,0(%1)\n" /* get next -- new zero_cache */
+			PPC405_ERR77(0,%3)
 			"    stwcx. %0,0,%3\n"  /* update zero_cache */
 			"    bne-   101b\n"     /* if lost reservation try again */
 			: "=&r" (tmp), "=&r" (page), "+m" (zero_cache)
@@ -233,6 +234,7 @@
 #ifdef CONFIG_SMP
 			"    sync\n"            /* let store settle */
 #endif
+			PPC405_ERR77(0,%2)
 			"    stwcx. %3,0,%2\n"  /* update zero_cache in mem */
 			"    bne-   101b\n"     /* if lost reservation try again */
 			: "=&r" (tmp), "+m" (zero_quicklist)
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/misc.S linux-bungo/arch/ppc/kernel/misc.S
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/misc.S	Sat Sep  8 21:03:46 2001
+++ linux-bungo/arch/ppc/kernel/misc.S	Thu Sep 20 16:59:31 2001
@@ -302,6 +302,7 @@
 10:	lwarx	r7,0,r9
 	cmpi	0,r7,0
 	bne-	10b
+	PPC405_ERR77(0,r9)
 	stwcx.	r8,0,r9
 	bne-	10b
 #endif /* CONFIG_SMP */
@@ -334,6 +335,7 @@
 10:	lwarx	r7,0,r9
 	cmpi	0,r7,0
 	bne-	10b
+	PPC405_ERR77(0,r9)
 	stwcx.	r8,0,r9
 	bne-	10b
 	eieio
@@ -616,6 +626,7 @@
 _GLOBAL(xchg_u32)
 	mr	r5,r3		/* Save pointer */
 10:	lwarx	r3,0,r5		/* Fetch old value & reserve */
+	PPC405_ERR77(0,r5)
 	stwcx.	r4,0,r5		/* Update with new value */
 	bne-	10b		/* Retry if "reservation" (i.e. lock) lost */
 	blr
@@ -627,12 +638,14 @@
 _GLOBAL(atomic_clear_mask)
 10:	lwarx	r5,0,r4
 	andc	r5,r5,r3
+	PPC405_ERR77(0,r4)
 	stwcx.	r5,0,r4
 	bne-	10b
 	blr
 _GLOBAL(atomic_set_mask)
 10:	lwarx	r5,0,r4
 	or	r5,r5,r3
+	PPC405_ERR77(0,r4)
 	stwcx.	r5,0,r4
 	bne-	10b
 	blr
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/ppc_asm.h linux-bungo/arch/ppc/kernel/ppc_asm.h
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/ppc_asm.h	Wed Aug 29 08:57:34 2001
+++ linux-bungo/arch/ppc/kernel/ppc_asm.h	Fri Sep 21 12:38:13 2001
@@ -165,5 +165,14 @@
 #define HMT_LOW		/* nothing */
 #define HMT_MEDIUM	/* nothing */
 #define HMT_HIGH	/* nothing */
+
+#ifdef CONFIG_IBM405_ERR77
+#define PPC405_ERR77(ra,rb)	dcbt	ra, rb;
+#define	PPC405_ERR77_SYNC	sync;
+#else
+#define PPC405_ERR77(ra,rb)
+#define PPC405_ERR77_SYNC
+#endif
+
 #endif /* CONFIG_PPC_ISERIES */

diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/semaphore.c linux-bungo/arch/ppc/kernel/semaphore.c
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/kernel/semaphore.c	Tue Jun  5 22:20:48 2001
+++ linux-bungo/arch/ppc/kernel/semaphore.c	Thu Sep 20 16:59:10 2001
@@ -39,6 +39,7 @@
 "	srawi	%1,%0,31\n"
 "	andc	%1,%0,%1\n"
 "	add	%1,%1,%4\n"
+	PPC405_ERR77(0,%3)
 "	stwcx.	%1,0,%3\n"
 "	bne	1b"
 	: "=&r" (old_count), "=&r" (tmp), "=m" (sem->count)
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/lib/locks.c linux-bungo/arch/ppc/lib/locks.c
--- /home/dgibson/kernel/linuxppc_2_4_devel/arch/ppc/lib/locks.c	Wed Aug 29 08:57:35 2001
+++ linux-bungo/arch/ppc/lib/locks.c	Fri Sep 21 11:52:46 2001
@@ -35,8 +35,9 @@
 	__asm__ __volatile__ ("\n\
 1:	lwarx	%0,0,%1\n\
 	cmpwi	0,%0,0\n\
-	bne	2f\n\
-	stwcx.	%2,0,%1\n\
+	bne	2f\n"
+	PPC405_ERR77(0,%1)
+"	stwcx.	%2,0,%1\n\
 	bne-	1b\n\
 	isync\n\
 2:"
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/atomic.h linux-bungo/include/asm-ppc/atomic.h
--- /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/atomic.h	Wed Aug 29 08:57:35 2001
+++ linux-bungo/include/asm-ppc/atomic.h	Fri Sep 21 12:25:06 2001
@@ -23,32 +23,16 @@
 #define SMP_ISYNC
 #endif

-/* This was needed on the 4xx processors.  We don't know why, yet. -- Dan
-*/
-#ifdef CONFIG_ATOMIC_SYNC_FIX
-#define ATOMIC_SYNC "	sync\n"
+/* Erratum #77 on the 405 means we need a sync or dcbt before every stwcx.
+ * The old ATOMIC_SYNC_FIX covered some but not all of this. */
+
+#ifdef CONFIG_IBM405_ERR77
+#define PPC405_ERR77(ra,rb)	"dcbt " #ra "," #rb ";"
 #else
-#define ATOMIC_SYNC
+#define PPC405_ERR77(ra,rb)
 #endif

-#ifdef CONFIG_ATOMIC_SYNC_FIX
-static __inline__ void atomic_set(atomic_t *v, int a)
-{
-	int t;
-
-	__asm__ __volatile__("\n\
-1:	lwarx   %0,0,%3\n\
-	ori     %0,%2,0\n\
-	sync\n\
-	stwcx.  %0,0,%3\n\
-	bne-    1b"
-	: "=&r" (t), "=m" (v->counter)
-	: "r" (a), "r" (v), "m" (v->counter)
-	: "cc");
-}
-#else
 #define atomic_set(v,i)		(((v)->counter) = (i))
-#endif

 static __inline__ void atomic_add(int a, atomic_t *v)
 {
@@ -56,9 +40,9 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%3		# atomic_add\n\
-	add	%0,%2,%0\n"\
-	ATOMIC_SYNC\
-"	stwcx.	%0,0,%3\n\
+	add	%0,%2,%0\n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne-	1b"
 	: "=&r" (t), "=m" (v->counter)
 	: "r" (a), "r" (&v->counter), "m" (v->counter)
@@ -71,9 +55,9 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%2		# atomic_add_return\n\
-	add	%0,%1,%0\n"\
-	ATOMIC_SYNC\
-"	stwcx.	%0,0,%2\n\
+	add	%0,%1,%0\n"
+	PPC405_ERR77(0,%2)
+"	stwcx.	%0,0,%2 \n\
 	bne-	1b"
 	SMP_ISYNC
 	: "=&r" (t)
@@ -89,9 +73,9 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%3		# atomic_sub\n\
-	subf	%0,%2,%0\n"\
-	ATOMIC_SYNC\
-"	stwcx.	%0,0,%3\n\
+	subf	%0,%2,%0\n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne-	1b"
 	: "=&r" (t), "=m" (v->counter)
 	: "r" (a), "r" (&v->counter), "m" (v->counter)
@@ -104,9 +88,9 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%2		# atomic_sub_return\n\
-	subf	%0,%1,%0\n"\
-	ATOMIC_SYNC\
-"	stwcx.	%0,0,%2\n\
+	subf	%0,%1,%0\n"
+	PPC405_ERR77(0,%2)
+"	stwcx.	%0,0,%2 \n\
 	bne-	1b"
 	SMP_ISYNC
 	: "=&r" (t)
@@ -122,9 +106,9 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%2		# atomic_inc\n\
-	addic	%0,%0,1\n"\
-	ATOMIC_SYNC\
-"	stwcx.	%0,0,%2\n\
+	addic	%0,%0,1\n"
+	PPC405_ERR77(0,%2)
+"	stwcx.	%0,0,%2 \n\
 	bne-	1b"
 	: "=&r" (t), "=m" (v->counter)
 	: "r" (&v->counter), "m" (v->counter)
@@ -137,9 +121,9 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%1		# atomic_inc_return\n\
-	addic	%0,%0,1\n"\
-	ATOMIC_SYNC\
-"	stwcx.	%0,0,%1\n\
+	addic	%0,%0,1\n"
+	PPC405_ERR77(0,%1)
+"	stwcx.	%0,0,%1 \n\
 	bne-	1b"
 	SMP_ISYNC
 	: "=&r" (t)
@@ -155,8 +139,8 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%2		# atomic_dec\n\
-	addic	%0,%0,-1\n"\
-	ATOMIC_SYNC\
+	addic	%0,%0,-1\n"
+	PPC405_ERR77(0,%2)\
 "	stwcx.	%0,0,%2\n\
 	bne-	1b"
 	: "=&r" (t), "=m" (v->counter)
@@ -170,8 +154,8 @@

 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%1		# atomic_dec_return\n\
-	addic	%0,%0,-1\n"\
-	ATOMIC_SYNC\
+	addic	%0,%0,-1\n"
+	PPC405_ERR77(0,%1)
 "	stwcx.	%0,0,%1\n\
 	bne-	1b"
 	SMP_ISYNC
@@ -196,8 +180,8 @@
 	__asm__ __volatile__(
 "1:	lwarx	%0,0,%1		# atomic_dec_if_positive\n\
 	addic.	%0,%0,-1\n\
-	blt-	2f\n"\
-	ATOMIC_SYNC\
+	blt-	2f\n"
+	PPC405_ERR77(0,%1)
 "	stwcx.	%0,0,%1\n\
 	bne-	1b"
 	SMP_ISYNC
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/bitops.h linux-bungo/include/asm-ppc/bitops.h
--- /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/bitops.h	Sat Jun  9 12:06:37 2001
+++ linux-bungo/include/asm-ppc/bitops.h	Fri Sep 21 12:26:51 2001
@@ -11,6 +11,7 @@

 #include <linux/config.h>
 #include <asm/byteorder.h>
+#include <asm/atomic.h>

 /*
  * The test_and_*_bit operations are taken to imply a memory barrier
@@ -36,8 +37,9 @@

 	__asm__ __volatile__("\n\
 1:	lwarx	%0,0,%3 \n\
-	or	%0,%0,%2 \n\
-	stwcx.	%0,0,%3 \n\
+	or	%0,%0,%2 \n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne-	1b"
 	: "=&r" (old), "=m" (*p)
 	: "r" (mask), "r" (p), "m" (*p)
@@ -69,8 +71,9 @@

 	__asm__ __volatile__("\n\
 1:	lwarx	%0,0,%3 \n\
-	andc	%0,%0,%2 \n\
-	stwcx.	%0,0,%3 \n\
+	andc	%0,%0,%2 \n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne-	1b"
 	: "=&r" (old), "=m" (*p)
 	: "r" (mask), "r" (p), "m" (*p)
@@ -96,8 +99,9 @@

 	__asm__ __volatile__("\n\
 1:	lwarx	%0,0,%3 \n\
-	xor	%0,%0,%2 \n\
-	stwcx.	%0,0,%3 \n\
+	xor	%0,%0,%2 \n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%0,0,%3 \n\
 	bne-	1b"
 	: "=&r" (old), "=m" (*p)
 	: "r" (mask), "r" (p), "m" (*p)
@@ -126,8 +130,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%4 \n\
-	or	%1,%0,%3 \n\
-	stwcx.	%1,0,%4 \n\
+	or	%1,%0,%3 \n"
+	PPC405_ERR77(0,%4)
+"	stwcx.	%1,0,%4 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=&r" (t), "=m" (*p)
@@ -158,8 +163,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%4 \n\
-	andc	%1,%0,%3 \n\
-	stwcx.	%1,0,%4 \n\
+	andc	%1,%0,%3 \n"
+	PPC405_ERR77(0,%4)
+"	stwcx.	%1,0,%4 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=&r" (t), "=m" (*p)
@@ -190,8 +196,9 @@

 	__asm__ __volatile__(SMP_WMB "\n\
 1:	lwarx	%0,0,%4 \n\
-	xor	%1,%0,%3 \n\
-	stwcx.	%1,0,%4 \n\
+	xor	%1,%0,%3 \n"
+	PPC405_ERR77(0,%4)
+"	stwcx.	%1,0,%4 \n\
 	bne	1b"
 	SMP_MB
 	: "=&r" (old), "=&r" (t), "=m" (*p)
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/pgtable.h linux-bungo/include/asm-ppc/pgtable.h
--- /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/pgtable.h	Tue Sep 18 11:25:44 2001
+++ linux-bungo/include/asm-ppc/pgtable.h	Fri Sep 21 12:30:01 2001
@@ -442,8 +440,9 @@
 	__asm__ __volatile__("\
 1:	lwarx	%0,0,%3\n\
 	andc	%1,%0,%4\n\
-	or	%1,%1,%5\n\
-	stwcx.	%1,0,%3\n\
+	or	%1,%1,%5\n"
+	PPC405_ERR77(0,%3)
+"	stwcx.	%1,0,%3\n\
 	bne-	1b"
 	: "=&r" (old), "=&r" (tmp), "=m" (*p)
 	: "r" (p), "r" (clr), "r" (set), "m" (*p)
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/spinlock.h linux-bungo/include/asm-ppc/spinlock.h
--- /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/spinlock.h	Wed Aug 29 08:57:36 2001
+++ linux-bungo/include/asm-ppc/spinlock.h	Fri Sep 21 12:25:06 2001
@@ -46,8 +46,9 @@
 	bne+	2b\n\
 1:	lwarx	%0,0,%1\n\
 	cmpwi	0,%0,0\n\
-	bne-	2b\n\
-	stwcx.	%2,0,%1\n\
+	bne-	2b\n"
+	PPC405_ERR77(0,%1)
+"	stwcx.	%2,0,%1\n\
 	bne-	2b\n\
 	isync"
 	: "=&r"(tmp)
@@ -114,8 +115,9 @@
 	blt+		1b\n\
 2:	lwarx		%0,0,%1\n\
 	addic.		%0,%0,1\n\
-	ble-		1b\n\
-	stwcx.		%0,0,%1\n\
+	ble-		1b\n"
+	PPC405_ERR77(0,%1)
+"	stwcx.		%0,0,%1\n\
 	bne-		2b\n\
 	isync"
 	: "=&r"(tmp)
@@ -130,8 +132,9 @@
 	__asm__ __volatile__(
 	"eieio				# read_unlock\n\
 1:	lwarx		%0,0,%1\n\
-	addic		%0,%0,-1\n\
-	stwcx.		%0,0,%1\n\
+	addic		%0,%0,-1\n"
+	PPC405_ERR77(0,%1)
+"	stwcx.		%0,0,%1\n\
 	bne-		1b"
 	: "=&r"(tmp)
 	: "r"(&rw->lock)
@@ -149,8 +152,9 @@
 	bne+		1b\n\
 2:	lwarx		%0,0,%1\n\
 	cmpwi		0,%0,0\n\
-	bne-		1b\n\
-	stwcx.		%2,0,%1\n\
+	bne-		1b\n"
+	PPC405_ERR77(0,%1)
+"	stwcx.		%2,0,%1\n\
 	bne-		2b\n\
 	isync"
 	: "=&r"(tmp)
diff -urN /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/system.h linux-bungo/include/asm-ppc/system.h
--- /home/dgibson/kernel/linuxppc_2_4_devel/include/asm-ppc/system.h	Thu Sep 20 09:19:15 2001
+++ linux-bungo/include/asm-ppc/system.h	Fri Sep 21 12:25:06 2001
@@ -130,8 +130,9 @@
 	unsigned long prev;

 	__asm__ __volatile__ ("\n\
-1:	lwarx	%0,0,%2 \n\
-	stwcx.	%3,0,%2 \n\
+1:	lwarx	%0,0,%2 \n"
+	PPC405_ERR77(0,%2)
+"	stwcx.	%3,0,%2 \n\
 	bne-	1b"
 	: "=&r" (prev), "=m" (*(volatile unsigned long *)p)
 	: "r" (p), "r" (val), "m" (*(volatile unsigned long *)p)
@@ -181,8 +182,9 @@
 	__asm__ __volatile__ ("\n\
 1:	lwarx	%0,0,%2 \n\
 	cmpw	0,%0,%3 \n\
-	bne	2f \n\
-	stwcx.	%4,0,%2 \n\
+	bne	2f \n"
+	PPC405_ERR77(0,%2)
+"	stwcx.	%4,0,%2 \n\
 	bne-	1b\n"
 #ifdef CONFIG_SMP
 "	sync\n"


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Dan Malek]

David Gibson wrote:

> Try the below...

Did it solve your system lockup problem?


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[David Gibson]

On Fri, Sep 21, 2001 at 01:23:06AM -0400, Dan Malek wrote:
>
> David Gibson wrote:
>
> > Try the below...
>
> Did it solve your system lockup problem?

Well I haven't managed to lock a process up yet, which isn't to say
that the problem is entirely fixed since I haven't pushed it really,
really hard yet, but it's certainly improved from what it was before.

--
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.  -- H.L. Mencken
http://www.ozlabs.org/people/dgibson


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Dan Malek]

David Gibson wrote:

> Well I haven't managed to lock a process up yet,

Cool.  I'll patch it and push it in right now.

Thanks.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Dan Malek]

David Gibson wrote:

> ... but it's certainly improved from what it was before.

I pushed the patch.  It would be nice if you could send patches
against the latest version of the tree, like do a pull and test
it just before you post it.  There were parts of the patch that
wouldn't apply and I had to do it manually.  Check it to ensure
I got everything OK.

Thanks.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[dank]

Mark Hatle <fray@mvista.com> wrote on 18 Sep 2001:
> David Gibson wrote:
> > Ah, yes, I discovered ATOMIC_SYNC_FIX after I sent that, and have now
> > turned it on. That should certainly fix the atomic ops, however there
> > are quite a number of other places where the kernel uses stwcx., which
> > ATOMIX_SYNC_FIX doesn't fix - notably arch/ppc/kernel/bitops.c and
> > include/asm-ppc/bitops.h. As well as activating ATOMIX_SYNC_FIX I
> > tried inserting a sync before every other stwcx. that I could find,
> > and I haven't managed to get a process to lock up yet.
>
> Just as an FYI, we also do this in glibc to be safe. We have never been
> able to pin down a problem in userspace due to this bug, but we thought
> it would be better safe then sorry until we can get definative proof
> that the bug will not happen in userspace.
>
> The following two files in glibc should be patched:
> linuxthreads/sysdeps/powerpc/pt-machine.h
> sysdeps/powerpc/atomicity.h

We may have an example of a problem in userspace caused by erratum 77.
We've been using a version of gcc3.0.2 running with your patched glibc, and
had a very mysterious crash that happened roughly once in 1000 minutes,
but only with a lucky build of our c++ app.  The problem disappeared
if you looked at it in any way.  Fortunately (?), we had a rack of 100
CPUs running the app, so the MTBF was 10 minutes.

It looks like the following file in gcc3 must be patched:
 gcc-3.0.2/libstdc++-v3/config/cpu/powerpc/bits/atomicity.h
Here's my and Jan's attempt at a patch:
 http://gcc.gnu.org/ml/libstdc++/2002-07/msg00152.html

Have your patches to glibc been contributed back to the mainline yet?
I'm looking carefully about how best to contribute my change;
I think it ought to be a multilib that is explicitly enabled.
- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Mark Hatle]

The atonicity patches had not been submitted back to glibc due to there being
now way for me to show it was needed, and also that it ONLY affects the 405 CPU,
which isn't the main target of glibc.

We are currently working on revising our glibc patches to the current CVS
version, and if a new patch is required I'll make sure it gets posted here.  I
really don't know the best way to handle this in a community glibc/gcc realm.
I'd almost like to wait and see what the GCC maintainers response is.
Specifically how they are going to accept the patch.  Then we propose a similar
thing to the glibc folks, explain the problem and hope they accept the patch as
well.  The biggest problem of course is that if we enable this for all PPCs
we're going to take a performance penalty for CPUs that don't have the problem.
As a community member I would hate to have to take the performance hit on my
750/74xx machines because of "some little embedded CPU".  (Yes thats how I have
heard them described by others in similar situations.)

Any suggestions, anything I can do to help I am more then willing to just let me
know.

--Mark

dank@kegel.com wrote:
>
> Mark Hatle <fray@mvista.com> wrote on 18 Sep 2001:
> > David Gibson wrote:
> > > Ah, yes, I discovered ATOMIC_SYNC_FIX after I sent that, and have now
> > > turned it on. That should certainly fix the atomic ops, however there
> > > are quite a number of other places where the kernel uses stwcx., which
> > > ATOMIX_SYNC_FIX doesn't fix - notably arch/ppc/kernel/bitops.c and
> > > include/asm-ppc/bitops.h. As well as activating ATOMIX_SYNC_FIX I
> > > tried inserting a sync before every other stwcx. that I could find,
> > > and I haven't managed to get a process to lock up yet.
> >
> > Just as an FYI, we also do this in glibc to be safe. We have never been
> > able to pin down a problem in userspace due to this bug, but we thought
> > it would be better safe then sorry until we can get definative proof
> > that the bug will not happen in userspace.
> >
> > The following two files in glibc should be patched:
> > linuxthreads/sysdeps/powerpc/pt-machine.h
> > sysdeps/powerpc/atomicity.h
>
> We may have an example of a problem in userspace caused by erratum 77.
> We've been using a version of gcc3.0.2 running with your patched glibc, and
> had a very mysterious crash that happened roughly once in 1000 minutes,
> but only with a lucky build of our c++ app.  The problem disappeared
> if you looked at it in any way.  Fortunately (?), we had a rack of 100
> CPUs running the app, so the MTBF was 10 minutes.
>
> It looks like the following file in gcc3 must be patched:
>  gcc-3.0.2/libstdc++-v3/config/cpu/powerpc/bits/atomicity.h
> Here's my and Jan's attempt at a patch:
>  http://gcc.gnu.org/ml/libstdc++/2002-07/msg00152.html
>
> Have your patches to glibc been contributed back to the mainline yet?
> I'm looking carefully about how best to contribute my change;
> I think it ought to be a multilib that is explicitly enabled.
> - Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[dank]

Mark Hatle wrote:
> The atonicity patches had not been submitted back to glibc due to there being
> now way for me to show it was needed

That *is* a problem, isn't it?  We have a test case, but not one we
can distribute.  I'm trying to create a clean one that can be distributed,
but this bug is hard to trigger.

> and also that it ONLY affects the 405 CPU,
> which isn't the main target of glibc.
>
> We are currently working on revising our glibc patches to the current CVS
> version, and if a new patch is required I'll make sure it gets posted here.  I
> really don't know the best way to handle this in a community glibc/gcc realm.
> I'd almost like to wait and see what the GCC maintainers response is.
> Specifically how they are going to accept the patch.  Then we propose a similar
> thing to the glibc folks, explain the problem and hope they accept the patch as
> well.  The biggest problem of course is that if we enable this for all PPCs
> we're going to take a performance penalty for CPUs that don't have the problem.

The patch can be made conditional, so that by default it affects nothing.
The libstdc++ mailing list has already blessed the part of the change
that affects them.

Here's how to do it:

1. Arrange for gcc to define __PPC405__ when appopriate by changing
gcc/config/rs6000/rs6000.h:

-%{mcpu=405: -D_ARCH_PPC} \
+%{mcpu=405: -D_ARCH_PPC -D__PPC405__} \

2. Surround the workaround with #ifdef __PPC405__.  For instance,
in gcc-3.0.2/libstdc++-v3/config/cpu/powerpc/bits/atomicity.h:

+#ifdef __PPC405__
+#define        __LIBSTDCPP_PPC405_ERR77_SYNC   "sync \n\t"
+#else
+#define __LIBSTDCPP_PPC405_ERR77_SYNC
+#endif
+
 static inline _Atomic_word
 __attribute__ ((__unused__))
 __exchange_and_add (volatile _Atomic_word* __mem, int __val)
@@ -42,6 +54,7 @@
        "0:\t"
        "lwarx    %0,0,%2 \n\t"
        "add%I3   %1,%0,%3 \n\t"
+       __LIBSTDCPP_PPC405_ERR77_SYNC

3. Arrange for ppc405-specific libraries to be generated.  I'm
still working on this, but it's something like

--- gcc/config/rs6000/t-ppcos.orig
+++ gcc/config/rs6000/t-ppcos
@@ -1,7 +1,7 @@
 # Multilibs for a powerpc hosted ELF target (linux, SVR4, solaris)

-MULTILIB_OPTIONS       = msoft-float
-MULTILIB_DIRNAMES      = nof
+MULTILIB_OPTIONS       = msoft-float/mcpu=405
+MULTILIB_DIRNAMES      = nof ppc405

4. Arrange to suppress generation of ppc405-specific libraries by default.
I have yet to do this, but it's probably a change to gcc/config-ml.in
to add an --enable-ppc405 option.

I could use help in two areas:
figuring out how to generate many interrupts on the ppc405 to make
a regression test more likely to run into the problem (see my previous
post to linuxppc-embedded), and
figuring out how gcc's MULTILIB_EXCEPTIONS parameter (I'll be posting
to the gcc list about that, so hopefully some expert there will
give me a clue).

Oh, and it'd be nice to hear what you think of my patch to gcc3, too.

It'd be nice to get ppc405 support in the gcc/glibc mainline, glad to hear
you're working on it too!
- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Mark Hatle]

dank@kegel.com wrote:
>
> Mark Hatle wrote:
> > The atonicity patches had not been submitted back to glibc due to there being
> > now way for me to show it was needed
>
> That *is* a problem, isn't it?  We have a test case, but not one we
> can distribute.  I'm trying to create a clean one that can be distributed,
> but this bug is hard to trigger.

Well at least you have hit the problem, so at least it is shown to affect
someone.  BTW what version of the kernel sources are you using?

> The patch can be made conditional, so that by default it affects nothing.
> The libstdc++ mailing list has already blessed the part of the change
> that affects them.
>
> Here's how to do it:
>
> 1. Arrange for gcc to define __PPC405__ when appopriate by changing
> gcc/config/rs6000/rs6000.h:
>
> -%{mcpu=405: -D_ARCH_PPC} \
> +%{mcpu=405: -D_ARCH_PPC -D__PPC405__} \
>
> 2. Surround the workaround with #ifdef __PPC405__.  For instance,
> in gcc-3.0.2/libstdc++-v3/config/cpu/powerpc/bits/atomicity.h:
>

.....

Ya that all makes sense to me, and from the couple of threads I saw on the gcc
mailing list at least it seems to be acceptable to those folks as well.

As far as the multilib stuff goes I'll leave that to the gcc experts, I really
don't understand how the multilib stuff works.  (We don't use it, we rebuild the
compiler to target the architecture itself instead of building one gcc that can
target all of the PPC archs.)

> 3. Arrange for ppc405-specific libraries to be generated.  I'm
> still working on this, but it's something like
> ...
>
> 4. Arrange to suppress generation of ppc405-specific libraries by default.
> I have yet to do this, but it's probably a change to gcc/config-ml.in
> to add an --enable-ppc405 option.

That is probably for the best.

> I could use help in two areas:
> figuring out how to generate many interrupts on the ppc405 to make
> a regression test more likely to run into the problem (see my previous
> post to linuxppc-embedded), and

One suggestion is to create a special kernel that has a hacked timer.c function
to generate a heck of a lot more timer interrupts..  But I don't know the
specifics on the kernel or the 405 CPU line to say if that is really practical
or not.  The other folks in the embedded list would be better then me at that.

> figuring out how gcc's MULTILIB_EXCEPTIONS parameter (I'll be posting
> to the gcc list about that, so hopefully some expert there will
> give me a clue).

It's very much voodoo magic, but unfortunatly I don't really understand it
either.  Sparc, ARM and MIPS might be good examples as they seem to have many
more multilib modes then PPC.  (Arm especially w/ the thumb modes).

> Oh, and it'd be nice to hear what you think of my patch to gcc3, too.

Everything looks fine to me.  It definatly looks correct.  (In addition someone
should suggest that the libjava atomic operation be fixed as well.. not sure how
many people would use the GCC libjava stuff on 405 though....)

> It'd be nice to get ppc405 support in the gcc/glibc mainline, glad to hear
> you're working on it too!

I'm just not holding my breath for glibc support, I'm a bit pesimistic about the
PPC glibc maintainer(s).  I've never had great luck submitting patches and
getting them accepted.  (But it may just be me or the problems I always seem to
run into....)

If you have access to irc, log in to irc.openprojects.net and "#mklinux".

--Mark

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[dank]

Mark Hatle wrote:
> dank@kegel.com wrote:
> > We have a test case, but not one we can distribute....
>
> Well at least you have hit the problem, so at least it is shown to affect
> someone.  BTW what version of the kernel sources are you using?

We're using 2.4.17 from linuxppc_2_4_devel as of late December 2001.
We do not yet have confirmation that we're running into erratum 77, but
it seems likely.

> > The patch can be made conditional, so that by default it affects nothing.
> >
> > 1. Arrange for gcc to define __PPC405__ when appopriate by changing
> > gcc/config/rs6000/rs6000.h:
> >
> > -%{mcpu=405: -D_ARCH_PPC} \
> > +%{mcpu=405: -D_ARCH_PPC -D__PPC405__} \
> >
> > 2. Surround the workaround with #ifdef __PPC405__.  For instance,
> > in gcc-3.0.2/libstdc++-v3/config/cpu/powerpc/bits/atomicity.h:
> .....
>
> Ya that all makes sense to me, and from the couple of threads I saw on the gcc
> mailing list at least it seems to be acceptable to those folks as well.
>
> As far as the multilib stuff goes I'll leave that to the gcc experts, I really
> don't understand how the multilib stuff works.  (We don't use it, we rebuild the
> compiler to target the architecture itself instead of building one gcc that can
> target all of the PPC archs.)

We don't use multilib, either -- well, that's not quite true.  When we
build gcc for the ppc405, we grab the libraries from the 'nof' directory.
Turns out that's multilib in action.  So all I'm doing is arranging
for a second directory, ppc405, to be generated next to nof.  Like nof,
it has softfloat, but it also will have __PPC405__ turned on during library
build, which will trigger the workaround.

> > I could use help in two areas:
> > figuring out how to generate many interrupts on the ppc405 to make
> > a regression test more likely to run into the problem (see my previous
> > post to linuxppc-embedded), and
>
> One suggestion is to create a special kernel that has a hacked timer.c function
> to generate a heck of a lot more timer interrupts..  But I don't know the
> specifics on the kernel or the 405 CPU line to say if that is really practical
> or not.  The other folks in the embedded list would be better then me at that.

OK.  Can some ppc405 expert speak up?

> > Oh, and it'd be nice to hear what you think of my patch to gcc3, too.
>
> Everything looks fine to me.  It definatly looks correct.  (In addition someone
> should suggest that the libjava atomic operation be fixed as well.. not sure how
> many people would use the GCC libjava stuff on 405 though....)

Thanks.  If I get a chance, I'll look at libjava/sysdep/powerpc/locks.h, too.

> > It'd be nice to get ppc405 support in the gcc/glibc mainline, glad to hear
> > you're working on it too!
>
> I'm just not holding my breath for glibc support ...

glibc is notoriously difficult to contribute to.  I would be satisfied
with a separately maintained patch that applies to the latest glibc,
but I am optimistic that if I dot all my i's and cross my t's, I
can get a patch accepted.

- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[dank]

Mark Hatle wrote:
>
> The atonicity patches had not been submitted back to glibc due to there being
> now way for me to show it was needed, and also that it ONLY affects the 405 CPU,
> which isn't the main target of glibc.
>
> We are currently working on revising our glibc patches to the current CVS
> version, and if a new patch is required I'll make sure it gets posted here.  I
> really don't know the best way to handle this in a community glibc/gcc realm.
> I'd almost like to wait and see what the GCC maintainers response is.
> Specifically how they are going to accept the patch.  Then we propose a similar
> thing to the glibc folks, explain the problem and hope they accept the patch as
> well.

Mark,
I've rediffed your glibc patch and made it conditional on defined(__PPC405__);
result at http://www.kegel.com/xgcc3/glibc-2.2.5-ppc405erratum77.patch
I haven't tested it yet, but something like that should make the maintainers happy.
What do you think?

Also, according to http://gcc.gnu.org/contribute.html, patches for bugs
are more likely to be accepted if there is a bug report in gnats,
so I opened a bug for stdlibc++
http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=7383
and one for glibc at
http://bugs.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&pr=4155

They also won't accept patches unless they've been thoroughly tested,
so let's agree on gcc and glibc patches, and use them for a few months.
If no problems pop up, let's submit them.

I've updated http://www.kegel.com/xgcc3/ppc405erratum77.html with the
above information.
- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Errata 67/77 / walnut bugs (was: Re: Erratum 51 bugfix?)

[Mark Hatle]

dank@kegel.com wrote:
>
> Mark Hatle wrote:
> >
> > The atonicity patches had not been submitted back to glibc due to there being
> > now way for me to show it was needed, and also that it ONLY affects the 405 CPU,
> > which isn't the main target of glibc.
> >
> > We are currently working on revising our glibc patches to the current CVS
> > version, and if a new patch is required I'll make sure it gets posted here.  I
> > really don't know the best way to handle this in a community glibc/gcc realm.
> > I'd almost like to wait and see what the GCC maintainers response is.
> > Specifically how they are going to accept the patch.  Then we propose a similar
> > thing to the glibc folks, explain the problem and hope they accept the patch as
> > well.
>
> Mark,
> I've rediffed your glibc patch and made it conditional on defined(__PPC405__);
> result at http://www.kegel.com/xgcc3/glibc-2.2.5-ppc405erratum77.patch
> I haven't tested it yet, but something like that should make the maintainers happy.
> What do you think?

Sounds good..  (as I said before we're in the process of re doing our diff for
2.2.5 as well, so I'll compare them and make sure neither of us missed
anything.  We havn't added the __PPC405__ bit yet, but planned on doing so
soon.)

> Also, according to http://gcc.gnu.org/contribute.html, patches for bugs
> are more likely to be accepted if there is a bug report in gnats,
> so I opened a bug for stdlibc++
> http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=7383
> and one for glibc at
> http://bugs.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&pr=4155
>
> They also won't accept patches unless they've been thoroughly tested,
> so let's agree on gcc and glibc patches, and use them for a few months.
> If no problems pop up, let's submit them.
>
> I've updated http://www.kegel.com/xgcc3/ppc405erratum77.html with the
> above information.

Again, sounds good.

--Mark

> - Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/