New 745x errata

New 745x errata

[Adrian Cox]

Since October there are new errata on Motorola's website for the 745x
family processors. The following patch against linuxppc-2.4-devel
contains workarounds to two problems:

1) The BTIC doesn't work reliably: it can cause execution of corrupted
instructions. (This is listed in the errata for the 7450 and 7457, but
not for the 7455. I'll only believe that the 7455 escaped this bug if I
hear confirmation out of Motorola.)
2) dcbt to protected addresses can cause spurious bus cycles. This could
potentially be exploited from userspace to lock up hardware.

- Adrian Cox
http://www.humboldt.co.uk/

===== arch/ppc/kernel/cpu_setup_6xx.S 1.3 vs edited =====
--- 1.3/arch/ppc/kernel/cpu_setup_6xx.S	Fri Jul 18 20:53:16 2003
+++ edited/arch/ppc/kernel/cpu_setup_6xx.S	Thu Nov 13 10:51:52 2003
@@ -191,9 +191,10 @@

 /* MPC 745x
  * Enable Store Gathering (SGE), Branch Folding (FOLD)
- * Branch History Table (BHTE), Branch Target ICache (BTIC)
+ * Branch History Table (BHTE)
  * Dynamic Power Management (DPM), Speculative (SPD)
- * Ensure our data cache instructions really operate.
+ * Disable Branch Target ICache (BTIC) and data cache touch instructions to
+ * work around published chip errata.
  * Timebase has to be running or we wouldn't have made it here,
  * just ensure we don't disable it.
  * Clear Instruction cache throttling (ICTC)
@@ -218,14 +219,14 @@

 	/* All of the bits we have to set.....
 	 */
-	ori	r11,r11,HID0_SGE | HID0_FOLD | HID0_BHTE | HID0_BTIC | HID0_LRSTK
+	ori	r11,r11,HID0_SGE | HID0_FOLD | HID0_BHTE | HID0_LRSTK | HID0_NOPTI
 BEGIN_FTR_SECTION
 	oris	r11,r11,HID0_DPM@h	/* enable dynamic power mgmt */
 END_FTR_SECTION_IFCLR(CPU_FTR_NO_DPM)

 	/* All of the bits we have to clear....
 	 */
-	li	r3,HID0_SPD | HID0_NOPDST | HID0_NOPTI
+	li	r3,HID0_SPD | HID0_NOPDST | HID0_BTIC
 	andc	r11,r11,r3		/* clear SPD: enable speculative */
  	li	r3,0


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

RE: New 745x errata

[Giuliano Pochini]

On 13-Nov-2003 Adrian Cox wrote:
>
> Since October there are new errata on Motorola's website for the 745x
> family processors. The following patch against linuxppc-2.4-devel
> contains workarounds to two problems:
>
> 1) The BTIC doesn't work reliably: it can cause execution of corrupted
> instructions. (This is listed in the errata for the 7450 and 7457, but
> not for the 7455. I'll only believe that the 7455 escaped this bug if I
> hear confirmation out of Motorola.)

Did you try to ask them about it ?

> 2) dcbt to protected addresses can cause spurious bus cycles. This could
> potentially be exploited from userspace to lock up hardware.

Hm, a 1-cycle window this is very small.  Anyway, if dc* instructions are
used inside the kernel, we could keep them disabled in userspace only.


--
Giuliano.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

RE: New 745x errata

[Adrian Cox]

On Fri, 2003-11-14 at 10:40, Giuliano Pochini wrote:

> > 1) The BTIC doesn't work reliably: it can cause execution of corrupted
> > instructions. (This is listed in the errata for the 7450 and 7457, but
> > not for the 7455. I'll only believe that the 7455 escaped this bug if I
> > hear confirmation out of Motorola.)
>
> Did you try to ask them about it ?

Yes, but an answer to this sort of question will normally take a few
days.  I thought it was worth raising the issue on the mailing list now.

> > 2) dcbt to protected addresses can cause spurious bus cycles. This could
> > potentially be exploited from userspace to lock up hardware.
>
> Hm, a 1-cycle window this is very small.  Anyway, if dc* instructions are
> used inside the kernel, we could keep them disabled in userspace only.

The kernel uses the dc* instructions in copy_page, and in prefetches
everywhere. These uses should all be safe, unless any driver prefetches
an unchecked userspace pointer. The only problem is that we don't
currently switch the value of HID0 when entering and leaving userspace.

- Adrian


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Tom Rini]

On Thu, Nov 13, 2003 at 11:05:17AM +0000, Adrian Cox wrote:
>
> Since October there are new errata on Motorola's website for the 745x
> family processors. The following patch against linuxppc-2.4-devel
> contains workarounds to two problems:
>
> 1) The BTIC doesn't work reliably: it can cause execution of corrupted
> instructions. (This is listed in the errata for the 7450 and 7457, but
> not for the 7455. I'll only believe that the 7455 escaped this bug if I
> hear confirmation out of Motorola.)

I've done the opposite in something I've been waiting for 2.4.24-pre to
come out to fix.  But I agree that it is odd that it's not listed on the
7455.

diff -Nru a/arch/ppc/kernel/cpu_setup_6xx.S b/arch/ppc/kernel/cpu_setup_6xx.S
--- a/arch/ppc/kernel/cpu_setup_6xx.S	Fri Nov 14 09:23:24 2003
+++ b/arch/ppc/kernel/cpu_setup_6xx.S	Fri Nov 14 09:23:24 2003
@@ -104,8 +104,8 @@

 /* 7400 <= rev 2.7 and 7410 rev = 1.0 suffer from some
  * erratas we work around here.
- * Moto MPC710CE.pdf describes them, those are errata
- * #3, #4 and #5
+ * Moto MPC7410CE.pdf describes them, those are errata
+ * #3, #4 and #5 (7400 erratas #13, #14 and #15).
  * Note that we assume the firmware didn't choose to
  * apply other workarounds (there are other ones documented
  * in the .pdf). It appear that Apple firmware only works
@@ -155,7 +155,20 @@
 BEGIN_FTR_SECTION
 	oris	r11,r11,HID0_DPM@h	/* enable dynamic power mgmt */
 END_FTR_SECTION_IFCLR(CPU_FTR_NO_DPM)
-	li	r3,HID0_SPD
+
+	/* 7400 Errata #7, don't enable SGE on < Rev 2.1 */
+	mfpvr	r10
+	rlwinm	r3,r10,16,16,31
+	cmplwi	r3,0x000c
+	bne	1f			/* Not a 7400. */
+	andi	r3,r10,0x0f0f
+	cmpwi	0,r3,0x0200
+	bgt	1f			/* Rev >= 2.1 */
+	li	r3,HID0_SGE		/* 7400 rev < 2.1, clear SGE. */
+	b	2f
+
+1:	li	r3,0			/* Don't clear SGE. */
+2:	ori	r3,r3,HID0_SPD
 	andc	r11,r11,r3		/* clear SPD: enable speculative */
  	li	r3,0
  	mtspr	ICTC,r3			/* Instruction Cache Throttling off */
@@ -223,9 +236,17 @@
 	oris	r11,r11,HID0_DPM@h	/* enable dynamic power mgmt */
 END_FTR_SECTION_IFCLR(CPU_FTR_NO_DPM)

+	/* We only want to enable BTIC on 7455's */
+	mfpvr	r3
+	rlwinm	r3,r3,16,16,31
+	cmplwi	r3,0x8001
+	li	r3,HID0_BTIC		/* Assume we want to clear BTIC. */
+	bne	2f			/* Not a 7455. */
+	li	r3,0			/* 7455, don't clear BTIC. */
+
 	/* All of the bits we have to clear....
 	 */
-	li	r3,HID0_SPD | HID0_NOPDST | HID0_NOPTI
+2:	ori	r3,r3,HID0_SPD | HID0_NOPDST | HID0_NOPTI
 	andc	r11,r11,r3		/* clear SPD: enable speculative */
  	li	r3,0

diff -Nru a/arch/ppc/kernel/cputable.c b/arch/ppc/kernel/cputable.c
--- a/arch/ppc/kernel/cputable.c	Fri Nov 14 09:23:24 2003
+++ b/arch/ppc/kernel/cputable.c	Fri Nov 14 09:23:24 2003
@@ -202,6 +202,15 @@
 	32, 32,
 	__setup_cpu_7410
     },
+    {	/* 7450 1.x - no doze/nap */
+    	0xffffff00, 0x80000100, "7450",
+    	CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_USE_TB |
+	CPU_FTR_L2CR | CPU_FTR_ALTIVEC_COMP | CPU_FTR_L3CR |
+	CPU_FTR_HPTE_TABLE | CPU_FTR_SPEC7450,
+	COMMON_PPC | PPC_FEATURE_HAS_ALTIVEC,
+	32, 32,
+	__setup_cpu_745x
+    },
     {	/* 7450 2.0 - no doze/nap */
     	0xffffffff, 0x80000200, "7450",
     	CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_USE_TB |

--
Tom Rini
http://gate.crashing.org/~trini/

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Adrian Cox]

On Fri, 2003-11-14 at 16:24, Tom Rini wrote:
> On Thu, Nov 13, 2003 at 11:05:17AM +0000, Adrian Cox wrote:
> > 1) The BTIC doesn't work reliably: it can cause execution of corrupted
> > instructions. (This is listed in the errata for the 7450 and 7457, but
> > not for the 7455. I'll only believe that the 7455 escaped this bug if I
> > hear confirmation out of Motorola.)
>
> I've done the opposite in something I've been waiting for 2.4.24-pre to
> come out to fix.  But I agree that it is odd that it's not listed on the
> 7455.

Still no word back from Motorola on the 7455. I think your patch should
go in.

Any opinion on the dcbt issue?  It looks like it could provide a way for
a malicious userspace application to crash the machine, though it needs
a combination of:
1) good timing
2) a peripheral that would be confused by an extra read cycle

- Adrian Cox
http://www.humboldt.co.uk/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Tom Rini]

On Mon, Nov 17, 2003 at 02:57:53PM +0000, Adrian Cox wrote:
> On Fri, 2003-11-14 at 16:24, Tom Rini wrote:
> > On Thu, Nov 13, 2003 at 11:05:17AM +0000, Adrian Cox wrote:
> > > 1) The BTIC doesn't work reliably: it can cause execution of corrupted
> > > instructions. (This is listed in the errata for the 7450 and 7457, but
> > > not for the 7455. I'll only believe that the 7455 escaped this bug if I
> > > hear confirmation out of Motorola.)
> >
> > I've done the opposite in something I've been waiting for 2.4.24-pre to
> > come out to fix.  But I agree that it is odd that it's not listed on the
> > 7455.
>
> Still no word back from Motorola on the 7455. I think your patch should
> go in.
>
> Any opinion on the dcbt issue?  It looks like it could provide a way for
> a malicious userspace application to crash the machine, though it needs
> a combination of:
> 1) good timing
> 2) a peripheral that would be confused by an extra read cycle

[ My knowledge of Motorola CPU errata has been temprorarily replaced
with other things, so I don't recall the details just yet of the dbct
issue ]
It depends on what the fix is.  For example, Linus just fixed a
theoretical race that has a 1-3 cycle (depending on x86 cpu I think he
said) race because he happened to be in there anyhow (and it turns out
the fix is easy).

--
Tom Rini
http://gate.crashing.org/~trini/

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Gabriel Paubert]

On Mon, Nov 17, 2003 at 02:57:53PM +0000, Adrian Cox wrote:
>
> On Fri, 2003-11-14 at 16:24, Tom Rini wrote:
> > On Thu, Nov 13, 2003 at 11:05:17AM +0000, Adrian Cox wrote:
> > > 1) The BTIC doesn't work reliably: it can cause execution of
> > > corrupted instructions. (This is listed in the errata for the 7450
> > > and 7457, but not for the 7455. I'll only believe that the 7455
> > > escaped this bug if I hear confirmation out of Motorola.)
> >
> > I've done the opposite in something I've been waiting for 2.4.24-pre
> > to come out to fix. But I agree that it is odd that it's not listed
> > on the 7455.
>
> Still no word back from Motorola on the 7455. I think your patch
> should go in.
>
> Any opinion on the dcbt issue? It looks like it could provide a way
> for a malicious userspace application to crash the machine, though it
> needs a combination of:
> 1) good timing
> 2) a peripheral that would be confused by an extra read cycle

Well, only privileged applications should have access to
peripherals, no?

If a malicious application can map a peripheral, you have more
problems than crashing the machine, which is annoying but _nothing_
compared to the exposure of confidential data.

dcbt can improve performance in some cases, and the bug does not
affect an application which only has access to memory with default
attributes (write-back caching).

But maybe I miss something.

	Gabriel

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Adrian Cox]

On Mon, 2003-11-17 at 15:04, Tom Rini wrote:
> On Mon, Nov 17, 2003 at 02:57:53PM +0000, Adrian Cox wrote:

> > Any opinion on the dcbt issue?  It looks like it could provide a way for
> > a malicious userspace application to crash the machine, though it needs
> > a combination of:
> > 1) good timing
> > 2) a peripheral that would be confused by an extra read cycle

> It depends on what the fix is.  For example, Linus just fixed a
> theoretical race that has a 1-3 cycle (depending on x86 cpu I think he
> said) race because he happened to be in there anyhow (and it turns out
> the fix is easy).

The fix is simple, but carries a performance cost: set the HID0_NOPTI
bit and disable all dcbt instructions. The kernel uses dcbt instructions
in a lot of places, but copy_page is what matters the most.

A copy_page routine using dst instructions might be faster anyway, and
would be immune to this problem.

On closer reading, I now think this problem may be harmless - see my
explanation to Gabriel Paubert for more details.

- Adrian Cox
http://www.humboldt.co.uk/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Adrian Cox]

On Mon, 2003-11-17 at 15:12, Gabriel Paubert wrote:
> On Mon, Nov 17, 2003 at 02:57:53PM +0000, Adrian Cox wrote:
> > Any opinion on the dcbt issue?  It looks like it could provide a way for
> > a malicious userspace application to crash the machine, though it needs
> > a combination of:
> > 1) good timing
> > 2) a peripheral that would be confused by an extra read cycle

> Well, only privileged applications should have access to
> peripherals, no?
[...]
> But maybe I miss something.

That's the bug - a dcbt to a protected region can cause a spurious read
cycle to that address. To trigger it:

1) the target address is in a BAT or TLB, marked as supervisor access
only.
2) a cache miss to a cache alias of the target address reaches the
load-store unit
2) you issue a dcbt to the target address within 1 clock cycle of step
2.

Actually, I now believe the bug may be harmless, as the peripheral has
an extra defence - its BAT or TLB entry will be non-cacheable, so no bus
cycle will occur. The text of the errata doesn't spell this out as
clearly as I'd like, but I think all it can do is cause a spurious bus
cycle to ram.

- Adrian Cox
http://www.humboldt.co.uk/


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Gabriel Paubert]

On Mon, Nov 17, 2003 at 03:37:00PM +0000, Adrian Cox wrote:
> On Mon, 2003-11-17 at 15:12, Gabriel Paubert wrote:
> > On Mon, Nov 17, 2003 at 02:57:53PM +0000, Adrian Cox wrote:
> > > Any opinion on the dcbt issue?  It looks like it could provide a way for
> > > a malicious userspace application to crash the machine, though it needs
> > > a combination of:
> > > 1) good timing
> > > 2) a peripheral that would be confused by an extra read cycle
>
> > Well, only privileged applications should have access to
> > peripherals, no?
> [...]
> > But maybe I miss something.
>
> That's the bug - a dcbt to a protected region can cause a spurious read
> cycle to that address. To trigger it:
>
> 1) the target address is in a BAT or TLB, marked as supervisor access
> only.
> 2) a cache miss to a cache alias of the target address reaches the
> load-store unit
> 2) you issue a dcbt to the target address within 1 clock cycle of step
> 2.
>
> Actually, I now believe the bug may be harmless, as the peripheral has
> an extra defence - its BAT or TLB entry will be non-cacheable, so no bus
> cycle will occur. The text of the errata doesn't spell this out as
> clearly as I'd like, but I think all it can do is cause a spurious bus
> cycle to ram.

If this is true, this is not very different from a speculative cache line
fill which turns out not to be necessary, wasteful but not a big deal.

	Gabriel

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Gabriel Paubert]

On Mon, Nov 17, 2003 at 03:37:00PM +0000, Adrian Cox wrote:
> On Mon, 2003-11-17 at 15:12, Gabriel Paubert wrote:
> > On Mon, Nov 17, 2003 at 02:57:53PM +0000, Adrian Cox wrote:
> > > Any opinion on the dcbt issue?  It looks like it could provide a way for
> > > a malicious userspace application to crash the machine, though it needs
> > > a combination of:
> > > 1) good timing
> > > 2) a peripheral that would be confused by an extra read cycle
>
> > Well, only privileged applications should have access to
> > peripherals, no?
> [...]
> > But maybe I miss something.
>
> That's the bug - a dcbt to a protected region can cause a spurious read
> cycle to that address. To trigger it:
>
> 1) the target address is in a BAT or TLB, marked as supervisor access
> only.
> 2) a cache miss to a cache alias of the target address reaches the
> load-store unit
> 2) you issue a dcbt to the target address within 1 clock cycle of step
> 2.
>
> Actually, I now believe the bug may be harmless, as the peripheral has
> an extra defence - its BAT or TLB entry will be non-cacheable, so no bus
> cycle will occur. The text of the errata doesn't spell this out as
> clearly as I'd like, but I think all it can do is cause a spurious bus
> cycle to ram.

Now that I downloaded the errata, it is rather clear that if the area
is cache-inhibited, there won't be any spurious access.

You might have a spurious access to a write-through area, even if
guarded it seems, but if something is marked write through, spurious
reads should have no side effects to start with.

In short, I believe that the erratum is harmless, until somebody
clearly shows that I'm wrong of course :-)

	Gabriel

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Jon Masters]

Gabriel Paubert wrote:

| In short, I believe that the erratum is harmless, until somebody
| clearly shows that I'm wrong of course :-)

There have been similar errata for 4xx chips I believe.

Jon.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: New 745x errata

[Giuliano Pochini]

On 17-Nov-2003 Adrian Cox wrote:
>
> On closer reading, I now think this problem may be harmless - see my
> explanation to Gabriel Paubert for more details.

The errata says: "For protection violation case only [...]". IMHO it
seems to imply that all other cases are not affected. But it's not as
clear as it should be.


--
Giuliano.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/