[QUESTION] Enable coherency for all pages on 83xx to fix PCI data corruption

[QUESTION] Enable coherency for all pages on 83xx to fix PCI data corruption

[Liu Dave-r63238]

All,

I want to know which PCI errata is solved by this patch
and if this patch did test on real hardware.

I know this patch turn on the 'M' bit -memory coherency.
But I don't believe this can solved the "PCI read multi-line"
errata.

-DAve

http://ozlabs.org/pipermail/linuxppc-dev/2006-February/021267.html

On the 83xx platform to ensure the PCI inbound memory is handled
properly we
have to turn on coherency for all pages in the MMU.  Otherwise we see
corruption if inbound "prefetching/streaming" is enabled on the PCI
controller.

Signed-off-by: Randy Vinson <rvinson at mvista.com>
Signed-off-by: Kumar Gala <galak at kernel.crashing.org>

---

(For 2.6.16 if we can get it in)

commit 4b2f4b1585f15d1c30cd2eda6d5f9a2ca7dcf998
tree 7aebf508d10127831cf92fb7ce919230924ad85d
parent 7cfb7344aae902edfd5d51dd5f734cbf2585649c
author Kumar Gala <galak at kernel.crashing.org> Wed, 22 Feb 2006
09:53:34 -0600
committer Kumar Gala <galak at kernel.crashing.org> Wed, 22 Feb 2006
09:53:34 -0600

 include/asm-powerpc/cputable.h |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/include/asm-powerpc/cputable.h
b/include/asm-powerpc/cputable.h
index 6421054..90d005b 100644
--- a/include/asm-powerpc/cputable.h
+++ b/include/asm-powerpc/cputable.h
@@ -159,9 +159,11 @@ extern void do_cpu_ftr_fixups(unsigned l
 #endif
=20
 /* We need to mark all pages as being coherent if we're SMP or we
- * have a 74[45]x and an MPC107 host bridge.
+ * have a 74[45]x and an MPC107 host bridge. Also 83xx requires
+ * it for PCI "streaming/prefetch" to work properly.
  */
-#if defined(CONFIG_SMP) || defined(CONFIG_MPC10X_BRIDGE)
+#if defined(CONFIG_SMP) || defined(CONFIG_MPC10X_BRIDGE) \
+	|| defined(CONFIG_PPC_83xx)
 #define CPU_FTR_COMMON                  CPU_FTR_NEED_COHERENT
 #else
 #define CPU_FTR_COMMON                  0
@@ -277,7 +279,8 @@ enum {
 	CPU_FTRS_G2_LE =3D CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_MAYBE_CAN_DOZE
|
 	    CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP |
CPU_FTR_HAS_HIGH_BATS,
 	CPU_FTRS_E300 =3D CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_MAYBE_CAN_DOZE
|
-	    CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP |
CPU_FTR_HAS_HIGH_BATS,
+	    CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP |
CPU_FTR_HAS_HIGH_BATS |
+	    CPU_FTR_COMMON,
 	CPU_FTRS_CLASSIC32 =3D CPU_FTR_COMMON | CPU_FTR_SPLIT_ID_CACHE |
 	    CPU_FTR_USE_TB | CPU_FTR_HPTE_TABLE,
 	CPU_FTRS_POWER3_32 =3D CPU_FTR_COMMON | CPU_FTR_SPLIT_ID_CACHE |

Re: [QUESTION] Enable coherency for all pages on 83xx to fix PCI data corruption

[Kumar Gala]

This was to address PCI5 if I remember correctly.

- kumar

On Aug 28, 2006, at 2:49 AM, Liu Dave-r63238 wrote:

> All,
>
> I want to know which PCI errata is solved by this patch
> and if this patch did test on real hardware.
>
> I know this patch turn on the 'M' bit -memory coherency.
> But I don't believe this can solved the "PCI read multi-line"
> errata.
>
> -DAve
>
> http://ozlabs.org/pipermail/linuxppc-dev/2006-February/021267.html
>
> On the 83xx platform to ensure the PCI inbound memory is handled
> properly we
> have to turn on coherency for all pages in the MMU.  Otherwise we see
> corruption if inbound "prefetching/streaming" is enabled on the PCI
> controller.
>
> Signed-off-by: Randy Vinson <rvinson at mvista.com>
> Signed-off-by: Kumar Gala <galak at kernel.crashing.org>
>
> ---
>
> (For 2.6.16 if we can get it in)
>
> commit 4b2f4b1585f15d1c30cd2eda6d5f9a2ca7dcf998
> tree 7aebf508d10127831cf92fb7ce919230924ad85d
> parent 7cfb7344aae902edfd5d51dd5f734cbf2585649c
> author Kumar Gala <galak at kernel.crashing.org> Wed, 22 Feb 2006
> 09:53:34 -0600
> committer Kumar Gala <galak at kernel.crashing.org> Wed, 22 Feb 2006
> 09:53:34 -0600
>
>  include/asm-powerpc/cputable.h |    9 ++++++---
>  1 files changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/include/asm-powerpc/cputable.h
> b/include/asm-powerpc/cputable.h
> index 6421054..90d005b 100644
> --- a/include/asm-powerpc/cputable.h
> +++ b/include/asm-powerpc/cputable.h
> @@ -159,9 +159,11 @@ extern void do_cpu_ftr_fixups(unsigned l
>  #endif
>
>  /* We need to mark all pages as being coherent if we're SMP or we
> - * have a 74[45]x and an MPC107 host bridge.
> + * have a 74[45]x and an MPC107 host bridge. Also 83xx requires
> + * it for PCI "streaming/prefetch" to work properly.
>   */
> -#if defined(CONFIG_SMP) || defined(CONFIG_MPC10X_BRIDGE)
> +#if defined(CONFIG_SMP) || defined(CONFIG_MPC10X_BRIDGE) \
> +	|| defined(CONFIG_PPC_83xx)
>  #define CPU_FTR_COMMON                  CPU_FTR_NEED_COHERENT
>  #else
>  #define CPU_FTR_COMMON                  0
> @@ -277,7 +279,8 @@ enum {
>  	CPU_FTRS_G2_LE = CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_MAYBE_CAN_DOZE
> |
>  	    CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP |
> CPU_FTR_HAS_HIGH_BATS,
>  	CPU_FTRS_E300 = CPU_FTR_SPLIT_ID_CACHE | CPU_FTR_MAYBE_CAN_DOZE
> |
> -	    CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP |
> CPU_FTR_HAS_HIGH_BATS,
> +	    CPU_FTR_USE_TB | CPU_FTR_MAYBE_CAN_NAP |
> CPU_FTR_HAS_HIGH_BATS |
> +	    CPU_FTR_COMMON,
>  	CPU_FTRS_CLASSIC32 = CPU_FTR_COMMON | CPU_FTR_SPLIT_ID_CACHE |
>  	    CPU_FTR_USE_TB | CPU_FTR_HPTE_TABLE,
>  	CPU_FTRS_POWER3_32 = CPU_FTR_COMMON | CPU_FTR_SPLIT_ID_CACHE |

RE: [QUESTION] Enable coherency for all pages on 83xx to fix PCI data corruption

[Liu Dave-r63238]

> This was to address PCI5 if I remember correctly.
>=20
> - kumar
>=20
> On Aug 28, 2006, at 2:49 AM, Liu Dave-r63238 wrote:
>=20
> > All,
> >
> > I want to know which PCI errata is solved by this patch and if this=20
> > patch did test on real hardware.
> >
> > I know this patch turn on the 'M' bit -memory coherency.
> > But I don't believe this can solved the "PCI read multi-line"
> > errata.
> >
> > -DAve
<snip>

I also don't believe this patch can solve the PCI5 errata, The PCI5
description:

When external PCI devices try to read from the memory which is defined
as
prefetchable and where the transaction is more than one cache line,
there
may be data corruption.

-DAve

Re: [QUESTION] Enable coherency for all pages on 83xx to fix PCI data corruption

[Kumar Gala]

On Aug 28, 2006, at 9:04 PM, Liu Dave-r63238 wrote:

>> This was to address PCI5 if I remember correctly.
>>
>> - kumar
>>
>> On Aug 28, 2006, at 2:49 AM, Liu Dave-r63238 wrote:
>>
>>> All,
>>>
>>> I want to know which PCI errata is solved by this patch and if this
>>> patch did test on real hardware.
>>>
>>> I know this patch turn on the 'M' bit -memory coherency.
>>> But I don't believe this can solved the "PCI read multi-line"
>>> errata.
>>>
>>> -DAve
> <snip>
>
> I also don't believe this patch can solve the PCI5 errata, The PCI5
> description:
>
> When external PCI devices try to read from the memory which is defined
> as
> prefetchable and where the transaction is more than one cache line,
> there
> may be data corruption.


Randy may remember but this fixed an issue related that he was seeing  
with an e100 or e1000 and it was the solution provided by the  
Freescale Apps team in Austin.

- kumar

Re: [QUESTION] Enable coherency for all pages on 83xx to fix PCI data corruption

[Randy Vinson]

Kumar Gala wrote:
> Randy may remember but this fixed an issue related that he was seeing  
> with an e100 or e1000 and it was the solution provided by the  Freescale 
> Apps team in Austin.
Hi All,
  Sorry for the delay in responding, I've been out sick on and off.

  The patch does not solve an erratum. It is required for proper operation if prefetching is enabled for any inbound PCI window.

  The problem is related to the buffers in the I/O Sequencer (Page 11-1 of the MPC8349 Manual Rev 2). There are 12 of these buffers, each the length of a cache line. When memory prefetching is enabled for an inbound PCI window, these buffers can get pre-loaded with data from main memory. If the processor subsequently modifies this data in main memory and cache coherency is not enabled, the contents of the I/O Sequencer's buffers do not get snooped and stale data is delivered to the PCI bus. If memory coherency is enabled, subsequent writes to pre-fetchable memory are snooped by the I/O Sequencer and the stale data problem is avoided.

  The connection to PCI erratum 5 was false. Marking the inbound PCI memory windows non-prefetchable solved the problem because the I/O Sequencers internal buffers were not pre-loaded and could not contain stale data.

  Kumar is correct in that the solution was provided by the folks at Freescale. Similar corruption has been seen when using a MPC107 host bridge with 74[45]x processors as the MPC107 also has an internal set of buffers that can contain stale data if prefetching is enabled on inbound PCI windows. In fact, there is a comment to that effect just above the comment adding the 83xx in the patch.

		Randy V.