[PATCH] powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata

[PATCH] powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata

[Josh Boyer]

The 440EPx/GRx chips don't support PCI MRM commands.  Drivers determine this
by looking for a zero value in the PCI cache line size register.  However,
some drivers write to this register upon initialization.  This can cause
MRMs to be used on these chips, which may cause deadlocks on PLB4.

The workaround implemented here introduces a new indirect_type flag, called
PPC_INDIRECT_TYPE_BROKEN_MRM.  This is set in the pci_controller structure in
the pci fixup function for 4xx PCI bridges by determining if the bridge is
compatible with 440EPx/GRx.  The flag is checked in the indirect_write_config
function, and forces any writes to the PCI_CACHE_LINE_SIZE register to be
zero, which will disable MRMs for these chips.

A similar workaround has been tested by AMCC on various PCI cards, such as
the Silicon Image ATA card and Intel E1000 GIGE card.  Hangs were seen with
the Silicon Image card, and MRMs were seen on the bus with a PCI analyzer.
With the workaround in place, the card functioned properly and only Memory
Reads were seen on the bus with the analyzer.

Signed-off-by: Josh Boyer <jwboyer@linux.vnet.ibm.com>

---
 arch/powerpc/sysdev/indirect_pci.c |    6 ++++++
 arch/powerpc/sysdev/ppc4xx_pci.c   |    5 +++++
 include/asm-powerpc/pci-bridge.h   |    3 +++
 3 files changed, 14 insertions(+)

--- linux-2.6.orig/arch/powerpc/sysdev/indirect_pci.c
+++ linux-2.6/arch/powerpc/sysdev/indirect_pci.c
@@ -123,6 +123,12 @@ indirect_write_config(struct pci_bus *bu
 			(bus->number == hose->first_busno))
 		val &= 0xffffff00;
 
+	/* Workaround for PCI_28 Errata in 440EPx/GRx */
+	if ((hose->indirect_type & PPC_INDIRECT_TYPE_BROKEN_MRM) &&
+			offset == PCI_CACHE_LINE_SIZE) {
+		val = 0;
+	}
+
 	/*
 	 * Note: the caller has already checked that offset is
 	 * suitably aligned and that len is 1, 2 or 4.
--- linux-2.6.orig/arch/powerpc/sysdev/ppc4xx_pci.c
+++ linux-2.6/arch/powerpc/sysdev/ppc4xx_pci.c
@@ -75,6 +75,11 @@ static void fixup_ppc4xx_pci_bridge(stru
 	    !of_device_is_compatible(hose->dn, "ibm,plb-pci"))
 		return;
 
+	if (of_device_is_compatible(hose->dn, "ibm,plb440epx-pci") ||
+		of_device_is_compatible(hose->dn, "ibm,plb440grx-pci")) {
+		hose->indirect_type |= PPC_INDIRECT_TYPE_BROKEN_MRM;
+	}
+
 	/* Hide the PCI host BARs from the kernel as their content doesn't
 	 * fit well in the resource management
 	 */
--- linux-2.6.orig/include/asm-powerpc/pci-bridge.h
+++ linux-2.6/include/asm-powerpc/pci-bridge.h
@@ -92,12 +92,15 @@ struct pci_controller {
 	 *   anything but the PHB.  Only allow talking to the PHB if this is
 	 *   set.
 	 *  BIG_ENDIAN - cfg_addr is a big endian register
+	 *  BROKEN_MRM - the 440EPx/GRx chips have an errata that causes hangs on
+	 *   the PLB4.  Effectively disable MRM commands by setting this.
 	 */
 #define PPC_INDIRECT_TYPE_SET_CFG_TYPE		0x00000001
 #define PPC_INDIRECT_TYPE_EXT_REG		0x00000002
 #define PPC_INDIRECT_TYPE_SURPRESS_PRIMARY_BUS	0x00000004
 #define PPC_INDIRECT_TYPE_NO_PCIE_LINK		0x00000008
 #define PPC_INDIRECT_TYPE_BIG_ENDIAN		0x00000010
+#define PPC_INDIRECT_TYPE_BROKEN_MRM		0x00000020
 	u32 indirect_type;
 #endif	/* !CONFIG_PPC64 */
 	/* Currently, we limit ourselves to 1 IO range 

Re: [PATCH] powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata

[Stefan Roese]

On Wednesday 11 June 2008, Josh Boyer wrote:
> The 440EPx/GRx chips don't support PCI MRM commands.  Drivers determine
> this by looking for a zero value in the PCI cache line size register. 
> However, some drivers write to this register upon initialization.  This can
> cause MRMs to be used on these chips, which may cause deadlocks on PLB4.
>
> The workaround implemented here introduces a new indirect_type flag, called
> PPC_INDIRECT_TYPE_BROKEN_MRM.  This is set in the pci_controller structure
> in the pci fixup function for 4xx PCI bridges by determining if the bridge
> is compatible with 440EPx/GRx.  The flag is checked in the
> indirect_write_config function, and forces any writes to the
> PCI_CACHE_LINE_SIZE register to be zero, which will disable MRMs for these
> chips.
>
> A similar workaround has been tested by AMCC on various PCI cards, such as
> the Silicon Image ATA card and Intel E1000 GIGE card.  Hangs were seen with
> the Silicon Image card, and MRMs were seen on the bus with a PCI analyzer.
> With the workaround in place, the card functioned properly and only Memory
> Reads were seen on the bus with the analyzer.
>
> Signed-off-by: Josh Boyer <jwboyer@linux.vnet.ibm.com>

Acked-by: Stefan Roese <sr@denx.de>

I manually applied your patch (since it doesn't apply clean as discussed on 
IRC) and tested it on my Sequoia with a modified PCI USB driver changing 
PCI_CACHE_LINE_SIZE.

Best regards,
Stefan

Re: [PATCH] powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata

[Josh Boyer]

On Thu, 12 Jun 2008 16:24:13 +0200
Stefan Roese <sr@denx.de> wrote:

> On Wednesday 11 June 2008, Josh Boyer wrote:
> > The 440EPx/GRx chips don't support PCI MRM commands.  Drivers determine
> > this by looking for a zero value in the PCI cache line size register. 
> > However, some drivers write to this register upon initialization.  This can
> > cause MRMs to be used on these chips, which may cause deadlocks on PLB4.
> >
> > The workaround implemented here introduces a new indirect_type flag, called
> > PPC_INDIRECT_TYPE_BROKEN_MRM.  This is set in the pci_controller structure
> > in the pci fixup function for 4xx PCI bridges by determining if the bridge
> > is compatible with 440EPx/GRx.  The flag is checked in the
> > indirect_write_config function, and forces any writes to the
> > PCI_CACHE_LINE_SIZE register to be zero, which will disable MRMs for these
> > chips.
> >
> > A similar workaround has been tested by AMCC on various PCI cards, such as
> > the Silicon Image ATA card and Intel E1000 GIGE card.  Hangs were seen with
> > the Silicon Image card, and MRMs were seen on the bus with a PCI analyzer.
> > With the workaround in place, the card functioned properly and only Memory
> > Reads were seen on the bus with the analyzer.
> >
> > Signed-off-by: Josh Boyer <jwboyer@linux.vnet.ibm.com>
> 
> Acked-by: Stefan Roese <sr@denx.de>
> 
> I manually applied your patch (since it doesn't apply clean as discussed on 
> IRC) and tested it on my Sequoia with a modified PCI USB driver changing 
> PCI_CACHE_LINE_SIZE.

Thanks.  I blame git for being dumb.  I guess I'll have to switch to
using git-format-patch instead of quilt.

In the meantime, I'll queue this up for .27.

josh