[net-next PATCH 1/2] ixgbe: fix Si errata - require L0's disable on upstream device

[net-next PATCH 1/2] ixgbe: fix Si errata - require L0's disable on upstream device

[Jeff Kirsher]

From: Don Skidmore <donald.c.skidmore@intel.com>

In order to work around a silicon errata on 82598 we need to disable L0's
in the PCIe switch port to which the 82598 is connected, to prevent the
duplicate completions caused by L0s.

Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
Acked-by: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---

 drivers/net/ixgbe/ixgbe_main.c |   30 +++++++++++++++++++++++++++++-
 1 files changed, 29 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_main.c b/drivers/net/ixgbe/ixgbe_main.c
index 8c32c18..a935949 100644
--- a/drivers/net/ixgbe/ixgbe_main.c
+++ b/drivers/net/ixgbe/ixgbe_main.c
@@ -28,6 +28,7 @@
 #include <linux/types.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/pci-aspm.h>
 #include <linux/netdevice.h>
 #include <linux/vmalloc.h>
 #include <linux/string.h>
@@ -3962,10 +3963,12 @@ static int __devinit ixgbe_probe(struct pci_dev *pdev,
 	struct net_device *netdev;
 	struct ixgbe_adapter *adapter = NULL;
 	struct ixgbe_hw *hw;
+	struct pci_dev *us_dev;
 	const struct ixgbe_info *ii = ixgbe_info_tbl[ent->driver_data];
 	static int cards_found;
-	int i, err, pci_using_dac;
+	int i, err, pci_using_dac, pos;
 	u16 link_status, link_speed, link_width;
+	u16 state = 0;
 	u32 part_num, eec;
 
 	err = pci_enable_device(pdev);
@@ -3994,6 +3997,31 @@ static int __devinit ixgbe_probe(struct pci_dev *pdev,
 		goto err_pci_reg;
 	}
 
+	/*
+	 * Workaround of Silicon errata on 82598.  Disable L0s in the PCIe
+	 * switch port to which the 82598 is connected to prevent duplicate
+	 * completions caused by L0s.  We check the mac type so that we only
+	 * do this on 82598 devices.
+	 */
+	if (ii->mac == ixgbe_mac_82598EB) {
+		us_dev = pdev->bus->self;
+		pos = pci_find_capability(us_dev, PCI_CAP_ID_EXP);
+		if (pos) {
+			pci_read_config_word(us_dev,
+			                     pos + PCI_EXP_LNKCTL,
+			                     &state);
+			state &= ~PCIE_LINK_STATE_L0S;
+			pci_write_config_word(us_dev,
+			                      pos + PCI_EXP_LNKCTL,
+			                      state);
+			printk(KERN_INFO "ixgbe: Disabling ASPM L0s between "
+			       "%x:%x.%x and %x:%x.%x\n",
+			       us_dev->bus->number, PCI_SLOT(us_dev->devfn),
+			       PCI_FUNC(us_dev->devfn), pdev->bus->number,
+			       PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn));
+		}
+	}
+
 	err = pci_enable_pcie_error_reporting(pdev);
 	if (err) {
 		dev_err(&pdev->dev, "pci_enable_pcie_error_reporting failed "

[net-next PATCH 2/2] ixgbe: fix for 82598 Si errata causing buffer overflow

[Jeff Kirsher]

From: Don Skidmore <donald.c.skidmore@intel.com>

The failure happens when an interrupt occurs and the driver is reading
EICR.  This read will cause a clear-by-read which leads to two TLP
being inserted in the PCIe retry buffer leading to an overflow of the
buffer and corruption of TLPs.

The solution is different depending where the reading of EICR takes place.

For ixgbe_msix_lsc() since we are in MSIX mode and know OCD is enabled a
clear-by-write is done instead of the normal clear-by-read.

For ixgbe_intr() 0xffffffff is written to EIMC before the read, masking the
interrupts.

Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
Acked-by: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---

 drivers/net/ixgbe/ixgbe_main.c |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_main.c b/drivers/net/ixgbe/ixgbe_main.c
index a935949..8fbd4c8 100644
--- a/drivers/net/ixgbe/ixgbe_main.c
+++ b/drivers/net/ixgbe/ixgbe_main.c
@@ -937,7 +937,16 @@ static irqreturn_t ixgbe_msix_lsc(int irq, void *data)
 	struct net_device *netdev = data;
 	struct ixgbe_adapter *adapter = netdev_priv(netdev);
 	struct ixgbe_hw *hw = &adapter->hw;
-	u32 eicr = IXGBE_READ_REG(hw, IXGBE_EICR);
+	u32 eicr;
+
+	/*
+	 * Workaround for Silicon errata.  Use clear-by-write instead
+	 * of clear-by-read.  Reading with EICS will return the
+	 * interrupt causes without clearing, which later be done
+	 * with the write to EICR.
+	 */
+	eicr = IXGBE_READ_REG(hw, IXGBE_EICS);
+	IXGBE_WRITE_REG(hw, IXGBE_EICR, eicr);
 
 	if (eicr & IXGBE_EICR_LSC)
 		ixgbe_check_lsc(adapter);
@@ -1356,6 +1365,12 @@ static irqreturn_t ixgbe_intr(int irq, void *data)
 	struct ixgbe_hw *hw = &adapter->hw;
 	u32 eicr;
 
+	/*
+	 * Workaround for silicon errata.  Mask the interrupts
+	 * before the read of EICR.
+	 */
+	IXGBE_WRITE_REG(hw, IXGBE_EIMC, IXGBE_IRQ_CLEAR_MASK);
+
 	/* for NAPI, using EIAM to auto-mask tx/rx interrupt bits on read
 	 * therefore no explict interrupt disable is necessary */
 	eicr = IXGBE_READ_REG(hw, IXGBE_EICR);

Re: [net-next PATCH 2/2] ixgbe: fix for 82598 Si errata causing buffer overflow

[David Miller]

From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Date: Sat, 21 Feb 2009 14:23:40 -0800

> The failure happens when an interrupt occurs and the driver is reading
> EICR.  This read will cause a clear-by-read which leads to two TLP
> being inserted in the PCIe retry buffer leading to an overflow of the
> buffer and corruption of TLPs.
> 
> The solution is different depending where the reading of EICR takes place.
> 
> For ixgbe_msix_lsc() since we are in MSIX mode and know OCD is enabled a
> clear-by-write is done instead of the normal clear-by-read.
> 
> For ixgbe_intr() 0xffffffff is written to EIMC before the read, masking the
> interrupts.
> 
> Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
> Acked-by: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>

Applied, thanks.

Re: [net-next PATCH 1/2] ixgbe: fix Si errata - require L0's disable on upstream device

[David Miller]

From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Date: Sat, 21 Feb 2009 14:23:19 -0800

> In order to work around a silicon errata on 82598 we need to disable L0's
> in the PCIe switch port to which the 82598 is connected, to prevent the
> duplicate completions caused by L0s.
> 
> Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
> Acked-by: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>

You can't implement this at this level guys.

PCI-E port hacks inside of individual child drivers will only
cause new bugs where these settings interfere with whatever
the PCI-E port driver wants to do.

Solve this either generically as a quirk in the PCI layer or
as some kind of PCI-E port driver attribute that can be
acted upon.