[PATCH] powerpc/powernv/pci: Work around races in PCI bridge enabling

[PATCH] powerpc/powernv/pci: Work around races in PCI bridge enabling

[Benjamin Herrenschmidt]

The generic code is race when multiple children of a PCI bridge try to
enable it simultaneously.

This leads to drivers trying to access a device through a not-yet-enabled
bridge, and this EEH errors under various circumstances when using parallel
driver probing.

There is work going on to fix that properly in the PCI core but it will
take some time.

x86 gets away with it because (outside of hotplug), the BIOS enables all
the bridges at boot time.

This patch does the same thing on powernv by enabling all bridges that
have child devices at boot time, thus avoiding subsequent races. It's suitable
for backporting to stable and distros, while the proper PCI fix will probably
be significantly more invasive.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
CC: stable@vger.kernel.org
---

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 70b2e1e0f23c..0975b0aaf210 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -3368,12 +3368,52 @@ static void pnv_pci_ioda_create_dbgfs(void)
 #endif /* CONFIG_DEBUG_FS */
 }
 
+static void pnv_pci_enable_bridge(struct pci_bus *bus)
+{
+	struct pci_dev *dev = bus->self;
+	struct pci_bus *child;
+
+	/* Empty bus ? bail */
+	if (list_empty(&bus->devices))
+		return;
+
+	/*
+	 * If there's a bridge associated with that bus enable it. This works
+	 * around races in the generic code if the enabling is done during
+	 * parallel probing. This can be removed once those races have been
+	 * fixed.
+	 */
+	if (dev) {
+		int rc = pci_enable_device(dev);
+		if (rc)
+			pci_err(dev, "Error enabling bridge (%d)\n", rc);
+		pci_set_master(dev);
+	}
+
+	/* Perform the same to child busses */
+	list_for_each_entry(child, &bus->children, node)
+		pnv_pci_enable_bridge(child);
+}
+
+static void pnv_pci_enable_bridges(void)
+{
+	struct pci_controller *hose;
+	struct pnv_phb *phb;
+	struct pci_bus *bus;
+	struct pci_dev *pdev;
+
+	list_for_each_entry(hose, &hose_list, list_node)
+		pnv_pci_enable_bridge(hose->bus);
+}
+
 static void pnv_pci_ioda_fixup(void)
 {
 	pnv_pci_ioda_setup_PEs();
 	pnv_pci_ioda_setup_iommu_api();
 	pnv_pci_ioda_create_dbgfs();
 
+	pnv_pci_enable_bridges();
+
 #ifdef CONFIG_EEH
 	pnv_eeh_post_init();
 #endif

Re: [PATCH] powerpc/powernv/pci: Work around races in PCI bridge enabling

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2028 bytes --]

Hi Benjamin,

I love your patch! Yet something to improve:

[auto build test ERROR on powerpc/next]
[also build test ERROR on v4.18 next-20180817]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Benjamin-Herrenschmidt/powerpc-powernv-pci-Work-around-races-in-PCI-bridge-enabling/20180817-113453
base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: powerpc-defconfig (attached as .config)
compiler: powerpc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.2.0 make.cross ARCH=powerpc 

All errors (new ones prefixed by >>):

   arch/powerpc/platforms/powernv/pci-ioda.c: In function 'pnv_pci_enable_bridges':
>> arch/powerpc/platforms/powernv/pci-ioda.c:3262:18: error: unused variable 'pdev' [-Werror=unused-variable]
     struct pci_dev *pdev;
                     ^~~~
>> arch/powerpc/platforms/powernv/pci-ioda.c:3261:18: error: unused variable 'bus' [-Werror=unused-variable]
     struct pci_bus *bus;
                     ^~~
>> arch/powerpc/platforms/powernv/pci-ioda.c:3260:18: error: unused variable 'phb' [-Werror=unused-variable]
     struct pnv_phb *phb;
                     ^~~
   cc1: all warnings being treated as errors

vim +/pdev +3262 arch/powerpc/platforms/powernv/pci-ioda.c

  3256	
  3257	static void pnv_pci_enable_bridges(void)
  3258	{
  3259		struct pci_controller *hose;
> 3260		struct pnv_phb *phb;
> 3261		struct pci_bus *bus;
> 3262		struct pci_dev *pdev;
  3263	
  3264		list_for_each_entry(hose, &hose_list, list_node)
  3265			pnv_pci_enable_bridge(hose->bus);
  3266	}
  3267	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 23684 bytes --]