Fix sysfs pci bus rescan on PowerNV (and other things)

Fix sysfs pci bus rescan on PowerNV (and other things)

[Oliver O'Halloran]

This series is based on top of my previously posted series which reworks
how devices are added to their IOMMU groups. The two series are largely
orthogonal to each other, but they both touch pnv_pci_ioda_dma_dev_setup()
so there's a minor merge conflict if they aren't applied together. I can
fix that if people think it's important, but applying them together is
probably easisest for everyone.

Base series: https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=168715

With that out of the way, what the bulk of the changes in here are in 2/4
which moves the point where we do the HW configuration to allow a bus to be
used. Currently it's done when we setup the parent bridge for that bus and
we're moving it to be done when we add the first device to that bus.

For an example of why this change is necssary this is what happens on the
current linux-next master. This has one extra patch applied to print an
error when pci_enable_device() is blocked by the platform since it helps
highlight the issue:

/sys/devices/pci0022:00/0022:00:00.0 # echo 1 > 0022\:01\:00.0/remove
e1000e 0022:01:00.0 enP34p1s0: removed PHC
e1000e 0022:01:00.0 enP34p1s0: NIC Link is Down
pci 0022:01:00.0: Removing from iommu group 11

At this point the bus 0022:01 is empty.

/sys/devices/pci0022:00/0022:00:00.0 # echo 1 > rescan
pci 0022:01:00.0: [8086:10d3] type 00 class 0x020000
pci 0022:01:00.0: reg 0x10: [mem 0x3fe9000c0000-0x3fe9000dffff]
pci 0022:01:00.0: reg 0x14: [mem 0x3fe900000000-0x3fe90007ffff]
pci 0022:01:00.0: reg 0x18: [io  0x0000-0x001f]
pci 0022:01:00.0: reg 0x1c: [mem 0x3fe9000e0000-0x3fe9000e3fff]
pci 0022:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref]
pci 0022:01:00.0: BAR3 [mem size 0x00004000]: requesting alignment to 0x10000
pci 0022:01:00.0: PME# supported from D0 D3hot D3cold
pci 0022:00:00.0: BAR 13: no space for [io  size 0x1000]
pci 0022:00:00.0: BAR 13: failed to assign [io  size 0x1000]
pci 0022:01:00.0: BAR 1: assigned [mem 0x3fe900000000-0x3fe90007ffff]
pci 0022:01:00.0: BAR 6: assigned [mem 0x3fe900080000-0x3fe9000bffff pref]
pci 0022:01:00.0: BAR 0: assigned [mem 0x3fe9000c0000-0x3fe9000dffff]
pci 0022:01:00.0: BAR 3: assigned [mem 0x3fe9000e0000-0x3fe9000e3fff]
pci 0022:01:00.0: BAR 2: no space for [io  size 0x0020]
pci 0022:01:00.0: BAR 2: failed to assign [io  size 0x0020]
e1000e 0022:01:00.0: pci_enable_device() blocked, no PE assigned.
e1000e: probe of 0022:01:00.0 failed with error -22

So on rescan we can re-discover the device, but the driver probe will
always fail at the point where the driver attemps to enable the device
because the PE was deconfigured.

Repeating this same experiment with this series (and dependency) applied:

/sys/devices/pci0022:00/0022:00:00.0 # echo 1 > rescan
pci 0022:01:00.0: [8086:10d3] type 00 class 0x020000
pci 0022:01:00.0: reg 0x10: [mem 0x3fe9000c0000-0x3fe9000dffff]
pci 0022:01:00.0: reg 0x14: [mem 0x3fe900000000-0x3fe90007ffff]
pci 0022:01:00.0: reg 0x18: [io  0x0000-0x001f]
pci 0022:01:00.0: reg 0x1c: [mem 0x3fe9000e0000-0x3fe9000e3fff]
pci 0022:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref]
pci 0022:01:00.0: BAR3 [mem size 0x00004000]: requesting alignment to 0x10000
pci 0022:01:00.0: PME# supported from D0 D3hot D3cold
pci 0022:00:00.0: BAR 13: no space for [io  size 0x1000]
pci 0022:00:00.0: BAR 13: failed to assign [io  size 0x1000]
pci 0022:01:00.0: BAR 1: assigned [mem 0x3fe900000000-0x3fe90007ffff]
pci 0022:01:00.0: BAR 6: assigned [mem 0x3fe900080000-0x3fe9000bffff pref]
pci 0022:01:00.0: BAR 0: assigned [mem 0x3fe9000c0000-0x3fe9000dffff]
pci 0022:01:00.0: BAR 3: assigned [mem 0x3fe9000e0000-0x3fe9000e3fff]
pci 0022:01:00.0: BAR 2: no space for [io  size 0x0020]
pci 0022:01:00.0: BAR 2: failed to assign [io  size 0x0020]
pci_bus 0022:01: Configuring PE for bus
pci 0022:01     : [PE# fd] Secondary bus 0x0000000000000001 associated with PE#fd
pci 0022:01     : [PE# fd] Setting up 32-bit TCE table at 0..80000000
pci 0022:01     : [PE# fd] Setting up window#0 0..7fffffffff pg=10000
pci 0022:01     : [PE# fd] Enabling 64-bit DMA bypass
pci 0022:01:00.0: Configured PE#fd
pci 0022:01:00.0: Adding to iommu group 12
e1000e 0022:01:00.0: enabling device (0140 -> 0142)
e1000e 0022:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
e1000e 0022:01:00.0 0022:01:00.0 (uninitialized): registered PHC clock
e1000e 0022:01:00.0 eth0: (PCI Express:2.5GT/s:Width x1) 68:05:ca:37:9c:d7
e1000e 0022:01:00.0 enP34p1s0: renamed from eth0
e1000e 0022:01:00.0 enP34p1s0: Intel(R) PRO/1000 Network Connection
e1000e 0022:01:00.0 enP34p1s0: MAC: 3, PHY: 8, PBA No: E46981-008
/sys/devices/pci0022:00/0022:00:00.0 #

Now, when the rescan happens we notice the PE was deconfigured after removing
the device and re-configure it. This allows the device to be enabled and
everything works. Probably.

Making this change also lays the groundwork for allowing devices to be
added to a bus PE as they're enabled rather than mapping all 256 devfns
on a bus to the PE in one go. This is going to be necessary for supporting
the native PCIe hotplug driver (rather than pnv_php) since currently
scanning an empty slot causes spurious PE freezes. Keeping inactive
devices mapped to the reserved PE would prevent that from occuring.

It might also be useful for (ab)using PEs to provide per-device
IOMMU contexts rather than per-bus. A per-device context would also be
necessary for allowing individual functions of a device to be passed
through to guests rather than requiring all of them to be passed as a
group.

Oliver

[PATCH 1/4] powerpc/powernv/pci: Add helper to find ioda_pe from BDFN

[Oliver O'Halloran]

For each PHB we maintain a reverse-map that can be used to find the
PE that a BDFN is currently mapped to. Add a helper for doing this
lookup so we can check if a PE has been configured without looking
at pdn->pe_number.

Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
---
 arch/powerpc/platforms/powernv/pci-ioda.c | 10 ++++++++++
 arch/powerpc/platforms/powernv/pci.h      |  1 +
 2 files changed, 11 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 8ae8836..934cbee 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -660,6 +660,16 @@ static int pnv_ioda_get_pe_state(struct pnv_phb *phb, int pe_no)
 	return state;
 }
 
+struct pnv_ioda_pe *pnv_pci_bdfn_to_pe(struct pnv_phb *phb, u16 bdfn)
+{
+	int pe_number = phb->ioda.pe_rmap[bdfn];
+
+	if (pe_number == IODA_INVALID_PE)
+		return NULL;
+
+	return &phb->ioda.pe_array[pe_number];
+}
+
 struct pnv_ioda_pe *pnv_ioda_get_pe(struct pci_dev *dev)
 {
 	struct pci_controller *hose = pci_bus_to_host(dev->bus);
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index fc05f9b..83d40a0 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -208,6 +208,7 @@ extern int pnv_eeh_phb_reset(struct pci_controller *hose, int option);
 
 extern int pnv_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type);
 extern void pnv_teardown_msi_irqs(struct pci_dev *pdev);
+extern struct pnv_ioda_pe *pnv_pci_bdfn_to_pe(struct pnv_phb *phb, u16 bdfn);
 extern struct pnv_ioda_pe *pnv_ioda_get_pe(struct pci_dev *dev);
 extern void pnv_set_msi_irq_chip(struct pnv_phb *phb, unsigned int virq);
 extern unsigned long pnv_pci_ioda2_get_table_size(__u32 page_shift,
-- 
2.9.5

[PATCH 2/4] powerpc/powernv/pci: Re-work bus PE configuration

[Oliver O'Halloran]

For normal PHBs IODA PEs are handled on a per-bus basis so all the devices
on that bus will share a PE. Which PE specificly is determined by the location
of the MMIO BARs for the devices on the bus so we can't actually configure the
bus PEs until after MMIO resources are allocated. As a result PEs are currently
configured by pcibios_setup_bridge(), which is called just before the bridge
windows are programmed into the bus' parent bridge. Configuring the bus PE here
causes a few problems:

1. The root bus doesn't have a parent bridge so setting up the PE for the root
   bus requires some hacks.

2. The PELT-V isn't setup correctly because pnv_ioda_set_peltv() assumes that
   PEs will be configured in root-to-leaf order. This assumption is broken
   because resource assignment is performed depth-first so the leaf bridges
   are setup before their parents are. The hack mentioned in 1) results in
   the "correct" PELT-V for busses immediately below the root port, but not
   for devices below a switch.

3. It's possible to break the sysfs PCI rescan feature by removing all
   the devices on a bus. When the last device is removed from a PE its
   will be de-configured. Rescanning the devices on a bus does not cause
   the bridge to be reconfigured rendering the devices on that bus
   unusable.

We can address most of these problems by moving the PE setup out of
pcibios_setup_bridge() and into pcibios_bus_add_device(). This fixes 1)
and 2) because pcibios_bus_add_device() is called on each device in
root-to-leaf order so PEs for parent buses will always be configured
before their children. It also fixes 3) by ensuring the PE is
configured before initialising DMA for the device. In the event the PE
was de-configured due to removing all the devices in that PE it will
now be reconfigured when a new device is added since there's no
dependecy on the bridge_setup() hook being called.

Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
---
 arch/powerpc/platforms/powernv/pci-ioda.c | 81 ++++++++++++-------------------
 1 file changed, 30 insertions(+), 51 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 934cbee..2ba730c 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -51,6 +51,7 @@ static const char * const pnv_phb_names[] = { "IODA1", "IODA2", "NPU_NVLINK",
 					      "NPU_OCAPI" };
 
 static void pnv_pci_ioda2_set_bypass(struct pnv_ioda_pe *pe, bool enable);
+static void pnv_pci_configure_bus(struct pci_bus *bus);
 
 void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
 			    const char *fmt, ...)
@@ -1120,34 +1121,6 @@ static struct pnv_ioda_pe *pnv_ioda_setup_dev_PE(struct pci_dev *dev)
 	return pe;
 }
 
-static void pnv_ioda_setup_same_PE(struct pci_bus *bus, struct pnv_ioda_pe *pe)
-{
-	struct pci_dev *dev;
-
-	list_for_each_entry(dev, &bus->devices, bus_list) {
-		struct pci_dn *pdn = pci_get_pdn(dev);
-
-		if (pdn == NULL) {
-			pr_warn("%s: No device node associated with device !\n",
-				pci_name(dev));
-			continue;
-		}
-
-		/*
-		 * In partial hotplug case, the PCI device might be still
-		 * associated with the PE and needn't attach it to the PE
-		 * again.
-		 */
-		if (pdn->pe_number != IODA_INVALID_PE)
-			continue;
-
-		pe->device_count++;
-		pdn->pe_number = pe->pe_number;
-		if ((pe->flags & PNV_IODA_PE_BUS_ALL) && dev->subordinate)
-			pnv_ioda_setup_same_PE(dev->subordinate, pe);
-	}
-}
-
 /*
  * There're 2 types of PCI bus sensitive PEs: One that is compromised of
  * single PCI bus. Another one that contains the primary PCI bus and its
@@ -1168,7 +1141,6 @@ static struct pnv_ioda_pe *pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)
 	pe_num = phb->ioda.pe_rmap[bus->number << 8];
 	if (pe_num != IODA_INVALID_PE) {
 		pe = &phb->ioda.pe_array[pe_num];
-		pnv_ioda_setup_same_PE(bus, pe);
 		return NULL;
 	}
 
@@ -1212,9 +1184,6 @@ static struct pnv_ioda_pe *pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)
 		return NULL;
 	}
 
-	/* Associate it with all child devices */
-	pnv_ioda_setup_same_PE(bus, pe);
-
 	/* Put PE to the list */
 	list_add_tail(&pe->list, &phb->ioda.pe_list);
 
@@ -1772,15 +1741,32 @@ static void pnv_pci_ioda_dma_dev_setup(struct pci_dev *pdev)
 	struct pci_dn *pdn = pci_get_pdn(pdev);
 	struct pnv_ioda_pe *pe;
 
-	/*
-	 * The function can be called while the PE#
-	 * hasn't been assigned. Do nothing for the
-	 * case.
-	 */
-	if (!pdn || pdn->pe_number == IODA_INVALID_PE)
-		return;
+	/* Check if the BDFN for this device is associated with a PE yet */
+	pe = pnv_pci_bdfn_to_pe(phb, pdev->devfn | (pdev->bus->number << 8));
+	if (!pe) {
+		/* VF PEs should be pre-configured in pnv_pci_sriov_enable() */
+		if (WARN_ON(pdev->is_virtfn))
+			return;
+
+		pnv_pci_configure_bus(pdev->bus);
+		pe = pnv_pci_bdfn_to_pe(phb, pdev->devfn | (pdev->bus->number << 8));
+		pci_info(pdev, "Configured PE#%x\n", pe ? pe->pe_number : 0xfffff);
+
+
+		/*
+		 * If we can't setup the IODA PE something has gone horribly
+		 * wrong and we can't enable DMA for the device.
+		 */
+		if (WARN_ON(!pe))
+			return;
+	} else {
+		pci_info(pdev, "Added to existing PE#%x\n", pe->pe_number);
+	}
+
+	if (pdn)
+		pdn->pe_number = pe->pe_number;
+	pe->device_count++;
 
-	pe = &phb->ioda.pe_array[pdn->pe_number];
 	WARN_ON(get_dma_ops(&pdev->dev) != &dma_iommu_ops);
 	pdev->dev.archdata.dma_offset = pe->tce_bypass_base;
 	set_iommu_table_base(&pdev->dev, pe->table_group.tables[0]);
@@ -2300,9 +2286,6 @@ static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,
 	pe->table_group.tce32_size = tbl->it_size << tbl->it_page_shift;
 	iommu_init_table(tbl, phb->hose->node, 0, 0);
 
-	if (pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL))
-		pnv_ioda_setup_bus_dma(pe, pe->pbus);
-
 	return;
  fail:
 	/* XXX Failure: Try to fallback to 64-bit only ? */
@@ -2633,9 +2616,6 @@ static void pnv_pci_ioda2_setup_dma_pe(struct pnv_phb *phb,
 	iommu_register_group(&pe->table_group, phb->hose->global_number,
 			     pe->pe_number);
 #endif
-
-	if (pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL))
-		pnv_ioda_setup_bus_dma(pe, pe->pbus);
 }
 
 int64_t pnv_opal_pci_msi_eoi(struct irq_chip *chip, unsigned int hw_irq)
@@ -3234,16 +3214,15 @@ static void pnv_pci_fixup_bridge_resources(struct pci_bus *bus,
 	}
 }
 
-static void pnv_pci_setup_bridge(struct pci_bus *bus, unsigned long type)
+static void pnv_pci_configure_bus(struct pci_bus *bus)
 {
 	struct pci_controller *hose = pci_bus_to_host(bus);
 	struct pnv_phb *phb = hose->private_data;
 	struct pci_dev *bridge = bus->self;
 	struct pnv_ioda_pe *pe;
-	bool all = (pci_pcie_type(bridge) == PCI_EXP_TYPE_PCI_BRIDGE);
+	bool all = (bridge && pci_pcie_type(bridge) == PCI_EXP_TYPE_PCI_BRIDGE);
 
-	/* Extend bridge's windows if necessary */
-	pnv_pci_fixup_bridge_resources(bus, type);
+	dev_info(&bus->dev, "Configuring PE for bus\n");
 
 	/* The PE for root bus should be realized before any one else */
 	if (!phb->ioda.root_pe_populated) {
@@ -3620,7 +3599,7 @@ static const struct pci_controller_ops pnv_pci_ioda_controller_ops = {
 	.enable_device_hook	= pnv_pci_enable_device_hook,
 	.release_device		= pnv_pci_release_device,
 	.window_alignment	= pnv_pci_window_alignment,
-	.setup_bridge		= pnv_pci_setup_bridge,
+	.setup_bridge		= pnv_pci_fixup_bridge_resources,
 	.reset_secondary_bus	= pnv_pci_reset_secondary_bus,
 	.shutdown		= pnv_pci_ioda_shutdown,
 };
-- 
2.9.5

[PATCH 3/4] powerpc/powernv/pci: Reserve the root bus PE during init

[Oliver O'Halloran]

Doing it once during boot rather than doing it on the fly and drop the janky
populated logic.

Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
---
 arch/powerpc/platforms/powernv/pci-ioda.c | 26 +++++++++-----------------
 arch/powerpc/platforms/powernv/pci.h      |  1 -
 2 files changed, 9 insertions(+), 18 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 2ba730c..05436a9 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1145,8 +1145,7 @@ static struct pnv_ioda_pe *pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)
 	}
 
 	/* PE number for root bus should have been reserved */
-	if (pci_is_root_bus(bus) &&
-	    phb->ioda.root_pe_idx != IODA_INVALID_PE)
+	if (pci_is_root_bus(bus))
 		pe = &phb->ioda.pe_array[phb->ioda.root_pe_idx];
 
 	/* Check if PE is determined by M64 */
@@ -3224,15 +3223,6 @@ static void pnv_pci_configure_bus(struct pci_bus *bus)
 
 	dev_info(&bus->dev, "Configuring PE for bus\n");
 
-	/* The PE for root bus should be realized before any one else */
-	if (!phb->ioda.root_pe_populated) {
-		pe = pnv_ioda_setup_bus_PE(phb->hose->bus, false);
-		if (pe) {
-			phb->ioda.root_pe_idx = pe->pe_number;
-			phb->ioda.root_pe_populated = true;
-		}
-	}
-
 	/* Don't assign PE to PCI bus, which doesn't have subordinate devices */
 	if (list_empty(&bus->devices))
 		return;
@@ -3517,11 +3507,10 @@ static void pnv_ioda_release_pe(struct pnv_ioda_pe *pe)
 	 * that it can be populated again in PCI hot add path. The PE
 	 * shouldn't be destroyed as it's the global reserved resource.
 	 */
-	if (phb->ioda.root_pe_populated &&
-	    phb->ioda.root_pe_idx == pe->pe_number)
-		phb->ioda.root_pe_populated = false;
-	else
-		pnv_ioda_free_pe(pe);
+	if (phb->ioda.root_pe_idx == pe->pe_number)
+		return;
+
+	pnv_ioda_free_pe(pe);
 }
 
 static void pnv_pci_release_device(struct pci_dev *pdev)
@@ -3629,6 +3618,7 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,
 	struct pnv_phb *phb;
 	unsigned long size, m64map_off, m32map_off, pemap_off;
 	unsigned long iomap_off = 0, dma32map_off = 0;
+	struct pnv_ioda_pe *root_pe;
 	struct resource r;
 	const __be64 *prop64;
 	const __be32 *prop32;
@@ -3796,7 +3786,9 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np,
 		phb->ioda.root_pe_idx = phb->ioda.reserved_pe_idx - 1;
 		pnv_ioda_reserve_pe(phb, phb->ioda.root_pe_idx);
 	} else {
-		phb->ioda.root_pe_idx = IODA_INVALID_PE;
+		/* otherwise just allocate one */
+		root_pe = pnv_ioda_alloc_pe(phb);
+		phb->ioda.root_pe_idx = root_pe->pe_number;
 	}
 
 	INIT_LIST_HEAD(&phb->ioda.pe_list);
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index 83d40a0..51c254f2 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -136,7 +136,6 @@ struct pnv_phb {
 		unsigned int		total_pe_num;
 		unsigned int		reserved_pe_idx;
 		unsigned int		root_pe_idx;
-		bool			root_pe_populated;
 
 		/* 32-bit MMIO window */
 		unsigned int		m32_size;
-- 
2.9.5

[PATCH 4/4] powerpc/powernv/pci: Sprinkle around some WARN_ON()s

[Oliver O'Halloran]

pnv_pci_ioda_configure_bus() should now only ever be called when a device is
added to the bus so add a WARN_ON() to the empty bus check. Similarly,
pnv_pci_ioda_setup_bus_PE() should only ever be called for an unconfigured PE,
so add a WARN_ON() for that case too.

Signed-off-by: Oliver O'Halloran <oohall@gmail.com>
---
 arch/powerpc/platforms/powernv/pci-ioda.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 05436a9..627420c 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1139,7 +1139,7 @@ static struct pnv_ioda_pe *pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)
 	 * We should reuse it instead of allocating a new one.
 	 */
 	pe_num = phb->ioda.pe_rmap[bus->number << 8];
-	if (pe_num != IODA_INVALID_PE) {
+	if (WARN_ON(pe_num != IODA_INVALID_PE)) {
 		pe = &phb->ioda.pe_array[pe_num];
 		return NULL;
 	}
@@ -3224,7 +3224,7 @@ static void pnv_pci_configure_bus(struct pci_bus *bus)
 	dev_info(&bus->dev, "Configuring PE for bus\n");
 
 	/* Don't assign PE to PCI bus, which doesn't have subordinate devices */
-	if (list_empty(&bus->devices))
+	if (WARN_ON(list_empty(&bus->devices)))
 		return;
 
 	/* Reserve PEs according to used M64 resources */
-- 
2.9.5

Re: [PATCH 1/4] powerpc/powernv/pci: Add helper to find ioda_pe from BDFN

[Michael Ellerman]

On Fri, 17 Apr 2020 17:35:05 +1000, Oliver O'Halloran wrote:
> For each PHB we maintain a reverse-map that can be used to find the
> PE that a BDFN is currently mapped to. Add a helper for doing this
> lookup so we can check if a PE has been configured without looking
> at pdn->pe_number.

Applied to powerpc/next.

[1/4] powerpc/powernv/pci: Add helper to find ioda_pe from BDFN
      https://git.kernel.org/powerpc/c/a8d7d5fc2e1672924a391aa37ef8c02d1ec84a4e
[2/4] powerpc/powernv/pci: Re-work bus PE configuration
      https://git.kernel.org/powerpc/c/dc3d8f85bb571c3640ebba24b82a527cf2cb3f24
[3/4] powerpc/powernv/pci: Reserve the root bus PE during init
      https://git.kernel.org/powerpc/c/718d249aeadff058f79c2e6b25212dd45bd711ae
[4/4] powerpc/powernv/pci: Sprinkle around some WARN_ON()s
      https://git.kernel.org/powerpc/c/6ae8aedf8fa932541f48a85219d75ca041c22080

cheers