[PATCH 1/3] pci/hotplug/pnv-php: Remove WARN_ON() in pnv_php_put_slot()

[PATCH 1/3] pci/hotplug/pnv-php: Remove WARN_ON() in pnv_php_put_slot()

[Gavin Shan]

The WARN_ON() causes unnecessary backtrace when putting the parent
slot, which is likely to be NULL.

 WARNING: CPU: 2 PID: 1071 at drivers/pci/hotplug/pnv_php.c:85 \
                              pnv_php_release+0xcc/0x150 [pnv_php]
    :
 Call Trace:
 [c0000003bc007c10] [d00000000ad613c4] pnv_php_release+0x144/0x150 [pnv_php]
 [c0000003bc007c40] [c0000000006641d8] pci_hp_deregister+0x238/0x330
 [c0000003bc007cd0] [d00000000ad61440] pnv_php_unregister_one+0x70/0xa0 [pnv_php]
 [c0000003bc007d10] [d00000000ad614c0] pnv_php_unregister+0x50/0x80 [pnv_php]
 [c0000003bc007d40] [d00000000ad61e84] pnv_php_exit+0x50/0xcb4 [pnv_php]
 [c0000003bc007d70] [c00000000019499c] SyS_delete_module+0x1fc/0x2a0
 [c0000003bc007e30] [c00000000000b184] system_call+0x38/0xe0

Cc: <stable@vger.kernel.org> # v4.8+
Fixes: 66725152fb9f ("PCI/hotplug: PowerPC PowerNV PCI hotplug driver")
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
 drivers/pci/hotplug/pnv_php.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/hotplug/pnv_php.c b/drivers/pci/hotplug/pnv_php.c
index 63cd9f3..da1bbf3 100644
--- a/drivers/pci/hotplug/pnv_php.c
+++ b/drivers/pci/hotplug/pnv_php.c
@@ -76,7 +76,7 @@ static void pnv_php_free_slot(struct kref *kref)
 static inline void pnv_php_put_slot(struct pnv_php_slot *php_slot)
 {
 
-	if (WARN_ON(!php_slot))
+	if (!php_slot)
 		return;
 
 	kref_put(&php_slot->kref, pnv_php_free_slot);
-- 
2.7.4

Re: [PATCH 1/3] pci/hotplug/pnv-php: Remove WARN_ON() in pnv_php_put_slot()

[Andrew Donnellan]

On 16/02/17 10:22, Gavin Shan wrote:
> The WARN_ON() causes unnecessary backtrace when putting the parent
> slot, which is likely to be NULL.
>
>  WARNING: CPU: 2 PID: 1071 at drivers/pci/hotplug/pnv_php.c:85 \
>                               pnv_php_release+0xcc/0x150 [pnv_php]
>     :
>  Call Trace:
>  [c0000003bc007c10] [d00000000ad613c4] pnv_php_release+0x144/0x150 [pnv_php]
>  [c0000003bc007c40] [c0000000006641d8] pci_hp_deregister+0x238/0x330
>  [c0000003bc007cd0] [d00000000ad61440] pnv_php_unregister_one+0x70/0xa0 [pnv_php]
>  [c0000003bc007d10] [d00000000ad614c0] pnv_php_unregister+0x50/0x80 [pnv_php]
>  [c0000003bc007d40] [d00000000ad61e84] pnv_php_exit+0x50/0xcb4 [pnv_php]
>  [c0000003bc007d70] [c00000000019499c] SyS_delete_module+0x1fc/0x2a0
>  [c0000003bc007e30] [c00000000000b184] system_call+0x38/0xe0
>
> Cc: <stable@vger.kernel.org> # v4.8+
> Fixes: 66725152fb9f ("PCI/hotplug: PowerPC PowerNV PCI hotplug driver")
> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>

Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>


-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: [PATCH 1/3] pci/hotplug/pnv-php: Remove WARN_ON() in pnv_php_put_slot()

[Vaibhav Jain]

Andrew Donnellan <andrew.donnellan@au1.ibm.com> writes:
>> Cc: <stable@vger.kernel.org> # v4.8+
>> Fixes: 66725152fb9f ("PCI/hotplug: PowerPC PowerNV PCI hotplug driver")
>> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
>
> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Tested-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>

Re: [1/3] pci/hotplug/pnv-php: Remove WARN_ON() in pnv_php_put_slot()

[Michael Ellerman]

On Wed, 2017-02-15 at 23:22:32 UTC, Gavin Shan wrote:
> The WARN_ON() causes unnecessary backtrace when putting the parent
> slot, which is likely to be NULL.
> 
>  WARNING: CPU: 2 PID: 1071 at drivers/pci/hotplug/pnv_php.c:85 \
>                               pnv_php_release+0xcc/0x150 [pnv_php]
>     :
>  Call Trace:
>  [c0000003bc007c10] [d00000000ad613c4] pnv_php_release+0x144/0x150 [pnv_php]
>  [c0000003bc007c40] [c0000000006641d8] pci_hp_deregister+0x238/0x330
>  [c0000003bc007cd0] [d00000000ad61440] pnv_php_unregister_one+0x70/0xa0 [pnv_php]
>  [c0000003bc007d10] [d00000000ad614c0] pnv_php_unregister+0x50/0x80 [pnv_php]
>  [c0000003bc007d40] [d00000000ad61e84] pnv_php_exit+0x50/0xcb4 [pnv_php]
>  [c0000003bc007d70] [c00000000019499c] SyS_delete_module+0x1fc/0x2a0
>  [c0000003bc007e30] [c00000000000b184] system_call+0x38/0xe0
> 
> Cc: <stable@vger.kernel.org> # v4.8+
> Fixes: 66725152fb9f ("PCI/hotplug: PowerPC PowerNV PCI hotplug driver")
> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
> Tested-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>

Series applied to powerpc next, thanks.

https://git.kernel.org/powerpc/c/36c7c9da40c408a71e5e6bfe12e57d

cheers

[PATCH 2/3] pci/hotplug/pnv-php: Disable surprise hotplug capability on conflicts

[Gavin Shan]

The root port or PCIe switch downstream port might have been associated
with driver other than pnv-php. The MSI or MSIx might also have been
enabled by that driver (e.g. pcieport_drv). Attempt to enable MSI incurs
below backtrace:

 PowerPC PowerNV PCI Hotplug Driver version: 0.1
 ------------[ cut here ]------------
 WARNING: CPU: 19 PID: 1004 at drivers/pci/msi.c:1071 \
                              __pci_enable_msi_range+0x84/0x4e0
 NIP [c000000000665c34] __pci_enable_msi_range+0x84/0x4e0
 LR [c000000000665c24] __pci_enable_msi_range+0x74/0x4e0
 Call Trace:
 [c000000384d67600] [c000000000665c24] __pci_enable_msi_range+0x74/0x4e0
 [c000000384d676e0] [d00000000aa31b04] pnv_php_register+0x564/0x5a0 [pnv_php]
 [c000000384d677c0] [d00000000aa31658] pnv_php_register+0xb8/0x5a0 [pnv_php]
 [c000000384d678a0] [d00000000aa31658] pnv_php_register+0xb8/0x5a0 [pnv_php]
 [c000000384d67980] [d00000000aa31dfc] pnv_php_init+0x60/0x98 [pnv_php]
 [c000000384d679f0] [c00000000000cfdc] do_one_initcall+0x6c/0x1d0
 [c000000384d67ab0] [c000000000b92354] do_init_module+0x94/0x254
 [c000000384d67b40] [c00000000019719c] load_module+0x258c/0x2c60
 [c000000384d67d30] [c000000000197bb0] SyS_finit_module+0xf0/0x170
 [c000000384d67e30] [c00000000000b184] system_call+0x38/0xe0

This fixes the issue by skipping enabling the surprise hotplug
capability if the MSI or MSIx on the PCI slot's upstream port has
been enabled by other driver.

Cc: <stable@vger.kernel.org> # v4.9+
Fixes: 360aebd85a4c ("drivers/pci/hotplug: Support surprise hotplug in powernv driver")
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
 drivers/pci/hotplug/pnv_php.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/pci/hotplug/pnv_php.c b/drivers/pci/hotplug/pnv_php.c
index da1bbf3..6d36b76 100644
--- a/drivers/pci/hotplug/pnv_php.c
+++ b/drivers/pci/hotplug/pnv_php.c
@@ -826,6 +826,14 @@ static void pnv_php_enable_irq(struct pnv_php_slot *php_slot)
 	struct pci_dev *pdev = php_slot->pdev;
 	int irq, ret;
 
+	/*
+	 * The MSI/MSIx interrupt might have been occupied by other
+	 * drivers. Don't populate the surprise hotplug capability
+	 * in that case.
+	 */
+	if (pci_dev_msi_enabled(pdev))
+		return;
+
 	ret = pci_enable_device(pdev);
 	if (ret) {
 		dev_warn(&pdev->dev, "Error %d enabling device\n", ret);
-- 
2.7.4

Re: [PATCH 2/3] pci/hotplug/pnv-php: Disable surprise hotplug capability on conflicts

[Andrew Donnellan]

On 16/02/17 10:22, Gavin Shan wrote:
> The root port or PCIe switch downstream port might have been associated
> with driver other than pnv-php. The MSI or MSIx might also have been
> enabled by that driver (e.g. pcieport_drv). Attempt to enable MSI incurs
> below backtrace:
>
>  PowerPC PowerNV PCI Hotplug Driver version: 0.1
>  ------------[ cut here ]------------
>  WARNING: CPU: 19 PID: 1004 at drivers/pci/msi.c:1071 \
>                               __pci_enable_msi_range+0x84/0x4e0
>  NIP [c000000000665c34] __pci_enable_msi_range+0x84/0x4e0
>  LR [c000000000665c24] __pci_enable_msi_range+0x74/0x4e0
>  Call Trace:
>  [c000000384d67600] [c000000000665c24] __pci_enable_msi_range+0x74/0x4e0
>  [c000000384d676e0] [d00000000aa31b04] pnv_php_register+0x564/0x5a0 [pnv_php]
>  [c000000384d677c0] [d00000000aa31658] pnv_php_register+0xb8/0x5a0 [pnv_php]
>  [c000000384d678a0] [d00000000aa31658] pnv_php_register+0xb8/0x5a0 [pnv_php]
>  [c000000384d67980] [d00000000aa31dfc] pnv_php_init+0x60/0x98 [pnv_php]
>  [c000000384d679f0] [c00000000000cfdc] do_one_initcall+0x6c/0x1d0
>  [c000000384d67ab0] [c000000000b92354] do_init_module+0x94/0x254
>  [c000000384d67b40] [c00000000019719c] load_module+0x258c/0x2c60
>  [c000000384d67d30] [c000000000197bb0] SyS_finit_module+0xf0/0x170
>  [c000000384d67e30] [c00000000000b184] system_call+0x38/0xe0
> 360aebd85a4c
> This fixes the issue by skipping enabling the surprise hotplug
> capability if the MSI or MSIx on the PCI slot's upstream port has
> been enabled by other driver.
>
> Cc: <stable@vger.kernel.org> # v4.9+
> Fixes: 360aebd85a4c ("drivers/pci/hotplug: Support surprise hotplug in powernv driver")
> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>

Should this print something to the kernel log to indicate that surprise 
hotplug is unavailable?

Otherwise:

Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: [PATCH 2/3] pci/hotplug/pnv-php: Disable surprise hotplug capability on conflicts

[Gavin Shan]

On Thu, Feb 16, 2017 at 03:31:48PM +1100, Andrew Donnellan wrote:
>On 16/02/17 10:22, Gavin Shan wrote:
>>The root port or PCIe switch downstream port might have been associated
>>with driver other than pnv-php. The MSI or MSIx might also have been
>>enabled by that driver (e.g. pcieport_drv). Attempt to enable MSI incurs
>>below backtrace:
>>
>> PowerPC PowerNV PCI Hotplug Driver version: 0.1
>> ------------[ cut here ]------------
>> WARNING: CPU: 19 PID: 1004 at drivers/pci/msi.c:1071 \
>>                              __pci_enable_msi_range+0x84/0x4e0
>> NIP [c000000000665c34] __pci_enable_msi_range+0x84/0x4e0
>> LR [c000000000665c24] __pci_enable_msi_range+0x74/0x4e0
>> Call Trace:
>> [c000000384d67600] [c000000000665c24] __pci_enable_msi_range+0x74/0x4e0
>> [c000000384d676e0] [d00000000aa31b04] pnv_php_register+0x564/0x5a0 [pnv_php]
>> [c000000384d677c0] [d00000000aa31658] pnv_php_register+0xb8/0x5a0 [pnv_php]
>> [c000000384d678a0] [d00000000aa31658] pnv_php_register+0xb8/0x5a0 [pnv_php]
>> [c000000384d67980] [d00000000aa31dfc] pnv_php_init+0x60/0x98 [pnv_php]
>> [c000000384d679f0] [c00000000000cfdc] do_one_initcall+0x6c/0x1d0
>> [c000000384d67ab0] [c000000000b92354] do_init_module+0x94/0x254
>> [c000000384d67b40] [c00000000019719c] load_module+0x258c/0x2c60
>> [c000000384d67d30] [c000000000197bb0] SyS_finit_module+0xf0/0x170
>> [c000000384d67e30] [c00000000000b184] system_call+0x38/0xe0
>>360aebd85a4c
>>This fixes the issue by skipping enabling the surprise hotplug
>>capability if the MSI or MSIx on the PCI slot's upstream port has
>>been enabled by other driver.
>>
>>Cc: <stable@vger.kernel.org> # v4.9+
>>Fixes: 360aebd85a4c ("drivers/pci/hotplug: Support surprise hotplug in powernv driver")
>>Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
>
>Should this print something to the kernel log to indicate that surprise
>hotplug is unavailable?
>

I don't think it's necessary as the situation can be known from lspci
and /proc/interrupts easily. Otherwise, we could add pr_warn_once()
to print the warning message for once, not on all slots.

Thanks,
Gavin

Re: [PATCH 2/3] pci/hotplug/pnv-php: Disable surprise hotplug capability on conflicts

[Vaibhav Jain]

Andrew Donnellan <andrew.donnellan@au1.ibm.com> writes:

>>
>> Cc: <stable@vger.kernel.org> # v4.9+
>> Fixes: 360aebd85a4c ("drivers/pci/hotplug: Support surprise hotplug in powernv driver")
>> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
> Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Tested-by: Vaibhav Jain <vaibhav@linux.vnet.ibm.com>

-- 
Vaibhav Jain <vaibhav@linux.vnet.ibm.com>
Linux Technology Center, IBM India Pvt. Ltd.

[PATCH 3/3] pci/hotplug/pnv-php: Disable MSI and PCI device properly

[Gavin Shan]

pnv_php_disable_irq() can be called in two paths: Bailing path in
pnv_php_enable_irq() or releasing slot. The MSI (or MSIx) interrupts
is disabled unconditionally in pnv_php_disable_irq(). It's wrong
because that might be enabled by drivers other than pnv-php.

This disables MSI (or MSIx) interrupts and the PCI device only if
it was enabled by pnv-php. In the error path of pnv_php_enable_irq(),
we rely on the newly added parameter @disable_device. In the path
of releasing slot, @pnv_php->irq is checked.

Cc: <stable@vger.kernel.org> # v4.9+
Fixes: 360aebd85a4c ("drivers/pci/hotplug: Support surprise hotplug in powernv driver")
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
 drivers/pci/hotplug/pnv_php.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/drivers/pci/hotplug/pnv_php.c b/drivers/pci/hotplug/pnv_php.c
index 6d36b76..335a556 100644
--- a/drivers/pci/hotplug/pnv_php.c
+++ b/drivers/pci/hotplug/pnv_php.c
@@ -35,9 +35,11 @@ static void pnv_php_register(struct device_node *dn);
 static void pnv_php_unregister_one(struct device_node *dn);
 static void pnv_php_unregister(struct device_node *dn);
 
-static void pnv_php_disable_irq(struct pnv_php_slot *php_slot)
+static void pnv_php_disable_irq(struct pnv_php_slot *php_slot,
+				bool disable_device)
 {
 	struct pci_dev *pdev = php_slot->pdev;
+	int irq = php_slot->irq;
 	u16 ctrl;
 
 	if (php_slot->irq > 0) {
@@ -56,10 +58,14 @@ static void pnv_php_disable_irq(struct pnv_php_slot *php_slot)
 		php_slot->wq = NULL;
 	}
 
-	if (pdev->msix_enabled)
-		pci_disable_msix(pdev);
-	else if (pdev->msi_enabled)
-		pci_disable_msi(pdev);
+	if (disable_device || irq > 0) {
+		if (pdev->msix_enabled)
+			pci_disable_msix(pdev);
+		else if (pdev->msi_enabled)
+			pci_disable_msi(pdev);
+
+		pci_disable_device(pdev);
+	}
 }
 
 static void pnv_php_free_slot(struct kref *kref)
@@ -68,7 +74,7 @@ static void pnv_php_free_slot(struct kref *kref)
 					struct pnv_php_slot, kref);
 
 	WARN_ON(!list_empty(&php_slot->children));
-	pnv_php_disable_irq(php_slot);
+	pnv_php_disable_irq(php_slot, false);
 	kfree(php_slot->name);
 	kfree(php_slot);
 }
@@ -777,7 +783,7 @@ static void pnv_php_init_irq(struct pnv_php_slot *php_slot, int irq)
 	php_slot->wq = alloc_workqueue("pciehp-%s", 0, 0, php_slot->name);
 	if (!php_slot->wq) {
 		dev_warn(&pdev->dev, "Cannot alloc workqueue\n");
-		pnv_php_disable_irq(php_slot);
+		pnv_php_disable_irq(php_slot, true);
 		return;
 	}
 
@@ -799,7 +805,7 @@ static void pnv_php_init_irq(struct pnv_php_slot *php_slot, int irq)
 	ret = request_irq(irq, pnv_php_interrupt, IRQF_SHARED,
 			  php_slot->name, php_slot);
 	if (ret) {
-		pnv_php_disable_irq(php_slot);
+		pnv_php_disable_irq(php_slot, true);
 		dev_warn(&pdev->dev, "Error %d enabling IRQ %d\n", ret, irq);
 		return;
 	}
-- 
2.7.4

Re: [PATCH 3/3] pci/hotplug/pnv-php: Disable MSI and PCI device properly

[Andrew Donnellan]

On 16/02/17 10:22, Gavin Shan wrote:
> pnv_php_disable_irq() can be called in two paths: Bailing path in
> pnv_php_enable_irq() or releasing slot. The MSI (or MSIx) interrupts
> is disabled unconditionally in pnv_php_disable_irq(). It's wrong
> because that might be enabled by drivers other than pnv-php.
>
> This disables MSI (or MSIx) interrupts and the PCI device only if
> it was enabled by pnv-php. In the error path of pnv_php_enable_irq(),
> we rely on the newly added parameter @disable_device. In the path
> of releasing slot, @pnv_php->irq is checked.
>
> Cc: <stable@vger.kernel.org> # v4.9+
> Fixes: 360aebd85a4c ("drivers/pci/hotplug: Support surprise hotplug in powernv driver")
> Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>

Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited