[PATCH] powerpc/pseries: Force 32 bit MSIs when tearing down

[PATCH] powerpc/pseries: Force 32 bit MSIs when tearing down

[Alexander Gordeev]

This fix just adds a missed call to a new PAPR function
which should have been done with commit e61133d ("powerpc/
pseries: Force 32 bit MSIs for devices that require it")

Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 arch/powerpc/platforms/pseries/msi.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
index 40c7db3..2b80a68 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -90,6 +90,7 @@ static int rtas_change_msi(struct pci_dn *pdn, u32 func, u32 num_irqs)
 static void rtas_disable_msi(struct pci_dev *pdev)
 {
 	struct pci_dn *pdn;
+	int rc;
 
 	pdn = get_pdn(pdev);
 	if (!pdn)
@@ -98,7 +99,12 @@ static void rtas_disable_msi(struct pci_dev *pdev)
 	/*
 	 * disabling MSI with the explicit interface also disables MSI-X
 	 */
-	if (rtas_change_msi(pdn, RTAS_CHANGE_MSI_FN, 0) != 0) {
+	if (pdn->force_32bit_msi)
+		rc = rtas_change_msi(pdn, RTAS_CHANGE_32MSI_FN, 0);
+	else
+		rc = rtas_change_msi(pdn, RTAS_CHANGE_MSI_FN, 0);
+
+	if ((rc != 0) && !pdn->force_32bit_msi) {
 		/* 
 		 * may have failed because explicit interface is not
 		 * present
-- 
1.7.7.6


-- 
Regards,
Alexander Gordeev
agordeev@redhat.com

Re: [PATCH] powerpc/pseries: Force 32 bit MSIs when tearing down

[Benjamin Herrenschmidt]

On Mon, 2013-05-27 at 18:20 +0200, Alexander Gordeev wrote:
> This fix just adds a missed call to a new PAPR function
> which should have been done with commit e61133d ("powerpc/
> pseries: Force 32 bit MSIs for devices that require it")

Arguably, PAPR should allow to disable MSIs using either interface,
we shouldn't have to know whether the MSI was a forced-32-bit one to be
able to disable it.

Brian, can you check with Colleen ?

Cheers,
Ben.

> Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
> ---
>  arch/powerpc/platforms/pseries/msi.c |    8 +++++++-
>  1 files changed, 7 insertions(+), 1 deletions(-)
> 
> diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
> index 40c7db3..2b80a68 100644
> --- a/arch/powerpc/platforms/pseries/msi.c
> +++ b/arch/powerpc/platforms/pseries/msi.c
> @@ -90,6 +90,7 @@ static int rtas_change_msi(struct pci_dn *pdn, u32 func, u32 num_irqs)
>  static void rtas_disable_msi(struct pci_dev *pdev)
>  {
>  	struct pci_dn *pdn;
> +	int rc;
>  
>  	pdn = get_pdn(pdev);
>  	if (!pdn)
> @@ -98,7 +99,12 @@ static void rtas_disable_msi(struct pci_dev *pdev)
>  	/*
>  	 * disabling MSI with the explicit interface also disables MSI-X
>  	 */
> -	if (rtas_change_msi(pdn, RTAS_CHANGE_MSI_FN, 0) != 0) {
> +	if (pdn->force_32bit_msi)
> +		rc = rtas_change_msi(pdn, RTAS_CHANGE_32MSI_FN, 0);
> +	else
> +		rc = rtas_change_msi(pdn, RTAS_CHANGE_MSI_FN, 0);
> +
> +	if ((rc != 0) && !pdn->force_32bit_msi) {
>  		/* 
>  		 * may have failed because explicit interface is not
>  		 * present
> -- 
> 1.7.7.6
> 
> 

Re: [PATCH] powerpc/pseries: Force 32 bit MSIs when tearing down

[Alexander Gordeev]

On Tue, May 28, 2013 at 07:52:11AM +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2013-05-27 at 18:20 +0200, Alexander Gordeev wrote:
> > This fix just adds a missed call to a new PAPR function
> > which should have been done with commit e61133d ("powerpc/
> > pseries: Force 32 bit MSIs for devices that require it")
> 
> Arguably, PAPR should allow to disable MSIs using either interface,
> we shouldn't have to know whether the MSI was a forced-32-bit one to be
> able to disable it.

BTW, it is not clear why MSIs do not get disabled on powernv on teardown.
Would this (pseudo-code) make sense?


diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 163bd74..1502c49 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -102,6 +102,7 @@ static void pnv_teardown_msi_irqs(struct pci_dev *pdev)
 	struct pci_controller *hose = pci_bus_to_host(pdev->bus);
 	struct pnv_phb *phb = hose->private_data;
 	struct msi_desc *entry;
+	unsigned int hwirq;
 
 	if (WARN_ON(!phb))
 		return;
@@ -109,10 +110,15 @@ static void pnv_teardown_msi_irqs(struct pci_dev *pdev)
 	list_for_each_entry(entry, &pdev->msi_list, list) {
 		if (entry->irq == NO_IRQ)
 			continue;
+
 		irq_set_msi_desc(entry->irq, NULL);
+		hwirq = virq_to_hw(entry->irq);
 		msi_bitmap_free_hwirqs(&phb->msi_bmp,
-			virq_to_hw(entry->irq) - phb->msi_base, 1);
+			hwirq - phb->msi_base, 1);
 		irq_dispose_mapping(entry->irq);
+		if (phb->msi_disable)
+			phb->msi_disable(phb, pdev, hwirq,
+					 entry->msi_attrib.is_64);
 	}
 }
 #endif /* CONFIG_PCI_MSI */
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index 25d76c4..049cbd3 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -79,10 +79,13 @@ struct pnv_phb {
 	unsigned int		msi_base;
 	unsigned int		msi32_support;
 	struct msi_bitmap	msi_bmp;
-#endif
+
 	int (*msi_setup)(struct pnv_phb *phb, struct pci_dev *dev,
 			 unsigned int hwirq, unsigned int virq,
 			 unsigned int is_64, struct msi_msg *msg);
+	int (*msi_disable)(struct pnv_phb *phb, struct pci_dev *dev,
+			   unsigned int hwirq, unsigned int is_64);
+#endif
 	void (*dma_dev_setup)(struct pnv_phb *phb, struct pci_dev *pdev);
 	void (*fixup_phb)(struct pci_controller *hose);
 	u32 (*bdfn_to_pe)(struct pnv_phb *phb, struct pci_bus *bus, u32 devfn);

-- 
Regards,
Alexander Gordeev
agordeev@redhat.com

Re: [PATCH] powerpc/pseries: Force 32 bit MSIs when tearing down

[Benjamin Herrenschmidt]

On Tue, 2013-05-28 at 09:20 +0200, Alexander Gordeev wrote:
> On Tue, May 28, 2013 at 07:52:11AM +1000, Benjamin Herrenschmidt wrote:
> > On Mon, 2013-05-27 at 18:20 +0200, Alexander Gordeev wrote:
> > > This fix just adds a missed call to a new PAPR function
> > > which should have been done with commit e61133d ("powerpc/
> > > pseries: Force 32 bit MSIs for devices that require it")
> > 
> > Arguably, PAPR should allow to disable MSIs using either interface,
> > we shouldn't have to know whether the MSI was a forced-32-bit one to be
> > able to disable it.
> 
> BTW, it is not clear why MSIs do not get disabled on powernv on teardown.
> Would this (pseudo-code) make sense?

Because we don't really have anything to do there. The MSI is just a normal
interrupt source in the PHB, so the "normal" disable_irq (part of free_irq)
will take care of masking it, and from there it's just returning it to the pool
of available interrupts (the bitmap) and removing the mapping.

Cheers,
Ben.

Re: [PATCH] powerpc/pseries: Force 32 bit MSIs when tearing down

[Brian King]

On 05/27/2013 04:52 PM, Benjamin Herrenschmidt wrote:
> On Mon, 2013-05-27 at 18:20 +0200, Alexander Gordeev wrote:
>> This fix just adds a missed call to a new PAPR function
>> which should have been done with commit e61133d ("powerpc/
>> pseries: Force 32 bit MSIs for devices that require it")
> 
> Arguably, PAPR should allow to disable MSIs using either interface,
> we shouldn't have to know whether the MSI was a forced-32-bit one to be
> able to disable it.
> 
> Brian, can you check with Colleen ?

That is correct, and is the exact path we went down when I tested the code
that is currently upstream. I don't think this patch should be needed.

Thanks,

Brian


> 
> Cheers,
> Ben.
> 
>> Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
>> ---
>>  arch/powerpc/platforms/pseries/msi.c |    8 +++++++-
>>  1 files changed, 7 insertions(+), 1 deletions(-)
>>
>> diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
>> index 40c7db3..2b80a68 100644
>> --- a/arch/powerpc/platforms/pseries/msi.c
>> +++ b/arch/powerpc/platforms/pseries/msi.c
>> @@ -90,6 +90,7 @@ static int rtas_change_msi(struct pci_dn *pdn, u32 func, u32 num_irqs)
>>  static void rtas_disable_msi(struct pci_dev *pdev)
>>  {
>>  	struct pci_dn *pdn;
>> +	int rc;
>>  
>>  	pdn = get_pdn(pdev);
>>  	if (!pdn)
>> @@ -98,7 +99,12 @@ static void rtas_disable_msi(struct pci_dev *pdev)
>>  	/*
>>  	 * disabling MSI with the explicit interface also disables MSI-X
>>  	 */
>> -	if (rtas_change_msi(pdn, RTAS_CHANGE_MSI_FN, 0) != 0) {
>> +	if (pdn->force_32bit_msi)
>> +		rc = rtas_change_msi(pdn, RTAS_CHANGE_32MSI_FN, 0);
>> +	else
>> +		rc = rtas_change_msi(pdn, RTAS_CHANGE_MSI_FN, 0);
>> +
>> +	if ((rc != 0) && !pdn->force_32bit_msi) {
>>  		/* 
>>  		 * may have failed because explicit interface is not
>>  		 * present
>> -- 
>> 1.7.7.6
>>
>>
> 
> 


-- 
Brian King
Power Linux I/O
IBM Linux Technology Center