Re: [PATCH v2] pseries/iommu: remove iommu device references via bus notifier

[Nishanth Aravamudan <nacc@linux.vnet.ibm.com>]

On 21.02.2015 [11:00:50 -0800], Nishanth Aravamudan wrote:
> On 20.02.2015 [15:31:29 +1100], Michael Ellerman wrote:
> > On Thu, 2015-02-19 at 10:41 -0800, Nishanth Aravamudan wrote:
> > > After d905c5df9aef ("PPC: POWERNV: move iommu_add_device earlier"), the
> > > refcnt on the kobject backing the IOMMU group for a PCI device is
> > > elevated by each call to pci_dma_dev_setup_pSeriesLP() (via
> > > set_iommu_table_base_and_group). When we go to dlpar a multi-function
> > > PCI device out:
> > > 
> > > 	iommu_reconfig_notifier ->
> > > 		iommu_free_table ->
> > > 			iommu_group_put
> > > 			BUG_ON(tbl->it_group)
> > > 
> > > We trip this BUG_ON, because there are still references on the table, so
> > > it is not freed. Fix this by also adding a bus notifier identical to
> > > PowerNV for pSeries.
> > 
> > Please put it somewhere common, arch/powerpc/kernel/iommu.c perhaps, and just
> > add a second machine_init_call() for pseries.
> 
> How does this look? Only compile-tested with CONFIG_IOMMU_API on/off so
> far, waiting for access to the test LPAR (should have it on Monday).
> 
> 
> After d905c5df9aef ("PPC: POWERNV: move iommu_add_device earlier"), the
> refcnt on the kobject backing the IOMMU group for a PCI device is
> elevated by each call to pci_dma_dev_setup_pSeriesLP() (via
> set_iommu_table_base_and_group). When we go to dlpar a multi-function
> PCI device out:
> 
>         iommu_reconfig_notifier ->
>                 iommu_free_table ->
>                         iommu_group_put
>                         BUG_ON(tbl->it_group)
> 
> We trip this BUG_ON, because there are still references on the table, so
> it is not freed. Fix this by moving the PowerNV bus notifier to common
> code and calling it for both PowerNV and pSeries.

Survived a remove -> add -> remove cycle, which always resulted in the
BUG_ON without the change.

> Fixes: d905c5df9aef ("PPC: POWERNV: move iommu_add_device earlier")
> Signed-off-by: Nishanth Aravamudan <nacc@linux.vnet.ibm.com>
> Cc: stable@kernel.org (3.13+)

Tested-by: Nishanth Aravamudan <nacc@linux.vnet.ibm.com>