Re: [PATCH kernel] powerpc/powernv: Update comment about shifting IOV BAR

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

On Thu, 2017-09-14 at 13:18 +1000, Alexey Kardashevskiy wrote:
> On 14/09/17 13:07, Benjamin Herrenschmidt wrote:
> > On Thu, 2017-09-14 at 12:45 +1000, Alexey Kardashevskiy wrote:
> > > On 31/08/17 13:34, Alexey Kardashevskiy wrote:
> > > > From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> > > 
> > > Oops, this was not right :)
> > > 
> > > Anyway, Ben, please comment. Thanks.
> > 
> > This is incorrect, we can do hotplug behind switches afaik.
> 
> Do we have an actual system which allows this? 

Tuleta no ?

> Anyway, what we do now is
> wrong and it needs what? Reserve that hole? I'd like to update the comment
> for now, at least, and state what bad thing can happen and what we expect.

The hole should be reserved unless another SR-IOV device can use it ...

> 
> > > 
> > > > 
> > > > From: Alexey Kardashevskiy <aik@ozlabs.ru>
> > > > 
> > > > This updates the comment about creating a hole in /proc/iomem which
> > > > should not be normally happening but it does in the powernv platform
> > > > due the way MMIO M64 BARs are organised in the IODA2-capable hardware.
> > > > 
> > > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > > > ---
> > > > 
> > > > It has been mentioned multiple times (last one -
> > > > https://www.spinics.net/lists/linux-pci/msg64084.html ) that the comment
> > > > is not informative enough for people not particularly familiar with
> > > > the POWER8 IO hardware.
> > > > 
> > > > This attempt aims to:
> > > > 1. explain why we shift the resource
> > > > 2. explain why nothing can use that hole as a resource while it is "free"
> > > > (I am not sure that this is the case actually)
> > > > 
> > > > Please comment, everyone, let's have this very well documented while
> > > > I remember these bits :) Thanks.
> > > > ---
> > > >  arch/powerpc/platforms/powernv/pci-ioda.c | 10 +++++++---
> > > >  1 file changed, 7 insertions(+), 3 deletions(-)
> > > > 
> > > > diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
> > > > index 48de308224d6..c4a36ae78c95 100644
> > > > --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> > > > +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> > > > @@ -1002,9 +1002,13 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
> > > >  	}
> > > >  
> > > >  	/*
> > > > -	 * After doing so, there would be a "hole" in the /proc/iomem when
> > > > -	 * offset is a positive value. It looks like the device return some
> > > > -	 * mmio back to the system, which actually no one could use it.
> > > > +	 * Since M64 BAR shares segments among all possible 256 PEs,
> > > > +	 * we have to shift the beginning of PF IOV BAR to make it start from
> > > > +	 * the segment which belongs to the PE number assigned to the first VF.
> > > > +	 * This creates a "hole" in the /proc/iomem which could be used for
> > > > +	 * allocating other resources, however this is not expected to happen
> > > > +	 * on IODA as the only possibility would be a PCI hotplug and IODA
> > > > +	 * hardware only allows it on a slot with dedicated PHB.
> > > >  	 */
> > > >  	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
> > > >  		res = &dev->resource[i + PCI_IOV_RESOURCES];
> > > > 
> > > 
> > > 
> 
>