Re: [Qemu-devel] [Qemu-ppc] [for-2.11 PATCH 18/26] spapr: create DR connectors for PHBs

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 2793 bytes --]

On Wed, Sep 13, 2017 at 02:56:51PM +0200, Greg Kurz wrote:
> On Wed, 13 Sep 2017 22:23:29 +1000
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> [...snip...]
> > > > > Also, if all PHBs are instanciated with index != -1, we're limited to 31.
> > > > > Maybe this could be the default value for the machine property instead of
> > > > > 256 then ?    
> > > > 
> > > > Actually, if we're binding it back to index, which has a hard limit,
> > > > then it no longer makes sense to have it as a property and we should
> > > > go back to a constant (well, it could vary by machine type version).  
> > 
> > Sorry I've taken so long to reply.
> > 
> 
> Oh, don't mention it. :)
> 
> > > But I guess that the hard limit of 31 as described in the changelog of
> > > commit 357d1e3bc7d2d80e5271bc4f3ac8537e30dc8046 still holds, doesn't
> > > it ?  
> > 
> > That's right.  Note that that is a limit of *31* PHBs (numbered
> > 0..30), not 32 PHBs numbered 0..31.
> > 
> 
> Yeah I saw that.
> 
> > >     Because some guest versions (including most current distro kernels) can't
> > >     access PCI MMIO above 64 TiB, we put all the PCI windows between 32 TiB and
> > >     64 TiB.  This is broken into 1 TiB chunks.  The first 1 TiB contains the
> > >     PIO (64 kiB) and 32-bit MMIO (2 GiB) windows for all of the PHBs.  Each
> > >     subsequent TiB chunk contains a naturally aligned 64-bit MMIO window for
> > >     one PHB each.
> > >     
> > >     This reduces the number of allowed PHBs (without full manual configuration
> > >     of all the windows) from 256 to 31, but this should still be plenty in
> > >     practice.
> > > 
> > > Not sure why a machine type version would have a different limit. Can
> > > you think of a use case ?  
> > 
> > Well, the older machine types had a different layout.  It allowed for
> > more indexes, but had smaller windows, which meant certain cards (e.g.
> > GPGPUs with huge BARs) didn't work properly.  It also had some weird
> > alignments that meant we were a bit wasteful of address space.
> > 
> > But we can't change the location of PHB windows on migration, so we
> > had to maintain that old layout for old machine types.  That's why
> > there's a different limit depending on machine type version.
> > 
> 
> Ok, so we *just* have 2 different maximum number of PHBs:
> - 256 for pseries <= 2.7
> - 31 for newer machine types

Yes, at least for now.  If we ever discover that 31 PHBs *isn't*
enough, we may have to come up with yet another layout, which could
have a different limit.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]