Re: [PATCH 1/2] irqchip/gicv3-its: Support share device ID

[Will Deacon <will.deacon@arm.com>]

On Wed, Apr 22, 2015 at 08:41:02PM +0100, Stuart Yoder wrote:
> > > However, there is an improvement we envision as possible due to
> > > the limited number of SMMU contexts (i.e. 64).  If there are
> > > 64 SMMU context registers it means that there is a max of
> > > 64 software contexts where things can be isolated.  But, say I have
> > > an SRIOV card with 64 VFs, and I want to assign 8 of the VFs
> > > to a KVM VM.  Those 8 PCI devices could share the same
> > > streamID/ITS-device-ID since they all share the same isolation
> > > context.
> > >
> > > What would be nice is at the time the 8 VFS are being added
> > > to the IOMMU domain is for the pcidevid -> streamID mapping
> > > table to be updated dynamically.  It simply lets us make
> > > more efficient use of the limited streamIDs we have.
> > >
> > > I think it is this improvement that Minghuan had in mind
> > > in this patch.
> > 
> > Ok, but in this case it should be possible to use a single context bank for
> > all of the VF streamIDs by configuring the appropriate SMR, no?
> 
> Yes, but there are limited SMRs.  In our case there are only
> 128 SMRs in SMMU-500 and we have potentially way more masters than
> that.

Right, but you still only have 64 context banks at the end of the day, so do
you really anticipate having more than 128 masters concurrently using the
SMMU? If so, then we have devices sharing context banks so we could consider
reusing SMRs across masters, but historically that's not been something that
we've managed to solve.

> > Wouldn't
> > that sort of thing be preferable to dynamic StreamID assignment? It would
> > certainly make life easier for the MSIs.
> 
> It would be preferable, but given only 128 total stream IDS and
> 64 context registers it's potentially an issue.  On our LS2085 SoC it is
> PCI and the fsl-mc bus (see description here:
> https://lkml.org/lkml/2015/3/5/795) that potentially have way
> more masters than streamIDS. So, for those busses we would essentially
> view a streamID as a "context ID"-- each SMR is associated with
> 1 context bank register.
> 
> For PCI we have a programmable "PCI req ID"-to-"stream ID"
> mapping table in the PCI controller that is dynamically
> programmable.
> 
> Looking at it like that means that we could have
> any number of masters but only 64 "contexts"
> and since the masters all all programmable it's
> seems feasbile to envision doing some bus/vendor
> specific set up when a device is added to an
> IOMMU domain.   One thing that would need to be conveyed
> to the SMMU driver if doing dynamic streamID setup
> is what streamIDs are available to be used.

Ok, but this is going to make life difficult for the MSI people, I suspect.

Marc...?

Will