Re: [Qemu-devel] RFC: vfio API changes needed for powerpc

[Alex Williamson <alex.williamson@redhat.com>]

On Wed, 2013-04-03 at 14:09 -0500, Stuart Yoder wrote:
> > Would is be possible for userspace to simply leave room for MSI bank
> > mapping (how much room could be determined by something like
> > VFIO_IOMMU_GET_MSI_BANK_COUNT) then document the API that userspace can
> > DMA_MAP starting at the 0x0 address of the aperture, growing up, and
> > VFIO will map banks on demand at the top of the aperture, growing down?
> > Wouldn't that avoid a lot of issues with userspace needing to know
> > anything about MSI banks (other than count) and coordinating irq numbers
> > and enabling handlers?
> 
> This is basically option #A in the original proposals sent.   I like
> this approach, in that it
> is simpler and keeps user space mostly out of this...which is
> consistent with how things are done
> on x86.  User space just needs to know how many windows to leave at
> the top of the aperture.
> The kernel then has the flexibility to use those windows how it wants.
> 
> But one question, is when should the kernel actually map (and unmap)
> the MSI banks.   One thing we need to do is enable the aperture...and current
> thinking is that is done on the first DMA_MAP.   Similarly when the last mapping
> is unmapped we could also umap the MSI banks.
> 
> Sequence would be something like:
> 
>         VFIO_GROUP_SET_CONTAINER     // add groups to the container
> 
>         VFIO_SET_IOMMU(VFIO_FSL_PAMU)    // set iommu model
> 
>         cnt = VFIO_IOMMU_GET_MSI_BANK_COUNT    // returns max # of MSI banks
> 
>         VFIO_IOMMU_SET_ATTR(ATTR_GEOMETRY) // set overall aperture
> 
>         VFIO_IOMMU_SET_ATTR(ATTR_WINDOWS) // set # of windows,
> including MSI banks
> 
>         VFIO_IOMMU_MAP_DMA    // map the guest's memory
>                ---> kernel enables aperture and maps needed MSI banks here
> 
>         VFIO_DEVICE_SET_IRQS
>                ---> kernel sets actual MSI addr/data in physical
> device here (I think)

You could also make use of the IOMMU_ENABLE/DISABLE entry points that
Alexey plans to use.  Ideally I'd think that you'd want to enable the
required MSI banks for a device on DEVICE_SET_IRQs.  That's effectively
what happens on x86.  Perhaps some information stored in the domain
structure would let architecture hooks in MSI setup enable those
mappings for you?  Thanks,

Alex