From mboxrd@z Thu Jan 1 00:00:00 1970 From: Scott Wood Subject: Re: RFC: vfio API changes needed for powerpc Date: Wed, 3 Apr 2013 15:00:53 -0500 Message-ID: <1365019253.25627.8@snotra> References: <1364943035.24520.28@snotra> <1364960240.2882.230.camel@bling.home> <1365016682.25627.6@snotra> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="Flowed"; DelSp="Yes" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: (from b08248-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org on Wed Apr 3 14:43:06 2013) Content-Disposition: inline List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Stuart Yoder Cc: Wood Scott-B07421 , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "agraf-l3A5Bk7waGM@public.gmane.org" , "qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org" , Yoder Stuart-B08248 , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , Bhushan Bharat-R65777 List-Id: iommu@lists.linux-foundation.org On 04/03/2013 02:43:06 PM, Stuart Yoder wrote: > On Wed, Apr 3, 2013 at 2:18 PM, Scott Wood > wrote: > > On 04/03/2013 02:09:45 PM, Stuart Yoder wrote: > >> > >> > Would is be possible for userspace to simply leave room for MSI > bank > >> > mapping (how much room could be determined by something like > >> > VFIO_IOMMU_GET_MSI_BANK_COUNT) then document the API that > userspace can > >> > DMA_MAP starting at the 0x0 address of the aperture, growing up, > and > >> > VFIO will map banks on demand at the top of the aperture, > growing down? > >> > Wouldn't that avoid a lot of issues with userspace needing to > know > >> > anything about MSI banks (other than count) and coordinating irq > numbers > >> > and enabling handlers? > >> > >> This is basically option #A in the original proposals sent. I > like > >> this approach, in that it > >> is simpler and keeps user space mostly out of this...which is > >> consistent with how things are done > >> on x86. User space just needs to know how many windows to leave at > >> the top of the aperture. > >> The kernel then has the flexibility to use those windows how it > wants. > >> > >> But one question, is when should the kernel actually map (and > unmap) > >> the MSI banks. > > > > > > I think userspace should explicitly request it. Userspace still > wouldn't > > need to know anything but the count: > > > > count = VFIO_IOMMU_GET_MSI_BANK_COUNT > > VFIO_IOMMU_SET_ATTR(ATTR_GEOMETRY) > > VFIO_IOMMU_SET_ATTR(ATTR_WINDOWS) > > // do other DMA maps now, or later, or not at all, doesn't matter > > for (i = 0; i < count; i++) > > VFIO_IOMMU_MAP_MSI_BANK(iova, i); > > // The kernel now knows where each bank has been mapped, and can > update PCI > > config space appropriately. > > And the overall aperture enable/disable would occur on the first > dma/msi map() and last dma/msi unmap()? Yes. We may want the optional ability to do an overall enable/disable for reasons we discussed a while ago, but in the absence of an explicit disable the domain would be enabled on first map. -Scott From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:57924) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UNTrn-0000CN-E2 for qemu-devel@nongnu.org; Wed, 03 Apr 2013 16:01:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UNTrk-0003Ne-Nk for qemu-devel@nongnu.org; Wed, 03 Apr 2013 16:01:07 -0400 Received: from co9ehsobe002.messaging.microsoft.com ([207.46.163.25]:27749 helo=co9outboundpool.messaging.microsoft.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UNTrk-0003NX-FX for qemu-devel@nongnu.org; Wed, 03 Apr 2013 16:01:04 -0400 Date: Wed, 3 Apr 2013 15:00:53 -0500 From: Scott Wood References: <1364943035.24520.28@snotra> <1364960240.2882.230.camel@bling.home> <1365016682.25627.6@snotra> In-Reply-To: (from b08248@gmail.com on Wed Apr 3 14:43:06 2013) Message-ID: <1365019253.25627.8@snotra> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; delsp=Yes; format=Flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] RFC: vfio API changes needed for powerpc List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stuart Yoder Cc: Wood Scott-B07421 , "kvm@vger.kernel.org" , "agraf@suse.de" , "qemu-devel@nongnu.org" , Yoder Stuart-B08248 , "iommu@lists.linux-foundation.org" , Bhushan Bharat-R65777 On 04/03/2013 02:43:06 PM, Stuart Yoder wrote: > On Wed, Apr 3, 2013 at 2:18 PM, Scott Wood =20 > wrote: > > On 04/03/2013 02:09:45 PM, Stuart Yoder wrote: > >> > >> > Would is be possible for userspace to simply leave room for MSI =20 > bank > >> > mapping (how much room could be determined by something like > >> > VFIO_IOMMU_GET_MSI_BANK_COUNT) then document the API that =20 > userspace can > >> > DMA_MAP starting at the 0x0 address of the aperture, growing up, =20 > and > >> > VFIO will map banks on demand at the top of the aperture, =20 > growing down? > >> > Wouldn't that avoid a lot of issues with userspace needing to =20 > know > >> > anything about MSI banks (other than count) and coordinating irq =20 > numbers > >> > and enabling handlers? > >> > >> This is basically option #A in the original proposals sent. I =20 > like > >> this approach, in that it > >> is simpler and keeps user space mostly out of this...which is > >> consistent with how things are done > >> on x86. User space just needs to know how many windows to leave at > >> the top of the aperture. > >> The kernel then has the flexibility to use those windows how it =20 > wants. > >> > >> But one question, is when should the kernel actually map (and =20 > unmap) > >> the MSI banks. > > > > > > I think userspace should explicitly request it. Userspace still =20 > wouldn't > > need to know anything but the count: > > > > count =3D VFIO_IOMMU_GET_MSI_BANK_COUNT > > VFIO_IOMMU_SET_ATTR(ATTR_GEOMETRY) > > VFIO_IOMMU_SET_ATTR(ATTR_WINDOWS) > > // do other DMA maps now, or later, or not at all, doesn't matter > > for (i =3D 0; i < count; i++) > > VFIO_IOMMU_MAP_MSI_BANK(iova, i); > > // The kernel now knows where each bank has been mapped, and can =20 > update PCI > > config space appropriately. >=20 > And the overall aperture enable/disable would occur on the first > dma/msi map() and last dma/msi unmap()? Yes. We may want the optional ability to do an overall enable/disable =20 for reasons we discussed a while ago, but in the absence of an explicit =20 disable the domain would be enabled on first map. -Scott=