From mboxrd@z Thu Jan 1 00:00:00 1970 From: Scott Wood Subject: Re: RFC: vfio API changes needed for powerpc Date: Wed, 3 Apr 2013 16:25:54 -0500 Message-ID: <1365024354.25627.14@snotra> References: <1364958751.2882.209.camel@bling.home> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="Flowed"; DelSp="Yes" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1364958751.2882.209.camel-xdHQ/5r00wBBDLzU/O5InQ@public.gmane.org> (from alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org on Tue Apr 2 22:12:31 2013) Content-Disposition: inline List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Alex Williamson Cc: Wood Scott-B07421 , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "agraf-l3A5Bk7waGM@public.gmane.org" , "qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org" , Yoder Stuart-B08248 , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , Bhushan Bharat-R65777 List-Id: iommu@lists.linux-foundation.org On 04/02/2013 10:12:31 PM, Alex Williamson wrote: > On Tue, 2013-04-02 at 17:44 -0500, Scott Wood wrote: > > On 04/02/2013 04:32:04 PM, Alex Williamson wrote: > > > On Tue, 2013-04-02 at 15:57 -0500, Scott Wood wrote: > > > > On 04/02/2013 03:32:17 PM, Alex Williamson wrote: > > > > > On x86 the interrupt remapper handles this transparently when > MSI > > > > > is enabled and userspace never gets direct access to the > device > > > MSI > > > > > address/data registers. > > > > > > > > x86 has a totally different mechanism here, as far as I > understand > > > -- > > > > even before you get into restrictions on mappings. > > > > > > So what control will userspace have over programming the actually > MSI > > > vectors on PAMU? > > > > Not sure what you mean -- PAMU doesn't get explicitly involved in > > MSIs. It's just another 4K page mapping (per relevant MSI bank). > If > > you want isolation, you need to make sure that an MSI group is only > > used by one VFIO group, and that you're on a chip that has alias > pages > > with just one MSI bank register each (newer chips do, but the first > > chip to have a PAMU didn't). > > How does a user figure this out? The user's involvement could be limited to setting a policy knob of whether that degree of isolation is required (if required and unavailable, all devices using an MSI bank would be forced into the same group). We'd need to do something with MSI allocation so that we avoid using an MSI bank with more than one IOMMU group where possible. I'm not sure about the details yet, or how practical this is. There might need to be some MSI bank assignment done as part of the VFIO device binding process, if there are going to be more VFIO groups than there are MSI banks (reserving one bank for host use). -Scott From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:56913) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UNVQc-0002vA-0M for qemu-devel@nongnu.org; Wed, 03 Apr 2013 17:41:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UNVQT-0003kK-Sx for qemu-devel@nongnu.org; Wed, 03 Apr 2013 17:41:09 -0400 Received: from tx2ehsobe003.messaging.microsoft.com ([65.55.88.13]:7896 helo=tx2outboundpool.messaging.microsoft.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UNVQT-0003k8-Le for qemu-devel@nongnu.org; Wed, 03 Apr 2013 17:41:01 -0400 Date: Wed, 3 Apr 2013 16:25:54 -0500 From: Scott Wood In-Reply-To: <1364958751.2882.209.camel@bling.home> (from alex.williamson@redhat.com on Tue Apr 2 22:12:31 2013) Message-ID: <1365024354.25627.14@snotra> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; delsp=Yes; format=Flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] RFC: vfio API changes needed for powerpc List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Williamson Cc: Wood Scott-B07421 , "kvm@vger.kernel.org" , "agraf@suse.de" , "qemu-devel@nongnu.org" , Yoder Stuart-B08248 , "iommu@lists.linux-foundation.org" , Bhushan Bharat-R65777 On 04/02/2013 10:12:31 PM, Alex Williamson wrote: > On Tue, 2013-04-02 at 17:44 -0500, Scott Wood wrote: > > On 04/02/2013 04:32:04 PM, Alex Williamson wrote: > > > On Tue, 2013-04-02 at 15:57 -0500, Scott Wood wrote: > > > > On 04/02/2013 03:32:17 PM, Alex Williamson wrote: > > > > > On x86 the interrupt remapper handles this transparently when =20 > MSI > > > > > is enabled and userspace never gets direct access to the =20 > device > > > MSI > > > > > address/data registers. > > > > > > > > x86 has a totally different mechanism here, as far as I =20 > understand > > > -- > > > > even before you get into restrictions on mappings. > > > > > > So what control will userspace have over programming the actually =20 > MSI > > > vectors on PAMU? > > > > Not sure what you mean -- PAMU doesn't get explicitly involved in > > MSIs. It's just another 4K page mapping (per relevant MSI bank). =20 > If > > you want isolation, you need to make sure that an MSI group is only > > used by one VFIO group, and that you're on a chip that has alias =20 > pages > > with just one MSI bank register each (newer chips do, but the first > > chip to have a PAMU didn't). >=20 > How does a user figure this out? The user's involvement could be limited to setting a policy knob of =20 whether that degree of isolation is required (if required and =20 unavailable, all devices using an MSI bank would be forced into the =20 same group). We'd need to do something with MSI allocation so that we =20 avoid using an MSI bank with more than one IOMMU group where possible. =20 I'm not sure about the details yet, or how practical this is. There =20 might need to be some MSI bank assignment done as part of the VFIO =20 device binding process, if there are going to be more VFIO groups than =20 there are MSI banks (reserving one bank for host use). -Scott=