From mboxrd@z Thu Jan 1 00:00:00 1970 From: Will Deacon Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP Date: Fri, 27 Jun 2014 09:47:22 +0100 Message-ID: <20140627084722.GB26276@arm.com> References: <20140616151329.GQ16758@arm.com> <20140616152157.GB31771@8bytes.org> <20140616152526.GR16758@arm.com> <20140616153832.GC31771@8bytes.org> <9353770066894e85809e1e443b71d1cd@BY2PR07MB203.namprd07.prod.outlook.com> <748dccbbd3284521af4659ccbbb11453@BY2PR07MB203.namprd07.prod.outlook.com> <1403809223.31091.137.camel@ul30vt.home> <1403811384.31091.151.camel@ul30vt.home> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <1403811384.31091.151.camel-85EaTFmN5p//9pzu0YdTqQ@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Alex Williamson Cc: "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , marc.zyngier-5wv7dgnIgG8@public.gmane.org, open list , "stuart.yoder-KZfg59tc24xl57MIdRCFDg@public.gmane.org" , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , "Chalamarla, Tirumalesh" , "tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org" , "kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org" , "moderated list:ARM SMMU DRIVER" List-Id: iommu@lists.linux-foundation.org T24gVGh1LCBKdW4gMjYsIDIwMTQgYXQgMDg6MzY6MjRQTSArMDEwMCwgQWxleCBXaWxsaWFtc29u IHdyb3RlOgo+IE9uIFRodSwgMjAxNC0wNi0yNiBhdCAxOToxMCArMDAwMCwgQ2hhbGFtYXJsYSwg VGlydW1hbGVzaCB3cm90ZToKPiA+IFRoYW5rcyBmb3IgdGhlIGNsYXJpZmljYXRpb24gQWxleCwg VGhhdOKAmXMgZXhhY3RseSBteSBwb2ludCwgd2h5IGFyZSB3ZQo+ID4gcmVseWluZyBvbiAgUUVN VSBvciBzb21ldGhpbmcgZWxzZSB0byBlbXVsYXRlIHRoZSBNU0kgc3BhY2Ugd2hlbiB3ZSBjYW4K PiA+IGRpcmVjdGx5IGdpdmUgYWNjZXNzIHRvIGRldmljZXMgdXNpbmcgSVRTIChvZiBjb3Vyc2Ug d2l0aCBhIHNtYWxsCj4gPiBlbXVsYXRpb24gY29kZSkuICBUaGlzIHdheSB3ZSBhcmUgYWxzbyBi ZW5lZml0ZWQgZnJvbSBhbGwgSVRTIHNlcnZpY2VzCj4gPiBsaWtlIFZDUFUgbWlncmF0aW9uIGV0 Yy4gIAo+IAo+IEkgaGF2ZSBubyBpZGVhIHdoYXQgSVRTIGlzLgoKSVRTIGlzIHRoZSBNU0kgZG9v cmJlbGwgZm9yIEdJQ3YzIChBUk0ncyBsYXRlc3QgaW50ZXJydXB0IGNvbnRyb2xsZXIpLgoKSSBh Z3JlZSB0aGF0IHdlIHdpbGwgbmVlZCBhbiBJVFMgZW11bGF0aW9uIGlmIHdlIHdhbnQgdG8gdXNl IE1TSXMgaW4gdGhlCmd1ZXN0LCBhbmQgSSBiZWxpZXZlIHRoYXQgTWFyYyAoQ0MnZCkgaGFkIGFs cmVhZHkgc3RhcnRlZCB0aGlua2luZyBhYm91dAp0aGF0LgoKPiA+IFdoYXQgYWJvdXQgbm9uIFFF TVUgVkZJTyB1c2VycywgZm9yIGV4YW1wbGUsIGlmIEkgd2FudGVkIHRvIHVzZSBWRklPIHRvCj4g PiBhc3NpZ24gYSBkZXZpY2UgdG8gYSB1c2VyIHByb2Nlc3MgSSBkb24ndCBuZWVkIHRvIGRlcGVu ZCBvbiBRRU1VLiAgIEkKPiA+IHRob3VnaHQgdGhpcyBpcyBvbmUgb2YgdGhlIG1haW4gZ29hbHMg b2YgdmZpbyB0byBtYWtlIGl0IGluZGVwZW5kZW50IG9mCj4gPiBoeXBlcnZpc29ycy4gICAgIAo+ IAo+IFdoZXJlIGRpZCBRRU1VIGJlY29tZSBhIHJlcXVpcmVtZW50PyAgTWF5YmUgSSdtIG1pc3Np bmcgc29tZXRoaW5nIGluIHRoZQo+IEFSTSBwYXJ0IG9mIHRoZSBjb252ZXJzYXRpb24gdGhhdCBn b3QgY2hvcHBlZCBvZmYsIGJ1dCB0aGlzIGlzIGV4YWN0bHkKPiB3aHkgd2UgaGF2ZSB0aGUgVkZJ Ty9RRU1VIHNwbGl0IHRoYXQgd2UgZG8uICBWRklPIHByb3ZpZGVzIGJhc2ljCj4gdmlydHVhbGl6 YXRpb24gZm9yIGNvbmZpZyBzcGFjZSBhbmQgcmVzdHJpY3RzIGFjY2VzcyB0byBvdGhlciBhcmVh cyB0aGF0Cj4gdXNlcnMgc2hvdWxkbid0IGJlIGFsbG93ZWQgdG8gY2hhbmdlLiAgUUVNVSBpcyBq dXN0IG9uZSBleGFtcGxlIG9mIGEKPiB1c2Vyc3BhY2UgVkZJTyBkcml2ZXIuICBRRU1VIHRha2Vz IHRoZSBkZWNvbXBvc2VkIGRldmljZSBleHBvc2VkIHRocm91Z2gKPiB0aGUgVkZJTyBBQkkgYW5k IHJlLWNyZWF0ZXMgYSBQQ0kgZGV2aWNlIG91dCBvZiBpdC4gIFZGSU8gaXRzZWxmIGhhcyBubwo+ IGRlcGVuZGVuY3kgb24gUUVNVS4gIFRoYW5rcywKCkkgYWxzbyBkb24ndCB1bmRlcnN0YW5kIHRo ZSBRRU1VIHBhcnQgaGVyZS4gVGhlIE1TSSBlbXVsYXRpb24gd291bGQgYmUgaW4KdGhlIGtlcm5l bCwganVzdCBsaWtlIHRoZSBHSUN2MiBlbXVsYXRpb24gdGhhdCB3ZSBhbHJlYWR5IGhhdmUuIEZv cgp1c2Vyc3BhY2UgZHJpdmVycywgd291bGRuJ3QgeW91IGp1c3QgdXNlIGV2ZW50ZmQgcmF0aGVy IHRoYW4gYm90aGVyIHdpdGgKZW11bGF0aW5nIE1TSXM/CgpGaW5hbGx5LCB0aGUgaW50ZXJydXB0 IHJlbWFwcGluZyBwYXJ0IGlzIGFib3V0IHRoZSBTTU1VIHByZXZlbnRpbmcgTVNJCndyaXRlcyB0 byBhcmJpdHJhcnkgcG9ydGlvbnMgb2YgdGhlIGhvc3QgYWRkcmVzcyBzcGFjZS4gVGhlIElUUyBp cyBhYm91dApyb3V0aW5nIGludGVycnVwdHMgdG8gQ1BVcy4KCldpbGwKX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11 QGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9u Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lvbW11 From mboxrd@z Thu Jan 1 00:00:00 1970 From: will.deacon@arm.com (Will Deacon) Date: Fri, 27 Jun 2014 09:47:22 +0100 Subject: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP In-Reply-To: <1403811384.31091.151.camel@ul30vt.home> References: <20140616151329.GQ16758@arm.com> <20140616152157.GB31771@8bytes.org> <20140616152526.GR16758@arm.com> <20140616153832.GC31771@8bytes.org> <9353770066894e85809e1e443b71d1cd@BY2PR07MB203.namprd07.prod.outlook.com> <748dccbbd3284521af4659ccbbb11453@BY2PR07MB203.namprd07.prod.outlook.com> <1403809223.31091.137.camel@ul30vt.home> <1403811384.31091.151.camel@ul30vt.home> Message-ID: <20140627084722.GB26276@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Thu, Jun 26, 2014 at 08:36:24PM +0100, Alex Williamson wrote: > On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote: > > Thanks for the clarification Alex, That?s exactly my point, why are we > > relying on QEMU or something else to emulate the MSI space when we can > > directly give access to devices using ITS (of course with a small > > emulation code). This way we are also benefited from all ITS services > > like VCPU migration etc. > > I have no idea what ITS is. ITS is the MSI doorbell for GICv3 (ARM's latest interrupt controller). I agree that we will need an ITS emulation if we want to use MSIs in the guest, and I believe that Marc (CC'd) had already started thinking about that. > > What about non QEMU VFIO users, for example, if I wanted to use VFIO to > > assign a device to a user process I don't need to depend on QEMU. I > > thought this is one of the main goals of vfio to make it independent of > > hypervisors. > > Where did QEMU become a requirement? Maybe I'm missing something in the > ARM part of the conversation that got chopped off, but this is exactly > why we have the VFIO/QEMU split that we do. VFIO provides basic > virtualization for config space and restricts access to other areas that > users shouldn't be allowed to change. QEMU is just one example of a > userspace VFIO driver. QEMU takes the decomposed device exposed through > the VFIO ABI and re-creates a PCI device out of it. VFIO itself has no > dependency on QEMU. Thanks, I also don't understand the QEMU part here. The MSI emulation would be in the kernel, just like the GICv2 emulation that we already have. For userspace drivers, wouldn't you just use eventfd rather than bother with emulating MSIs? Finally, the interrupt remapping part is about the SMMU preventing MSI writes to arbitrary portions of the host address space. The ITS is about routing interrupts to CPUs. Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753009AbaF0Iu0 (ORCPT ); Fri, 27 Jun 2014 04:50:26 -0400 Received: from cam-admin0.cambridge.arm.com ([217.140.96.50]:65305 "EHLO cam-admin0.cambridge.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750756AbaF0IuX (ORCPT ); Fri, 27 Jun 2014 04:50:23 -0400 Date: Fri, 27 Jun 2014 09:47:22 +0100 From: Will Deacon To: Alex Williamson Cc: "Chalamarla, Tirumalesh" , Joerg Roedel , "kvm@vger.kernel.org" , open list , "stuart.yoder@freescale.com" , "iommu@lists.linux-foundation.org" , "tech@virtualopensystems.com" , "kvmarm@lists.cs.columbia.edu" , "moderated list:ARM SMMU DRIVER" , marc.zyngier@arm.com Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP Message-ID: <20140627084722.GB26276@arm.com> References: <20140616151329.GQ16758@arm.com> <20140616152157.GB31771@8bytes.org> <20140616152526.GR16758@arm.com> <20140616153832.GC31771@8bytes.org> <9353770066894e85809e1e443b71d1cd@BY2PR07MB203.namprd07.prod.outlook.com> <748dccbbd3284521af4659ccbbb11453@BY2PR07MB203.namprd07.prod.outlook.com> <1403809223.31091.137.camel@ul30vt.home> <1403811384.31091.151.camel@ul30vt.home> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1403811384.31091.151.camel@ul30vt.home> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jun 26, 2014 at 08:36:24PM +0100, Alex Williamson wrote: > On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote: > > Thanks for the clarification Alex, That’s exactly my point, why are we > > relying on QEMU or something else to emulate the MSI space when we can > > directly give access to devices using ITS (of course with a small > > emulation code). This way we are also benefited from all ITS services > > like VCPU migration etc. > > I have no idea what ITS is. ITS is the MSI doorbell for GICv3 (ARM's latest interrupt controller). I agree that we will need an ITS emulation if we want to use MSIs in the guest, and I believe that Marc (CC'd) had already started thinking about that. > > What about non QEMU VFIO users, for example, if I wanted to use VFIO to > > assign a device to a user process I don't need to depend on QEMU. I > > thought this is one of the main goals of vfio to make it independent of > > hypervisors. > > Where did QEMU become a requirement? Maybe I'm missing something in the > ARM part of the conversation that got chopped off, but this is exactly > why we have the VFIO/QEMU split that we do. VFIO provides basic > virtualization for config space and restricts access to other areas that > users shouldn't be allowed to change. QEMU is just one example of a > userspace VFIO driver. QEMU takes the decomposed device exposed through > the VFIO ABI and re-creates a PCI device out of it. VFIO itself has no > dependency on QEMU. Thanks, I also don't understand the QEMU part here. The MSI emulation would be in the kernel, just like the GICv2 emulation that we already have. For userspace drivers, wouldn't you just use eventfd rather than bother with emulating MSIs? Finally, the interrupt remapping part is about the SMMU preventing MSI writes to arbitrary portions of the host address space. The ITS is about routing interrupts to CPUs. Will