From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Xu Subject: Re: [RFC PATCH 03/20] intel_iommu: add "svm" option Date: Mon, 8 May 2017 19:20:34 +0800 Message-ID: <20170508112034.GE2820@pxdev.xzpeter.org> References: <1493201210-14357-1-git-send-email-yi.l.liu@linux.intel.com> <1493201210-14357-4-git-send-email-yi.l.liu@linux.intel.com> <20170427105317.GE1542@pxdev.xzpeter.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: "Liu, Yi L" Cc: "Lan, Tianyu" , "Liu, Yi L" , "Tian, Kevin" , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "jasowang-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org" , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , "qemu-devel-qX2TKyscuCcdnm+yROfE0A@public.gmane.org" , "Pan, Jacob jun" List-Id: iommu@lists.linux-foundation.org T24gTW9uLCBNYXkgMDgsIDIwMTcgYXQgMTA6Mzg6MDlBTSArMDAwMCwgTGl1LCBZaSBMIHdyb3Rl Ogo+IE9uIFRodSwgMjcgQXByIDIwMTcgMTg6NTM6MTcgKzA4MDAKPiBQZXRlciBYdSA8cGV0ZXJ4 QHJlZGhhdC5jb20+IHdyb3RlOgo+IAo+ID4gT24gV2VkLCBBcHIgMjYsIDIwMTcgYXQgMDY6MDY6 MzNQTSArMDgwMCwgTGl1LCBZaSBMIHdyb3RlOgo+ID4gPiBFeHBvc2UgIlNoYXJlZCBWaXJ0dWFs IE1lbW9yeSIgdG8gZ3Vlc3QgYnkgdXNpbmcgInN2bSIgb3B0aW9uLgo+ID4gPiBBbHNvIHVzZSAi c3ZtIiB0byBleHBvc2UgU1ZNIHJlbGF0ZWQgY2FwYWJpbGl0aWVzIHRvIGd1ZXN0Lgo+ID4gPiBl LmcuICItZGV2aWNlIGludGVsLWlvbW11LCBzdm09b24iCj4gPiA+Cj4gPiA+IFNpZ25lZC1vZmYt Ynk6IExpdSwgWWkgTCA8eWkubC5saXVAbGludXguaW50ZWwuY29tPgo+ID4gPiAtLS0KPiA+ID4g IGh3L2kzODYvaW50ZWxfaW9tbXUuYyAgICAgICAgICB8IDEwICsrKysrKysrKysKPiA+ID4gIGh3 L2kzODYvaW50ZWxfaW9tbXVfaW50ZXJuYWwuaCB8ICA1ICsrKysrCj4gPiA+IGluY2x1ZGUvaHcv aTM4Ni9pbnRlbF9pb21tdS5oICB8ICAxICsKPiA+ID4gIDMgZmlsZXMgY2hhbmdlZCwgMTYgaW5z ZXJ0aW9ucygrKQo+ID4gPgo+ID4gPiBkaWZmIC0tZ2l0IGEvaHcvaTM4Ni9pbnRlbF9pb21tdS5j IGIvaHcvaTM4Ni9pbnRlbF9pb21tdS5jIGluZGV4Cj4gPiA+IGJmOThmYTUuLmJhMWU3ZWIgMTAw NjQ0Cj4gPiA+IC0tLSBhL2h3L2kzODYvaW50ZWxfaW9tbXUuYwo+ID4gPiArKysgYi9ody9pMzg2 L2ludGVsX2lvbW11LmMKPiA+ID4gQEAgLTI0NTMsNiArMjQ1Myw3IEBAIHN0YXRpYyBQcm9wZXJ0 eSB2dGRfcHJvcGVydGllc1tdID0gewo+ID4gPiAgICAgIERFRklORV9QUk9QX0JPT0woIngtYnVn Z3ktZWltIiwgSW50ZWxJT01NVVN0YXRlLCBidWdneV9laW0sIGZhbHNlKSwKPiA+ID4gICAgICBE RUZJTkVfUFJPUF9CT09MKCJjYWNoaW5nLW1vZGUiLCBJbnRlbElPTU1VU3RhdGUsIGNhY2hpbmdf bW9kZSwKPiA+IEZBTFNFKSwKPiA+ID4gICAgICBERUZJTkVfUFJPUF9CT09MKCJlY3MiLCBJbnRl bElPTU1VU3RhdGUsIGVjcywgRkFMU0UpLAo+ID4gPiArICAgIERFRklORV9QUk9QX0JPT0woInN2 bSIsIEludGVsSU9NTVVTdGF0ZSwgc3ZtLCBGQUxTRSksCj4gPiA+ICAgICAgREVGSU5FX1BST1Bf RU5EX09GX0xJU1QoKSwKPiA+ID4gIH07Cj4gPiA+Cj4gPiA+IEBAIC0yOTczLDYgKzI5NzQsMTUg QEAgc3RhdGljIHZvaWQgdnRkX2luaXQoSW50ZWxJT01NVVN0YXRlICpzKQo+ID4gPiAgICAgICAg ICBzLT5lY2FwIHw9IFZURF9FQ0FQX0VDUzsKPiA+ID4gICAgICB9Cj4gPiA+Cj4gPiA+ICsgICAg aWYgKHMtPnN2bSkgewo+ID4gPiArICAgICAgICBpZiAoIXMtPmVjcyB8fCAheDg2X2lvbW11LT5w dF9zdXBwb3J0ZWQgfHwgIXMtPmNhY2hpbmdfbW9kZSkgewo+ID4gPiArICAgICAgICAgICAgZXJy b3JfcmVwb3J0KCJOZWVkIHRvIHNldCBlY3MsIHB0LCBjYWNoaW5nLW1vZGUgZm9yIHN2bSIpOwo+ ID4gPiArICAgICAgICAgICAgZXhpdCgxKTsKPiA+ID4gKyAgICAgICAgfQo+ID4gPiArICAgICAg ICBzLT5jYXAgfD0gVlREX0NBUF9EV0QgfCBWVERfQ0FQX0RSRDsKPiA+ID4gKyAgICAgICAgcy0+ ZWNhcCB8PSBWVERfRUNBUF9QUlMgfCBWVERfRUNBUF9QVFMgfCBWVERfRUNBUF9QQVNJRDI4Owo+ ID4gPiArICAgIH0KPiA+ID4gKwo+ID4gPiAgICAgIGlmIChzLT5jYWNoaW5nX21vZGUpIHsKPiA+ ID4gICAgICAgICAgcy0+Y2FwIHw9IFZURF9DQVBfQ007Cj4gPiA+ICAgICAgfQo+ID4gPiBkaWZm IC0tZ2l0IGEvaHcvaTM4Ni9pbnRlbF9pb21tdV9pbnRlcm5hbC5oCj4gPiA+IGIvaHcvaTM4Ni9p bnRlbF9pb21tdV9pbnRlcm5hbC5oIGluZGV4IDcxYTFjMWUuLmYyYTdkMTIgMTAwNjQ0Cj4gPiA+ IC0tLSBhL2h3L2kzODYvaW50ZWxfaW9tbXVfaW50ZXJuYWwuaAo+ID4gPiArKysgYi9ody9pMzg2 L2ludGVsX2lvbW11X2ludGVybmFsLmgKPiA+ID4gQEAgLTE5MSw2ICsxOTEsOSBAQAo+ID4gPiAg I2RlZmluZSBWVERfRUNBUF9QVCAgICAgICAgICAgICAgICAgKDFVTEwgPDwgNikKPiA+ID4gICNk ZWZpbmUgVlREX0VDQVBfTUhNViAgICAgICAgICAgICAgICgxNVVMTCA8PCAyMCkKPiA+ID4gICNk ZWZpbmUgVlREX0VDQVBfRUNTICAgICAgICAgICAgICAgICgxVUxMIDw8IDI0KQo+ID4gPiArI2Rl ZmluZSBWVERfRUNBUF9QQVNJRDI4ICAgICAgICAgICAgKDFVTEwgPDwgMjgpCj4gPiAKPiA+IENv dWxkIEkgYXNrIHdoYXQncyB0aGlzIGJpdD8gT24gbXkgc3BlYywgaXQgc2F5cyB0aGlzIGJpdCBp cyByZXNlcnZlZCBhbmQgZGVmdW5jdCAoc3BlYwo+ID4gdmVyc2lvbjogSnVuZSAyMDE2KS4KPiAK PiBBcyBBc2hvayBjb25maXJtZWQsIHllcyBpdCBzaG91bGQgYmUgYml0IDQwLiB3b3VsZCB1cGRh dGUgaXQuCgpPay4KCj4gCj4gPiA+ICsjZGVmaW5lIFZURF9FQ0FQX1BSUyAgICAgICAgICAgICAg ICAoMVVMTCA8PCAyOSkKPiA+ID4gKyNkZWZpbmUgVlREX0VDQVBfUFRTICAgICAgICAgICAgICAg ICgweGVVTEwgPDwgMzUpCj4gPiAKPiA+IFdvdWxkIGl0IGJldHRlciB3ZSBhdm9pZCB1c2luZyAw eGUgaGVyZSwgb3IgYXQgbGVhc3QgYWRkIHNvbWUgY29tbWVudD8KPiAKPiBGb3IgdGhpcyB2YWx1 ZSwgaXQgbXVzdCBiZSBubyBtb3JlIHRoYW4gdGhlIGJpdHMgaG9zdCBzdXBwb3J0cy4gU28gaXQg bWF5IGJlCj4gYmV0dGVyIHRvIGhhdmUgYSBkZWZhdWx0IHZhbHVlIGFuZCBtZWFud2hpbGUgZXhw b3NlIGFuIG9wdGlvbiB0byBsZXQgdXNlcgo+IHNldCBpdC4gaG93IGFib3V0IHlvdXIgb3Bpbmlv bj8KCkkgdGhpbmsgYSBtb3JlIGltcG9ydGFudCBwb2ludCBpcyB0aGF0IHdlIG5lZWQgdG8gbWFr ZSBzdXJlIHRoaXMgdmFsdWUKaXMgbm8gbGFyZ2VyIHRoYW4gaGFyZHdhcmUgc3VwcG9ydD8gU2lu Y2UgeW91IGFyZSBhbHNvIHdvcmtpbmcgb24gdGhlCnZmaW8gaW50ZXJmYWNlIGZvciB2aXJ0LXN2 bS4uLiB3b3VsZCBpdCBiZSBwb3NzaWJsZSB0aGF0IHdlIGNhbiB0YWxrCnRvIGtlcm5lbCBpbiBz b21lIHdheSBzbyB0aGF0IHdlIGNhbiBrbm93IHRoZSBzdXBwb3J0ZWQgcGFzaWQgc2l6ZSBpbgpo b3N0IElPTU1VPyBTbyB0aGF0IHdoZW4gZ3Vlc3Qgc3BlY2lmaWVzIHNvbWV0aGluZyBiaWdnZXIs IHdlIGNhbiBzdG9wCnRoZSB1c2VyLgoKSSBkb24ndCBrbm93IHRoZSBwcmFjdGljYWwgdmFsdWUg Zm9yIHRoaXMgZmllbGQsIGlmIGl0J3Mgc3RhdGljCmVub3VnaCwgSSB0aGluayBpdCdzIGFsc28g b2theSB3ZSBtYWtlIGl0IHN0YXRpYyBoZXJlIGFzIHdlbGwuIEJ1dAphZ2FpbiwgSSB3b3VsZCBw cmVmZXIgYXQgbGVhc3Qgc29tZSBjb21tZW50LCBsaWtlOgoKICAvKiBWYWx1ZSBOIGluZGljYXRl cyBQQVNJRCBmaWVsZCBvZiBOKzEgYml0cywgaGVyZSAweGUgc3RhbmRzIGZvci4uICovCgo+IAo+ ID4gCj4gPiA+Cj4gPiA+ICAvKiBDQVBfUkVHICovCj4gPiA+ICAvKiAob2Zmc2V0ID4+IDQpIDw8 IDI0ICovCj4gPiA+IEBAIC0yMDcsNiArMjEwLDggQEAKPiA+ID4gICNkZWZpbmUgVlREX0NBUF9Q U0kgICAgICAgICAgICAgICAgICgxVUxMIDw8IDM5KQo+ID4gPiAgI2RlZmluZSBWVERfQ0FQX1NM TFBTICAgICAgICAgICAgICAgKCgxVUxMIDw8IDM0KSB8ICgxVUxMIDw8IDM1KSkKPiA+ID4gICNk ZWZpbmUgVlREX0NBUF9DTSAgICAgICAgICAgICAgICAgICgxVUxMIDw8IDcpCj4gPiA+ICsjZGVm aW5lIFZURF9DQVBfRFdEICAgICAgICAgICAgICAgICAoMVVMTCA8PCA1NCkKPiA+ID4gKyNkZWZp bmUgVlREX0NBUF9EUkQgICAgICAgICAgICAgICAgICgxVUxMIDw8IDU1KQo+ID4gCj4gPiBKdXN0 IHRvIGNvbmZpcm06IGFmdGVyIHRoaXMgc2VyaWVzLCB3ZSBzaG91bGQgc3VwcG9ydCBkcmFpbiBy ZWFkL3dyaXRlIHRoZW4sIHJpZ2h0Pwo+IAo+IEkgaGF2ZW7igJl0IGRvbmUgc3BlY2lhbCBwcm9j ZXNzIGFnYWluc3QgaXQgaW4gSU9NTVUgZW11bGF0b3IuIEl0J3Mgc2V0IHRvIGtlZXAKPiBjb25z aXN0ZW5jZSB3aXRoIFZULWQgc3BlYyBzaW5jZSBEV0QgYW5kIERSVyBpcyByZXF1aXJlZCBjYXBh YmlsaXR5IHdoZW4KPiBQQVNJRCBpdCByZXBvcnRlZCBhcyBTZXQuIEhvd2V2ZXIsIEkgdGhpbmsg aXQgc2hvdWxkIGJlIGZpbmUgaWYgZ3Vlc3QgaXNzdWUgUUkKPiB3aXRoIGRyYWluIHJlYWQvd3Jp dGUgc2V0IGluIHRoZSBkZXNjcmlwdG9yLiBIb3N0IHNob3VsZCBiZSBhYmxlIHRvIHByb2Nlc3Mg aXQuCgpJIHNlZS4gSUlVQyB0aGUgcG9pbnQgaGVyZSBpcyB3ZSBuZWVkIHRvIGRlbGl2ZXIgdGhl c2UgcmVxdWVzdHMgdG8KaG9zdCBJT01NVSwgYW5kIEkgZ3Vlc3Mgd2UgbmVlZCB0byBiZSBhYmxl IHRvIGRvIHRoaXMgaW4gYSBzeW5jaHJvbm91cwp3YXkgYXMgd2VsbC4KClRoYW5rcywKCi0tIApQ ZXRlciBYdQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpp b21tdSBtYWlsaW5nIGxpc3QKaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6 Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vaW9tbXU= From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45892) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d7gid-0004zZ-2M for qemu-devel@nongnu.org; Mon, 08 May 2017 07:20:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d7giZ-0006gW-Rw for qemu-devel@nongnu.org; Mon, 08 May 2017 07:20:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55284) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d7giZ-0006gQ-In for qemu-devel@nongnu.org; Mon, 08 May 2017 07:20:43 -0400 Date: Mon, 8 May 2017 19:20:34 +0800 From: Peter Xu Message-ID: <20170508112034.GE2820@pxdev.xzpeter.org> References: <1493201210-14357-1-git-send-email-yi.l.liu@linux.intel.com> <1493201210-14357-4-git-send-email-yi.l.liu@linux.intel.com> <20170427105317.GE1542@pxdev.xzpeter.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [RFC PATCH 03/20] intel_iommu: add "svm" option List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Liu, Yi L" Cc: "Liu, Yi L" , "qemu-devel@nongnu.org" , "alex.williamson@redhat.com" , "kvm@vger.kernel.org" , "jasowang@redhat.com" , "iommu@lists.linux-foundation.org" , "Tian, Kevin" , "Raj, Ashok" , "Pan, Jacob jun" , "Lan, Tianyu" , "jean-philippe.brucker@arm.com" On Mon, May 08, 2017 at 10:38:09AM +0000, Liu, Yi L wrote: > On Thu, 27 Apr 2017 18:53:17 +0800 > Peter Xu wrote: >=20 > > On Wed, Apr 26, 2017 at 06:06:33PM +0800, Liu, Yi L wrote: > > > Expose "Shared Virtual Memory" to guest by using "svm" option. > > > Also use "svm" to expose SVM related capabilities to guest. > > > e.g. "-device intel-iommu, svm=3Don" > > > > > > Signed-off-by: Liu, Yi L > > > --- > > > hw/i386/intel_iommu.c | 10 ++++++++++ > > > hw/i386/intel_iommu_internal.h | 5 +++++ > > > include/hw/i386/intel_iommu.h | 1 + > > > 3 files changed, 16 insertions(+) > > > > > > diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index > > > bf98fa5..ba1e7eb 100644 > > > --- a/hw/i386/intel_iommu.c > > > +++ b/hw/i386/intel_iommu.c > > > @@ -2453,6 +2453,7 @@ static Property vtd_properties[] =3D { > > > DEFINE_PROP_BOOL("x-buggy-eim", IntelIOMMUState, buggy_eim, fa= lse), > > > DEFINE_PROP_BOOL("caching-mode", IntelIOMMUState, caching_mode= , > > FALSE), > > > DEFINE_PROP_BOOL("ecs", IntelIOMMUState, ecs, FALSE), > > > + DEFINE_PROP_BOOL("svm", IntelIOMMUState, svm, FALSE), > > > DEFINE_PROP_END_OF_LIST(), > > > }; > > > > > > @@ -2973,6 +2974,15 @@ static void vtd_init(IntelIOMMUState *s) > > > s->ecap |=3D VTD_ECAP_ECS; > > > } > > > > > > + if (s->svm) { > > > + if (!s->ecs || !x86_iommu->pt_supported || !s->caching_mod= e) { > > > + error_report("Need to set ecs, pt, caching-mode for sv= m"); > > > + exit(1); > > > + } > > > + s->cap |=3D VTD_CAP_DWD | VTD_CAP_DRD; > > > + s->ecap |=3D VTD_ECAP_PRS | VTD_ECAP_PTS | VTD_ECAP_PASID2= 8; > > > + } > > > + > > > if (s->caching_mode) { > > > s->cap |=3D VTD_CAP_CM; > > > } > > > diff --git a/hw/i386/intel_iommu_internal.h > > > b/hw/i386/intel_iommu_internal.h index 71a1c1e..f2a7d12 100644 > > > --- a/hw/i386/intel_iommu_internal.h > > > +++ b/hw/i386/intel_iommu_internal.h > > > @@ -191,6 +191,9 @@ > > > #define VTD_ECAP_PT (1ULL << 6) > > > #define VTD_ECAP_MHMV (15ULL << 20) > > > #define VTD_ECAP_ECS (1ULL << 24) > > > +#define VTD_ECAP_PASID28 (1ULL << 28) > >=20 > > Could I ask what's this bit? On my spec, it says this bit is reserved= and defunct (spec > > version: June 2016). >=20 > As Ashok confirmed, yes it should be bit 40. would update it. Ok. >=20 > > > +#define VTD_ECAP_PRS (1ULL << 29) > > > +#define VTD_ECAP_PTS (0xeULL << 35) > >=20 > > Would it better we avoid using 0xe here, or at least add some comment= ? >=20 > For this value, it must be no more than the bits host supports. So it m= ay be > better to have a default value and meanwhile expose an option to let us= er > set it. how about your opinion? I think a more important point is that we need to make sure this value is no larger than hardware support? Since you are also working on the vfio interface for virt-svm... would it be possible that we can talk to kernel in some way so that we can know the supported pasid size in host IOMMU? So that when guest specifies something bigger, we can stop the user. I don't know the practical value for this field, if it's static enough, I think it's also okay we make it static here as well. But again, I would prefer at least some comment, like: /* Value N indicates PASID field of N+1 bits, here 0xe stands for.. */ >=20 > >=20 > > > > > > /* CAP_REG */ > > > /* (offset >> 4) << 24 */ > > > @@ -207,6 +210,8 @@ > > > #define VTD_CAP_PSI (1ULL << 39) > > > #define VTD_CAP_SLLPS ((1ULL << 34) | (1ULL << 35)) > > > #define VTD_CAP_CM (1ULL << 7) > > > +#define VTD_CAP_DWD (1ULL << 54) > > > +#define VTD_CAP_DRD (1ULL << 55) > >=20 > > Just to confirm: after this series, we should support drain read/writ= e then, right? >=20 > I haven=E2=80=99t done special process against it in IOMMU emulator. It= 's set to keep > consistence with VT-d spec since DWD and DRW is required capability whe= n > PASID it reported as Set. However, I think it should be fine if guest i= ssue QI > with drain read/write set in the descriptor. Host should be able to pro= cess it. I see. IIUC the point here is we need to deliver these requests to host IOMMU, and I guess we need to be able to do this in a synchronous way as well. Thanks, --=20 Peter Xu