From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jike Song Subject: Re: [Announcement] 2015-Q3 release of XenGT - a Mediated Graphics Passthrough Solution from Intel Date: Thu, 19 Nov 2015 15:22:56 +0800 Message-ID: <564D78D0.80904@intel.com> References: <53D215D3.50608@intel.com> <547FCAAD.2060406@intel.com> <54AF967B.3060503@intel.com> <5527CEC4.9080700@intel.com> <559B3E38.1080707@intel.com> <562F4311.9@intel.com> <1447870341.4697.92.camel@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by gabe.freedesktop.org (Postfix) with ESMTP id 84A5B6E8D1 for ; Wed, 18 Nov 2015 23:23:23 -0800 (PST) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: Alex Williamson Cc: "igvt-g@ml01.01.org" , "Reddy, Raghuveer" , "White, Michael L" , "Cowperthwaite, David J" , "intel-gfx@lists.freedesktop.org" , "Li, Susie" , "Dong, Eddie" , "linux-kernel@vger.kernel.org" , "xen-devel@lists.xen.org" , qemu-devel , "Zhou, Chao" , Paolo Bonzini , "Zhu, Libo" , "Wang, Hongbo" List-Id: intel-gfx@lists.freedesktop.org SGkgQWxleCwKT24gMTEvMTkvMjAxNSAxMjowNiBQTSwgVGlhbiwgS2V2aW4gd3JvdGU6Cj4+IEZy b206IEFsZXggV2lsbGlhbXNvbiBbbWFpbHRvOmFsZXgud2lsbGlhbXNvbkByZWRoYXQuY29tXQo+ PiBTZW50OiBUaHVyc2RheSwgTm92ZW1iZXIgMTksIDIwMTUgMjoxMiBBTQo+Pgo+PiBbY2MgK3Fl bXUtZGV2ZWwsICtwYW9sbywgK2dlcmRdCj4+Cj4+IE9uIFR1ZSwgMjAxNS0xMC0yNyBhdCAxNzoy NSArMDgwMCwgSmlrZSBTb25nIHdyb3RlOgo+Pj4ge3NuaXB9Cj4+Cj4+IEhpIQo+Pgo+PiBBdCBy ZWRoYXQgd2UndmUgYmVlbiB0aGlua2luZyBhYm91dCBob3cgdG8gc3VwcG9ydCB2R1BVcyBmcm9t IG11bHRpcGxlCj4+IHZlbmRvcnMgaW4gYSBjb21tb24gd2F5IHdpdGhpbiBRRU1VLiAgV2Ugd2Fu dCB0byBlbmFibGUgY29kZSBzaGFyaW5nCj4+IGJldHdlZW4gdmVuZG9ycyBhbmQgZ2l2ZSBuZXcg dmVuZG9ycyBhbiBlYXN5IHBhdGggdG8gYWRkIHRoZWlyIG93bgo+PiBzdXBwb3J0LiAgV2UgYWxz byBoYXZlIHRoZSBjb21wbGljYXRpb24gdGhhdCBub3QgYWxsIHZHUFUgdmVuZG9ycyBhcmUgYXMK Pj4gb3BlbiBzb3VyY2UgZnJpZW5kbHkgYXMgSW50ZWwsIHNvIGJlaW5nIGFibGUgdG8gYWJzdHJh Y3QgdGhlIGRldmljZQo+PiBtZWRpYXRpb24gYW5kIGFjY2VzcyBvdXRzaWRlIG9mIFFFTVUgaXMg YSBiaWcgYWR2YW50YWdlLgo+Pgo+PiBUaGUgcHJvcG9zYWwgSSdkIGxpa2UgdG8gbWFrZSBpcyB0 aGF0IGEgdkdQVSwgd2hldGhlciBpdCBpcyBmcm9tIEludGVsCj4+IG9yIGFub3RoZXIgdmVuZG9y LCBpcyBwcmVkb21pbmFudGx5IGEgUENJKGUpIGRldmljZS4gIFdlIGhhdmUgYW4KPj4gaW50ZXJm YWNlIGluIFFFTVUgYWxyZWFkeSBmb3IgZXhwb3NpbmcgYXJiaXRyYXJ5IFBDSSBkZXZpY2VzLCB2 ZmlvLXBjaS4KPj4gQ3VycmVudGx5IHZmaW8tcGNpIHVzZXMgdGhlIFZGSU8gQVBJIHRvIGludGVy YWN0IHdpdGggInBoeXNpY2FsIiBkZXZpY2VzCj4+IGFuZCBzeXN0ZW0gSU9NTVVzLiAgSSBoaWdo bGlnaHQgL3BoeXNpY2FsLyB0aGVyZSBiZWNhdXNlIHNvbWUgb2YgdGhlc2UKPj4gcGh5c2ljYWwg ZGV2aWNlcyBhcmUgU1ItSU9WIFZGcywgd2hpY2ggaXMgc29tZXdoYXQgb2YgYSBmdXp6eSBjb25j ZXB0LAo+PiBzb21ld2hlcmUgYmV0d2VlbiBmaXhlZCBoYXJkd2FyZSBhbmQgYSB2aXJ0dWFsIGRl dmljZSBpbXBsZW1lbnRlZCBpbgo+PiBzb2Z0d2FyZS4gIFRoYXQgc29mdHdhcmUganVzdCBoYXBw ZW5zIHRvIGJlIHJ1bm5pbmcgb24gdGhlIHBoeXNpY2FsCj4+IGVuZHBvaW50Lgo+Cj4gQWdyZWUu Cj4KPiBPbmUgY2xhcmlmaWNhdGlvbiBmb3IgcmVzdCBkaXNjdXNzaW9uLCBpcyB0aGF0IHdlJ3Jl IHRhbGtpbmcgYWJvdXQgR1ZULWcgdkdQVQo+IGhlcmUgd2hpY2ggaXMgYSBwdXJlIHNvZnR3YXJl IEdQVSB2aXJ0dWFsaXphdGlvbiB0ZWNobmlxdWUuIEdWVC1kIChub3RlCj4gc29tZSB1c2UgaW4g dGhlIHRleHQpIHJlZmVycyB0byBwYXNzaW5nIHRocm91Z2ggdGhlIHdob2xlIEdQVSBvciBhIHNw ZWNpZmljCj4gVkYuIEdWVC1kIGFscmVhZHkgZmFsbHMgaW50byBleGlzdGluZyBWRklPIEFQSXMg bmljZWx5ICh0aG91Z2ggc29tZSBvbi1nb2luZwo+IGVmZm9ydCB0byByZW1vdmUgSW50ZWwgc3Bl Y2lmaWMgcGxhdGZvcm0gc3RpY2tuZXNzIGZyb20gZ2Z4IGRyaXZlcikuIDotKQo+CgpIaSBBbGV4 LCB0aGFua3MgZm9yIHRoZSBkaXNjdXNzaW9uLgoKSW4gYWRkaXRpb24gdG8gS2V2aW4ncyByZXBs aWVzLCBJIGhhdmUgYSBoaWdoLWxldmVsIHF1ZXN0aW9uOiBjYW4gVkZJTwpiZSB1c2VkIGJ5IFFF TVUgZm9yIGJvdGggS1ZNIGFuZCBYZW4/CgotLQpUaGFua3MsCkppa2UKCiAgCj4+Cj4+IHZHUFVz IGFyZSBzaW1pbGFyLCB3aXRoIHRoZSB2aXJ0dWFsIGRldmljZSBjcmVhdGVkIGF0IGEgZGlmZmVy ZW50IHBvaW50LAo+PiBob3N0IHNvZnR3YXJlLiAgVGhleSBhbHNvIHJlbHkgb24gZGlmZmVyZW50 IElPTU1VIGNvbnN0cnVjdHMsIG1ha2luZyB1c2UKPj4gb2YgdGhlIE1NVSBjYXBhYmlsaXRpZXMg b2YgdGhlIEdQVSAoR1RUcyBhbmQgc3VjaCksIGJ1dCByZWFsbHkgaGF2aW5nCj4+IHNpbWlsYXIg cmVxdWlyZW1lbnRzLgo+Cj4gT25lIGltcG9ydGFudCBkaWZmZXJlbmNlIGJldHdlZW4gc3lzdGVt IElPTU1VIGFuZCBHUFUtTU1VIGhlcmUuCj4gU3lzdGVtIElPTU1VIGlzIHZlcnkgbXVjaCBhYm91 dCB0cmFuc2xhdGlvbiBmcm9tIGEgRE1BIHRhcmdldAo+IChJT1ZBIG9uIG5hdGl2ZSwgb3IgR1BB IGluIHZpcnR1YWxpemF0aW9uIGNhc2UpIHRvIEhQQS4gSG93ZXZlciBHUFUKPiBpbnRlcm5hbCBN TVVzIGlzIHRvIHRyYW5zbGF0ZSBmcm9tIEdyYXBoaWNzIE1lbW9yeSBBZGRyZXNzIChHTUEpCj4g dG8gRE1BIHRhcmdldCAoSFBBIGlmIHN5c3RlbSBJT01NVSBpcyBkaXNhYmxlZCwgb3IgSU9WQS9H UEEgaWYgc3lzdGVtCj4gSU9NTVUgaXMgZW5hYmxlZCkuIEdNQSBpcyBhbiBpbnRlcm5hbCBhZGRy IHNwYWNlIHdpdGhpbiBHUFUsIG5vdAo+IGV4cG9zZWQgdG8gUWVtdSBhbmQgZnVsbHkgbWFuYWdl ZCBieSBHVlQtZyBkZXZpY2UgbW9kZWwuIFNpbmNlIGl0J3MKPiBub3QgYSBzdGFuZGFyZCBQQ0kg ZGVmaW5lZCByZXNvdXJjZSwgd2UgZG9uJ3QgbmVlZCBhYnN0cmFjdCB0aGlzIGNhcGFiaWxpdHkK PiBpbiBWRklPIGludGVyZmFjZS4KPgo+Pgo+PiBUaGUgcHJvcG9zYWwgaXMgdGhlcmVmb3JlIHRo YXQgR1BVIHZlbmRvcnMgY2FuIGV4cG9zZSB2R1BVcyB0bwo+PiB1c2Vyc3BhY2UsIGFuZCB0aHVz IHRvIFFFTVUsIHVzaW5nIHRoZSBWRklPIEFQSS4gIEZvciBpbnN0YW5jZSwgdmZpbwo+PiBzdXBw b3J0cyBtb2R1bGFyIGJ1cyBkcml2ZXJzIGFuZCBJT01NVSBkcml2ZXJzLiAgQW4gaW50ZWwtdmZp by1ndnQtZAo+PiBtb2R1bGUgKG9yIGV4dGVuc2lvbiBvZiBpOTE1KSBjYW4gcmVnaXN0ZXIgYXMg YSB2ZmlvIGJ1cyBkcml2ZXIsIGNyZWF0ZQo+PiBhIHN0cnVjdCBkZXZpY2UgcGVyIHZHUFUsIGNy ZWF0ZSBhbiBJT01NVSBncm91cCBmb3IgdGhhdCBkZXZpY2UsIGFuZAo+PiByZWdpc3RlciB0aGF0 IGRldmljZSB3aXRoIHRoZSB2ZmlvLWNvcmUuICBTaW5jZSB3ZSBkb24ndCByZWx5IG9uIHRoZQo+ PiBzeXN0ZW0gSU9NTVUgZm9yIEdWVC1kIHZHUFUgYXNzaWdubWVudCwgYW5vdGhlciB2R1BVIHZl bmRvciBkcml2ZXIgKG9yCj4+IGV4dGVuc2lvbiBvZiB0aGUgc2FtZSBtb2R1bGUpIGNhbiByZWdp c3RlciBhICJ0eXBlMSIgY29tcGxpYW50IElPTU1VCj4+IGRyaXZlciBpbnRvIHZmaW8tY29yZS4g IEZyb20gdGhlIHBlcnNwZWN0aXZlIG9mIFFFTVUgdGhlbiwgYWxsIG9mIHRoZQo+PiBleGlzdGlu ZyB2ZmlvLXBjaSBjb2RlIGlzIHJlLXVzZWQsIFFFTVUgcmVtYWlucyBsYXJnZWx5IHVuYXdhcmUg b2YgYW55Cj4+IHNwZWNpZmljcyBvZiB0aGUgdkdQVSBiZWluZyBhc3NpZ25lZCwgYW5kIHRoZSBv bmx5IG5lY2Vzc2FyeSBjaGFuZ2Ugc28KPj4gZmFyIGlzIGhvdyBRRU1VIHRyYXZlcnNlcyBzeXNm cyB0byBmaW5kIHRoZSBkZXZpY2UgYW5kIHRodXMgdGhlIElPTU1VCj4+IGdyb3VwIGxlYWRpbmcg dG8gdGhlIHZmaW8gZ3JvdXAuCj4KPiBHVlQtZyByZXF1aXJlcyB0byBwaW4gZ3Vlc3QgbWVtb3J5 IGFuZCBxdWVyeSBHUEEtPkhQQSBpbmZvcm1hdGlvbiwKPiB1cG9uIHdoaWNoIHNoYWRvdyBHVFRz IHdpbGwgYmUgdXBkYXRlZCBhY2NvcmRpbmdseSBmcm9tIChHTUEtPkdQQSkKPiB0byAoR01BLT5I UEEpLiBTbyB5ZXMsIGhlcmUgYSBkdW1teSBvciBzaW1wbGUgInR5cGUxIiBjb21wbGlhbnQgSU9N TVUKPiBjYW4gYmUgaW50cm9kdWNlZCBqdXN0IGZvciB0aGlzIHJlcXVpcmVtZW50Lgo+Cj4gSG93 ZXZlciB0aGVyZSdzIG9uZSB0cmlja3kgcG9pbnQgd2hpY2ggSSdtIG5vdCBzdXJlIHdoZXRoZXIg b3ZlcmFsbAo+IFZGSU8gY29uY2VwdCB3aWxsIGJlIHZpb2xhdGVkLiBHVlQtZyBkb2Vzbid0IHJl cXVpcmUgc3lzdGVtIElPTU1VCj4gdG8gZnVuY3Rpb24sIGhvd2V2ZXIgaG9zdCBzeXN0ZW0gbWF5 IGVuYWJsZSBzeXN0ZW0gSU9NTVUganVzdCBmb3IKPiBoYXJkZW5pbmcgcHVycG9zZS4gVGhpcyBt ZWFucyB0d28tbGV2ZWwgdHJhbnNsYXRpb25zIGV4aXN0aW5nIChHTUEtPgo+IElPVkEtPkhQQSks IHNvIHRoZSBkdW1teSBJT01NVSBkcml2ZXIgaGFzIHRvIHJlcXVlc3Qgc3lzdGVtIElPTU1VCj4g ZHJpdmVyIHRvIGFsbG9jYXRlIElPVkEgZm9yIFZNcyBhbmQgdGhlbiBzZXR1cCBJT1ZBLT5IUEEg bWFwcGluZwo+IGluIElPTU1VIHBhZ2UgdGFibGUuIEluIHRoaXMgY2FzZSwgbXVsdGlwbGUgVk0n cyB0cmFuc2xhdGlvbnMgYXJlCj4gbXVsdGlwbGV4ZWQgaW4gb25lIElPTU1VIHBhZ2UgdGFibGUu Cj4KPiBXZSBtaWdodCBuZWVkIGNyZWF0ZSBzb21lIGdyb3VwL3N1Yi1ncm91cCBvciBwYXJlbnQv Y2hpbGQgY29uY2VwdHMKPiBhbW9uZyB0aG9zZSBJT01NVXMgZm9yIHRob3JvdWdoIHBlcm1pc3Np b24gY29udHJvbC4KPgo+Pgo+PiBUaGVyZSBhcmUgYSBmZXcgYXJlYXMgd2hlcmUgd2Uga25vdyB3 ZSdsbCBuZWVkIHRvIGV4dGVuZCB0aGUgVkZJTyBBUEkgdG8KPj4gbWFrZSB0aGlzIHdvcmssIGJ1 dCBpdCBzZWVtcyBsaWtlIHRoZXkgY2FuIGFsbCBiZSBkb25lIGdlbmVyaWNhbGx5LiAgT25lCj4+ IGlzIHRoYXQgUENJIEJBUnMgYXJlIGRlc2NyaWJlZCB0aHJvdWdoIHRoZSBWRklPIEFQSSBhcyBy ZWdpb25zIGFuZCBlYWNoCj4+IHJlZ2lvbiBoYXMgYSBzaW5nbGUgZmxhZyBkZXNjcmliaW5nIHdo ZXRoZXIgbW1hcCAoaWUuIGRpcmVjdCBtYXBwaW5nKSBvZgo+PiB0aGF0IHJlZ2lvbiBpcyBwb3Nz aWJsZS4gIFdlIGV4cGVjdCB0aGF0IHZHUFVzIGxpa2VseSBuZWVkIGZpbmVyCj4+IGdyYW51bGFy aXR5LCBlbmFibGluZyBzb21lIGFyZWFzIHdpdGhpbiBhIEJBUiB0byBiZSB0cmFwcGVkIGFuZCBm b3dhcmRlZAo+PiBhcyBhIHJlYWQgb3Igd3JpdGUgYWNjZXNzIGZvciB0aGUgdkdQVS12ZmlvLWRl dmljZSBtb2R1bGUgdG8gZW11bGF0ZSwKPj4gd2hpbGUgb3RoZXIgcmVnaW9ucywgbGlrZSBmcmFt ZWJ1ZmZlcnMgb3IgdGV4dHVyZSByZWdpb25zLCBhcmUgZGlyZWN0bHkKPj4gbWFwcGVkLiAgSSBo YXZlIHByb3RvdHlwZSBjb2RlIHRvIGVuYWJsZSB0aGlzIGFscmVhZHkuCj4KPiBZZXMgaW4gR1ZU LWcgb25lIEJBUiByZXNvdXJjZSBtaWdodCBiZSBwYXJ0aXRpb25lZCBhbW9uZyBtdWx0aXBsZSB2 R1BVcy4KPiBJZiBWRklPIGNhbiBzdXBwb3J0IHN1Y2ggcGFydGlhbCByZXNvdXJjZSBhc3NpZ25t ZW50LCBpdCdkIGJlIGdyZWF0LiBTaW1pbGFyCj4gcGFyZW50L2NoaWxkIGNvbmNlcHQgbWlnaHQg YWxzbyBiZSByZXF1aXJlZCBoZXJlLCBzbyBhbnkgcmVzb3VyY2UgZW51bWVyYXRlZAo+IG9uIGEg dkdQVSBzaG91bGRuJ3QgYnJlYWsgbGltaXRhdGlvbnMgZW5mb3JjZWQgb24gdGhlIHBoeXNpY2Fs IGRldmljZS4KPgo+IE9uZSB1bmlxdWUgcmVxdWlyZW1lbnQgZm9yIEdWVC1nIGhlcmUsIHRob3Vn aCwgaXMgdGhhdCB2R1BVIGRldmljZSBtb2RlbAo+IG5lZWQgdG8ga25vdyBndWVzdCBCQVIgY29u ZmlndXJhdGlvbiBmb3IgcHJvcGVyIGVtdWxhdGlvbiAoZS5nLiByZWdpc3Rlcgo+IElPIGVtdWxh dGlvbiBoYW5kbGVyIHRvIEtWTSkuIFNpbWlsYXIgaXMgYWJvdXQgZ3Vlc3QgTVNJIHZlY3RvciBm b3IgdmlydHVhbAo+IGludGVycnVwdCBpbmplY3Rpb24uIE5vdCBzdXJlIGhvdyB0aGlzIGNhbiBi ZSBmaXQgaW50byBjb21tb24gVkZJTyBtb2RlbC4KPiBEb2VzIFZGSU8gYWxsb3cgdmVuZG9yIHNw ZWNpZmljIGV4dGVuc2lvbiB0b2RheT8KPgo+Pgo+PiBBbm90aGVyIGFyZWEgaXMgdGhhdCB3ZSBy ZWFsbHkgZG9uJ3Qgd2FudCB0byBwcm9saWZlcmF0ZSBlYWNoIHZHUFUKPj4gbmVlZGluZyBhIG5l dyBJT01NVSB0eXBlIHdpdGhpbiB2ZmlvLiAgVGhlIGV4aXN0aW5nIHR5cGUxIElPTU1VIHByb3Zp ZGVzCj4+IHBvdGVudGlhbGx5IHRoZSBtb3N0IHNpbXBsZSBtYXBwaW5nIGFuZCB1bm1hcHBpbmcg aW50ZXJmYWNlIHBvc3NpYmxlLgo+PiBXZSdkIHRoZXJlZm9yZSBuZWVkIHRvIGFsbG93IG11bHRp cGxlICJ0eXBlMSIgSU9NTVUgZHJpdmVycyBmb3IgdmZpbywKPj4gbWFraW5nIHR5cGUxIGJlIG1v cmUgb2YgYW4gaW50ZXJmYWNlIHNwZWNpZmljYXRpb24gcmF0aGVyIHRoYW4gYSBzaW5nbGUKPj4g aW1wbGVtZW50YXRpb24uICBUaGlzIGlzIGEgdHJpdmlhbCBjaGFuZ2UgdG8gbWFrZSB3aXRoaW4g dmZpbyBhbmQgb25lCj4+IHRoYXQgSSBiZWxpZXZlIGlzIGNvbXBhdGlibGUgd2l0aCB0aGUgZXhp c3RpbmcgQVBJLiAgTm90ZSB0aGF0Cj4+IGltcGxlbWVudGluZyBhIHR5cGUxLWNvbXBsaWFudCB2 ZmlvIElPTU1VIGRvZXMgbm90IGltcGx5IHBpbm5pbmcgYW4KPj4gbWFwcGluZyBldmVyeSByZWdp c3RlcmVkIHBhZ2UuICBBIHZHUFUsIHdpdGggbWVkaWF0ZWQgZGV2aWNlIGFjY2VzcywgbWF5Cj4+ IHVzZSB0aGlzIG9ubHkgdG8gdHJhY2sgdGhlIGN1cnJlbnQgSFZBIHRvIEdQQSBtYXBwaW5ncyBm b3IgYSBWTS4gIE9ubHkKPj4gd2hlbiBhIERNQSBpcyBlbmFibGVkIGZvciB0aGUgdkdQVSBpbnN0 YW5jZSBpcyB0aGF0IEhWQSBwaW5uZWQgYW5kIGFuCj4+IEhQQSB0byBHUEEgdHJhbnNsYXRpb24g cHJvZ3JhbW1lZCBpbnRvIHRoZSBHUFUgTU1VLgo+Pgo+PiBBbm90aGVyIGFyZWEgb2YgZXh0ZW5z aW9uIGlzIGhvdyB0byBleHBvc2UgYSBmcmFtZWJ1ZmZlciB0byBRRU1VIGZvcgo+PiBzZWFtbGVz cyBpbnRlZ3JhdGlvbiBpbnRvIGEgU1BJQ0UvVk5DIGNoYW5uZWwuICBGb3IgdGhpcyBJIGJlbGll dmUgd2UKPj4gY291bGQgdXNlIGEgbmV3IHJlZ2lvbiwgbXVjaCBsaWtlIHdlJ3ZlIGRvbmUgdG8g ZXhwb3NlIFZHQSBhY2Nlc3MKPj4gdGhyb3VnaCBhIHZmaW8gZGV2aWNlIGZpbGUgZGVzY3JpcHRv ci4gIEFuIGFyZWEgd2l0aGluIHRoaXMgbmV3Cj4+IGZyYW1lYnVmZmVyIHJlZ2lvbiBjb3VsZCBi ZSBkaXJlY3RseSBtYXBwYWJsZSBpbiBRRU1VIHdoaWxlIGEKPj4gbm9uLW1hcHBhYmxlIHBhZ2Us IGF0IGEgc3RhbmRhcmQgbG9jYXRpb24gd2l0aCBzdGFuZGFyZGl6ZWQgZm9ybWF0LAo+PiBwcm92 aWRlcyBhIGRlc2NyaXB0aW9uIG9mIGZyYW1lYnVmZmVyIGFuZCBwb3RlbnRpYWxseSBldmVuIGEK Pj4gY29tbXVuaWNhdGlvbiBjaGFubmVsIHRvIHN5bmNocm9uaXplIGZyYW1lYnVmZmVyIGNhcHR1 cmVzLiAgVGhpcyB3b3VsZAo+PiBiZSBuZXcgY29kZSBmb3IgUUVNVSwgYnV0IHNvbWV0aGluZyB3 ZSBjb3VsZCBzaGFyZSBhbW9uZyBhbGwgdkdQVQo+PiBpbXBsZW1lbnRhdGlvbnMuCj4KPiBOb3cg R1ZULWcgYWxyZWFkeSBwcm92aWRlcyBhbiBpbnRlcmZhY2UgdG8gZGVjb2RlIGZyYW1lYnVmZmVy IGluZm9ybWF0aW9uLAo+IHcvIGFuIGFzc3VtcHRpb24gdGhhdCB0aGUgZnJhbWVidWZmZXIgd2ls bCBiZSBmdXJ0aGVyIGNvbXBvc2l0ZWQgaW50bwo+IE9wZW5HTCBBUElzLiBTbyB0aGUgZm9ybWF0 IGlzIGRlZmluZWQgYWNjb3JkaW5nIHRvIE9wZW5HTCBkZWZpbml0aW9uLgo+IERvZXMgdGhhdCBt ZWV0IFNQSUNFIHJlcXVpcmVtZW50Pwo+Cj4gQW5vdGhlciB0aGluZyB0byBiZSBhZGRlZC4gRnJh bWVidWZmZXJzIGFyZSBmcmVxdWVudGx5IHN3aXRjaGVkIGluCj4gcmVhbGl0eS4gU28gZWl0aGVy IFFlbXUgbmVlZHMgdG8gcG9sbCBvciBhIG5vdGlmaWNhdGlvbiBtZWNoYW5pc20gaXMgcmVxdWly ZWQuCj4gQW5kIHNpbmNlIGl0J3MgZHluYW1pYywgaGF2aW5nIGZyYW1lYnVmZmVyIHBhZ2UgZGly ZWN0bHkgZXhwb3NlZCBpbiB0aGUKPiBuZXcgcmVnaW9uIG1pZ2h0IGJlIHRyaWNreS4gV2UgY2Fu IGp1c3QgZXhwb3NlIGZyYW1lYnVmZmVyIGluZm9ybWF0aW9uCj4gKGluY2x1ZGluZyBiYXNlLCBm b3JtYXQsIGV0Yy4pIGFuZCBsZXQgUWVtdSB0byBtYXAgc2VwYXJhdGVseSBvdXQgb2YgVkZJTwo+ IGludGVyZmFjZS4KPgo+IEFuZC4uLiB0aGlzIHdvcmtzIGZpbmUgd2l0aCB2R1BVIG1vZGVsIHNp bmNlIHNvZnR3YXJlIGtub3dzIGFsbCB0aGUKPiBkZXRhaWwgYWJvdXQgZnJhbWVidWZmZXIuIEhv d2V2ZXIgaW4gcGFzcy10aHJvdWdoIGNhc2UsIHdobyBkbyB5b3UgZXhwZWN0Cj4gdG8gcHJvdmlk ZSB0aGF0IGluZm9ybWF0aW9uPyBJcyBpdCBPSyB0byBpbnRyb2R1Y2UgdkdQVSBzcGVjaWZpYyBB UElzIGluCj4gVkZJTz8KPgo+Pgo+PiBBbm90aGVyIG9idmlvdXMgYXJlYSB0byBiZSBzdGFuZGFy ZGl6ZWQgd291bGQgYmUgaG93IHRvIGRpc2NvdmVyLAo+PiBjcmVhdGUsIGFuZCBkZXN0cm95IHZH UFUgaW5zdGFuY2VzLiAgU1ItSU9WIGhhcyBhIHN0YW5kYXJkIG1lY2hhbmlzbSB0bwo+PiBjcmVh dGUgVkZzIGluIHN5c2ZzIGFuZCBJIHdvdWxkIHByb3Bvc2UgdGhhdCB2R1BVIHZlbmRvcnMgdHJ5 IHRvCj4+IHN0YW5kYXJkaXplIG9uIHNpbWlsYXIgaW50ZXJmYWNlcyB0byBlbmFibGUgbGlidmly dCB0byBlYXNpbHkgZGlzY292ZXIKPj4gdGhlIHZHUFUgY2FwYWJpbGl0aWVzIG9mIGEgZ2l2ZW4g R1BVIGFuZCBtYW5hZ2UgdGhlIGxpZmVjeWNsZSBvZiBhIHZHUFUKPj4gaW5zdGFuY2UuCj4KPiBO b3cgdGhlcmUgaXMgbm8gc3RhbmRhcmQuIFdlIGV4cG9zZSB2R1BVIGxpZmUtY3ljbGUgbWdtdC4g QVBJcyB0aHJvdWdoCj4gc3lzZnMgKHVuZGVyIGk5MTUgbm9kZSksIHdoaWNoIGlzIHZlcnkgSW50 ZWwgc3BlY2lmaWMuIEluIHJlYWxpdHkgZGlmZmVyZW50Cj4gdmVuZG9ycyBoYXZlIHF1aXRlIGRp ZmZlcmVudCBjYXBhYmlsaXRpZXMgZm9yIHRoZWlyIG93biB2R1BVcywgc28gbm90IHN1cmUKPiBo b3cgc3RhbmRhcmQgd2UgY2FuIGRlZmluZSBzdWNoIGEgbWVjaGFuaXNtLiBCdXQgdGhpcyBjb2Rl IHNob3VsZCBiZQo+IG1pbm9yIHRvIGJlIG1haW50YWluZWQgaW4gbGlidmlydC4KPgo+Pgo+PiBU aGlzIGlzIG9idmlvdXNseSBhIGxvdCB0byBkaWdlc3QsIGJ1dCBJJ2QgY2VydGFpbmx5IGJlIGlu dGVyZXN0ZWQgaW4KPj4gaGVhcmluZyBmZWVkYmFjayBvbiB0aGlzIHByb3Bvc2FsIGFzIHdlbGwg YXMgdHJ5IHRvIGNsYXJpZnkgYW55dGhpbmcKPj4gSSd2ZSBsZWZ0IG91dCBvciBtaXNyZXByZXNl bnRlZCBhYm92ZS4gIEFub3RoZXIgYmVuZWZpdCB0byB0aGlzCj4+IG1lY2hhbmlzbSBpcyB0aGF0 IGRpcmVjdCBHUFUgYXNzaWdubWVudCBhbmQgdkdQVSBhc3NpZ25tZW50IHVzZSB0aGUgc2FtZQo+ PiBjb2RlIHdpdGhpbiBRRU1VIGFuZCBzYW1lIEFQSSB0byB0aGUga2VybmVsLCB3aGljaCBzaG91 bGQgbWFrZSBkZWJ1Z2dpbmcKPj4gYW5kIGNvZGUgc3VwcG9ydCBiZXR3ZWVuIHRoZSB0d28gZWFz aWVyLiAgSSdkIHJlYWxseSBsaWtlIHRvIHN0YXJ0IGEKPj4gZGlzY3Vzc2lvbiBhcm91bmQgdGhp cyBwcm9wb3NhbCwgYW5kIG9mIGNvdXJzZSB0aGUgZmlyc3Qgb3BlbiBzb3VyY2UKPj4gaW1wbGVt ZW50YXRpb24gb2YgdGhpcyBzb3J0IG9mIG1vZGVsIHdpbGwgcmVhbGx5IGhlbHAgdG8gZHJpdmUg dGhlCj4+IGRpcmVjdGlvbiBpdCB0YWtlcy4gIFRoYW5rcyEKPj4KPgo+IFRoYW5rcyBmb3Igc3Rh cnRpbmcgdGhpcyBkaXNjdXNzaW9uLiBJbnRlbCB3aWxsIGRlZmluaXRlbHkgd29yayB3aXRoCj4g Y29tbXVuaXR5IG9uIHRoaXMgd29yay4gQmFzZWQgb24gZWFybGllciBjb21tZW50cywgSSdtIG5v dCBzdXJlCj4gd2hldGhlciB3ZSBjYW4gZXhhY3RseSBzYW1lIGNvZGUgZm9yIGRpcmVjdCBHUFUg YXNzaWdubWVudCBhbmQKPiB2R1BVIGFzc2lnbm1lbnQsIHNpbmNlIGV2ZW4gd2UgZXh0ZW5kIFZG SU8gc29tZSBpbnRlcmZhY2VzIG1pZ2h0Cj4gYmUgdkdQVSBzcGVjaWZpYy4gRG9lcyB0aGlzIHdh eSBzdGlsbCBhY2hpZXZlIHlvdXIgZW5kIGdvYWw/Cj4KPiBUaGFua3MKPiBLZXZpbgo+Cl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkludGVsLWdmeCBtYWls aW5nIGxpc3QKSW50ZWwtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwOi8vbGlzdHMuZnJl ZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757856AbbKSHXZ (ORCPT ); Thu, 19 Nov 2015 02:23:25 -0500 Received: from mga11.intel.com ([192.55.52.93]:18288 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753065AbbKSHXX (ORCPT ); Thu, 19 Nov 2015 02:23:23 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,317,1444719600"; d="scan'208";a="854556431" Message-ID: <564D78D0.80904@intel.com> Date: Thu, 19 Nov 2015 15:22:56 +0800 From: Jike Song User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Alex Williamson CC: "Tian, Kevin" , "xen-devel@lists.xen.org" , "igvt-g@ml01.01.org" , "intel-gfx@lists.freedesktop.org" , "linux-kernel@vger.kernel.org" , "White, Michael L" , "Dong, Eddie" , "Li, Susie" , "Cowperthwaite, David J" , "Reddy, Raghuveer" , "Zhu, Libo" , "Zhou, Chao" , "Wang, Hongbo" , "Lv, Zhiyuan" , qemu-devel , Paolo Bonzini , Gerd Hoffmann Subject: Re: [Intel-gfx] [Announcement] 2015-Q3 release of XenGT - a Mediated Graphics Passthrough Solution from Intel References: <53D215D3.50608@intel.com> <547FCAAD.2060406@intel.com> <54AF967B.3060503@intel.com> <5527CEC4.9080700@intel.com> <559B3E38.1080707@intel.com> <562F4311.9@intel.com> <1447870341.4697.92.camel@redhat.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Alex, On 11/19/2015 12:06 PM, Tian, Kevin wrote: >> From: Alex Williamson [mailto:alex.williamson@redhat.com] >> Sent: Thursday, November 19, 2015 2:12 AM >> >> [cc +qemu-devel, +paolo, +gerd] >> >> On Tue, 2015-10-27 at 17:25 +0800, Jike Song wrote: >>> {snip} >> >> Hi! >> >> At redhat we've been thinking about how to support vGPUs from multiple >> vendors in a common way within QEMU. We want to enable code sharing >> between vendors and give new vendors an easy path to add their own >> support. We also have the complication that not all vGPU vendors are as >> open source friendly as Intel, so being able to abstract the device >> mediation and access outside of QEMU is a big advantage. >> >> The proposal I'd like to make is that a vGPU, whether it is from Intel >> or another vendor, is predominantly a PCI(e) device. We have an >> interface in QEMU already for exposing arbitrary PCI devices, vfio-pci. >> Currently vfio-pci uses the VFIO API to interact with "physical" devices >> and system IOMMUs. I highlight /physical/ there because some of these >> physical devices are SR-IOV VFs, which is somewhat of a fuzzy concept, >> somewhere between fixed hardware and a virtual device implemented in >> software. That software just happens to be running on the physical >> endpoint. > > Agree. > > One clarification for rest discussion, is that we're talking about GVT-g vGPU > here which is a pure software GPU virtualization technique. GVT-d (note > some use in the text) refers to passing through the whole GPU or a specific > VF. GVT-d already falls into existing VFIO APIs nicely (though some on-going > effort to remove Intel specific platform stickness from gfx driver). :-) > Hi Alex, thanks for the discussion. In addition to Kevin's replies, I have a high-level question: can VFIO be used by QEMU for both KVM and Xen? -- Thanks, Jike >> >> vGPUs are similar, with the virtual device created at a different point, >> host software. They also rely on different IOMMU constructs, making use >> of the MMU capabilities of the GPU (GTTs and such), but really having >> similar requirements. > > One important difference between system IOMMU and GPU-MMU here. > System IOMMU is very much about translation from a DMA target > (IOVA on native, or GPA in virtualization case) to HPA. However GPU > internal MMUs is to translate from Graphics Memory Address (GMA) > to DMA target (HPA if system IOMMU is disabled, or IOVA/GPA if system > IOMMU is enabled). GMA is an internal addr space within GPU, not > exposed to Qemu and fully managed by GVT-g device model. Since it's > not a standard PCI defined resource, we don't need abstract this capability > in VFIO interface. > >> >> The proposal is therefore that GPU vendors can expose vGPUs to >> userspace, and thus to QEMU, using the VFIO API. For instance, vfio >> supports modular bus drivers and IOMMU drivers. An intel-vfio-gvt-d >> module (or extension of i915) can register as a vfio bus driver, create >> a struct device per vGPU, create an IOMMU group for that device, and >> register that device with the vfio-core. Since we don't rely on the >> system IOMMU for GVT-d vGPU assignment, another vGPU vendor driver (or >> extension of the same module) can register a "type1" compliant IOMMU >> driver into vfio-core. From the perspective of QEMU then, all of the >> existing vfio-pci code is re-used, QEMU remains largely unaware of any >> specifics of the vGPU being assigned, and the only necessary change so >> far is how QEMU traverses sysfs to find the device and thus the IOMMU >> group leading to the vfio group. > > GVT-g requires to pin guest memory and query GPA->HPA information, > upon which shadow GTTs will be updated accordingly from (GMA->GPA) > to (GMA->HPA). So yes, here a dummy or simple "type1" compliant IOMMU > can be introduced just for this requirement. > > However there's one tricky point which I'm not sure whether overall > VFIO concept will be violated. GVT-g doesn't require system IOMMU > to function, however host system may enable system IOMMU just for > hardening purpose. This means two-level translations existing (GMA-> > IOVA->HPA), so the dummy IOMMU driver has to request system IOMMU > driver to allocate IOVA for VMs and then setup IOVA->HPA mapping > in IOMMU page table. In this case, multiple VM's translations are > multiplexed in one IOMMU page table. > > We might need create some group/sub-group or parent/child concepts > among those IOMMUs for thorough permission control. > >> >> There are a few areas where we know we'll need to extend the VFIO API to >> make this work, but it seems like they can all be done generically. One >> is that PCI BARs are described through the VFIO API as regions and each >> region has a single flag describing whether mmap (ie. direct mapping) of >> that region is possible. We expect that vGPUs likely need finer >> granularity, enabling some areas within a BAR to be trapped and fowarded >> as a read or write access for the vGPU-vfio-device module to emulate, >> while other regions, like framebuffers or texture regions, are directly >> mapped. I have prototype code to enable this already. > > Yes in GVT-g one BAR resource might be partitioned among multiple vGPUs. > If VFIO can support such partial resource assignment, it'd be great. Similar > parent/child concept might also be required here, so any resource enumerated > on a vGPU shouldn't break limitations enforced on the physical device. > > One unique requirement for GVT-g here, though, is that vGPU device model > need to know guest BAR configuration for proper emulation (e.g. register > IO emulation handler to KVM). Similar is about guest MSI vector for virtual > interrupt injection. Not sure how this can be fit into common VFIO model. > Does VFIO allow vendor specific extension today? > >> >> Another area is that we really don't want to proliferate each vGPU >> needing a new IOMMU type within vfio. The existing type1 IOMMU provides >> potentially the most simple mapping and unmapping interface possible. >> We'd therefore need to allow multiple "type1" IOMMU drivers for vfio, >> making type1 be more of an interface specification rather than a single >> implementation. This is a trivial change to make within vfio and one >> that I believe is compatible with the existing API. Note that >> implementing a type1-compliant vfio IOMMU does not imply pinning an >> mapping every registered page. A vGPU, with mediated device access, may >> use this only to track the current HVA to GPA mappings for a VM. Only >> when a DMA is enabled for the vGPU instance is that HVA pinned and an >> HPA to GPA translation programmed into the GPU MMU. >> >> Another area of extension is how to expose a framebuffer to QEMU for >> seamless integration into a SPICE/VNC channel. For this I believe we >> could use a new region, much like we've done to expose VGA access >> through a vfio device file descriptor. An area within this new >> framebuffer region could be directly mappable in QEMU while a >> non-mappable page, at a standard location with standardized format, >> provides a description of framebuffer and potentially even a >> communication channel to synchronize framebuffer captures. This would >> be new code for QEMU, but something we could share among all vGPU >> implementations. > > Now GVT-g already provides an interface to decode framebuffer information, > w/ an assumption that the framebuffer will be further composited into > OpenGL APIs. So the format is defined according to OpenGL definition. > Does that meet SPICE requirement? > > Another thing to be added. Framebuffers are frequently switched in > reality. So either Qemu needs to poll or a notification mechanism is required. > And since it's dynamic, having framebuffer page directly exposed in the > new region might be tricky. We can just expose framebuffer information > (including base, format, etc.) and let Qemu to map separately out of VFIO > interface. > > And... this works fine with vGPU model since software knows all the > detail about framebuffer. However in pass-through case, who do you expect > to provide that information? Is it OK to introduce vGPU specific APIs in > VFIO? > >> >> Another obvious area to be standardized would be how to discover, >> create, and destroy vGPU instances. SR-IOV has a standard mechanism to >> create VFs in sysfs and I would propose that vGPU vendors try to >> standardize on similar interfaces to enable libvirt to easily discover >> the vGPU capabilities of a given GPU and manage the lifecycle of a vGPU >> instance. > > Now there is no standard. We expose vGPU life-cycle mgmt. APIs through > sysfs (under i915 node), which is very Intel specific. In reality different > vendors have quite different capabilities for their own vGPUs, so not sure > how standard we can define such a mechanism. But this code should be > minor to be maintained in libvirt. > >> >> This is obviously a lot to digest, but I'd certainly be interested in >> hearing feedback on this proposal as well as try to clarify anything >> I've left out or misrepresented above. Another benefit to this >> mechanism is that direct GPU assignment and vGPU assignment use the same >> code within QEMU and same API to the kernel, which should make debugging >> and code support between the two easier. I'd really like to start a >> discussion around this proposal, and of course the first open source >> implementation of this sort of model will really help to drive the >> direction it takes. Thanks! >> > > Thanks for starting this discussion. Intel will definitely work with > community on this work. Based on earlier comments, I'm not sure > whether we can exactly same code for direct GPU assignment and > vGPU assignment, since even we extend VFIO some interfaces might > be vGPU specific. Does this way still achieve your end goal? > > Thanks > Kevin > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34422) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZzJZ3-0003kS-3e for qemu-devel@nongnu.org; Thu, 19 Nov 2015 02:23:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZzJYz-0007O0-T1 for qemu-devel@nongnu.org; Thu, 19 Nov 2015 02:23:29 -0500 Received: from mga14.intel.com ([192.55.52.115]:4413) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZzJYz-0007Ni-HM for qemu-devel@nongnu.org; Thu, 19 Nov 2015 02:23:25 -0500 Message-ID: <564D78D0.80904@intel.com> Date: Thu, 19 Nov 2015 15:22:56 +0800 From: Jike Song MIME-Version: 1.0 References: <53D215D3.50608@intel.com> <547FCAAD.2060406@intel.com> <54AF967B.3060503@intel.com> <5527CEC4.9080700@intel.com> <559B3E38.1080707@intel.com> <562F4311.9@intel.com> <1447870341.4697.92.camel@redhat.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [Intel-gfx] [Announcement] 2015-Q3 release of XenGT - a Mediated Graphics Passthrough Solution from Intel List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Williamson Cc: "igvt-g@ml01.01.org" , "Tian, Kevin" , "Reddy, Raghuveer" , qemu-devel , "White, Michael L" , "Cowperthwaite, David J" , "intel-gfx@lists.freedesktop.org" , "Li, Susie" , "Dong, Eddie" , "linux-kernel@vger.kernel.org" , "xen-devel@lists.xen.org" , Gerd Hoffmann , "Zhou, Chao" , Paolo Bonzini , "Zhu, Libo" , "Wang, Hongbo" , "Lv, Zhiyuan" Hi Alex, On 11/19/2015 12:06 PM, Tian, Kevin wrote: >> From: Alex Williamson [mailto:alex.williamson@redhat.com] >> Sent: Thursday, November 19, 2015 2:12 AM >> >> [cc +qemu-devel, +paolo, +gerd] >> >> On Tue, 2015-10-27 at 17:25 +0800, Jike Song wrote: >>> {snip} >> >> Hi! >> >> At redhat we've been thinking about how to support vGPUs from multiple >> vendors in a common way within QEMU. We want to enable code sharing >> between vendors and give new vendors an easy path to add their own >> support. We also have the complication that not all vGPU vendors are as >> open source friendly as Intel, so being able to abstract the device >> mediation and access outside of QEMU is a big advantage. >> >> The proposal I'd like to make is that a vGPU, whether it is from Intel >> or another vendor, is predominantly a PCI(e) device. We have an >> interface in QEMU already for exposing arbitrary PCI devices, vfio-pci. >> Currently vfio-pci uses the VFIO API to interact with "physical" devices >> and system IOMMUs. I highlight /physical/ there because some of these >> physical devices are SR-IOV VFs, which is somewhat of a fuzzy concept, >> somewhere between fixed hardware and a virtual device implemented in >> software. That software just happens to be running on the physical >> endpoint. > > Agree. > > One clarification for rest discussion, is that we're talking about GVT-g vGPU > here which is a pure software GPU virtualization technique. GVT-d (note > some use in the text) refers to passing through the whole GPU or a specific > VF. GVT-d already falls into existing VFIO APIs nicely (though some on-going > effort to remove Intel specific platform stickness from gfx driver). :-) > Hi Alex, thanks for the discussion. In addition to Kevin's replies, I have a high-level question: can VFIO be used by QEMU for both KVM and Xen? -- Thanks, Jike >> >> vGPUs are similar, with the virtual device created at a different point, >> host software. They also rely on different IOMMU constructs, making use >> of the MMU capabilities of the GPU (GTTs and such), but really having >> similar requirements. > > One important difference between system IOMMU and GPU-MMU here. > System IOMMU is very much about translation from a DMA target > (IOVA on native, or GPA in virtualization case) to HPA. However GPU > internal MMUs is to translate from Graphics Memory Address (GMA) > to DMA target (HPA if system IOMMU is disabled, or IOVA/GPA if system > IOMMU is enabled). GMA is an internal addr space within GPU, not > exposed to Qemu and fully managed by GVT-g device model. Since it's > not a standard PCI defined resource, we don't need abstract this capability > in VFIO interface. > >> >> The proposal is therefore that GPU vendors can expose vGPUs to >> userspace, and thus to QEMU, using the VFIO API. For instance, vfio >> supports modular bus drivers and IOMMU drivers. An intel-vfio-gvt-d >> module (or extension of i915) can register as a vfio bus driver, create >> a struct device per vGPU, create an IOMMU group for that device, and >> register that device with the vfio-core. Since we don't rely on the >> system IOMMU for GVT-d vGPU assignment, another vGPU vendor driver (or >> extension of the same module) can register a "type1" compliant IOMMU >> driver into vfio-core. From the perspective of QEMU then, all of the >> existing vfio-pci code is re-used, QEMU remains largely unaware of any >> specifics of the vGPU being assigned, and the only necessary change so >> far is how QEMU traverses sysfs to find the device and thus the IOMMU >> group leading to the vfio group. > > GVT-g requires to pin guest memory and query GPA->HPA information, > upon which shadow GTTs will be updated accordingly from (GMA->GPA) > to (GMA->HPA). So yes, here a dummy or simple "type1" compliant IOMMU > can be introduced just for this requirement. > > However there's one tricky point which I'm not sure whether overall > VFIO concept will be violated. GVT-g doesn't require system IOMMU > to function, however host system may enable system IOMMU just for > hardening purpose. This means two-level translations existing (GMA-> > IOVA->HPA), so the dummy IOMMU driver has to request system IOMMU > driver to allocate IOVA for VMs and then setup IOVA->HPA mapping > in IOMMU page table. In this case, multiple VM's translations are > multiplexed in one IOMMU page table. > > We might need create some group/sub-group or parent/child concepts > among those IOMMUs for thorough permission control. > >> >> There are a few areas where we know we'll need to extend the VFIO API to >> make this work, but it seems like they can all be done generically. One >> is that PCI BARs are described through the VFIO API as regions and each >> region has a single flag describing whether mmap (ie. direct mapping) of >> that region is possible. We expect that vGPUs likely need finer >> granularity, enabling some areas within a BAR to be trapped and fowarded >> as a read or write access for the vGPU-vfio-device module to emulate, >> while other regions, like framebuffers or texture regions, are directly >> mapped. I have prototype code to enable this already. > > Yes in GVT-g one BAR resource might be partitioned among multiple vGPUs. > If VFIO can support such partial resource assignment, it'd be great. Similar > parent/child concept might also be required here, so any resource enumerated > on a vGPU shouldn't break limitations enforced on the physical device. > > One unique requirement for GVT-g here, though, is that vGPU device model > need to know guest BAR configuration for proper emulation (e.g. register > IO emulation handler to KVM). Similar is about guest MSI vector for virtual > interrupt injection. Not sure how this can be fit into common VFIO model. > Does VFIO allow vendor specific extension today? > >> >> Another area is that we really don't want to proliferate each vGPU >> needing a new IOMMU type within vfio. The existing type1 IOMMU provides >> potentially the most simple mapping and unmapping interface possible. >> We'd therefore need to allow multiple "type1" IOMMU drivers for vfio, >> making type1 be more of an interface specification rather than a single >> implementation. This is a trivial change to make within vfio and one >> that I believe is compatible with the existing API. Note that >> implementing a type1-compliant vfio IOMMU does not imply pinning an >> mapping every registered page. A vGPU, with mediated device access, may >> use this only to track the current HVA to GPA mappings for a VM. Only >> when a DMA is enabled for the vGPU instance is that HVA pinned and an >> HPA to GPA translation programmed into the GPU MMU. >> >> Another area of extension is how to expose a framebuffer to QEMU for >> seamless integration into a SPICE/VNC channel. For this I believe we >> could use a new region, much like we've done to expose VGA access >> through a vfio device file descriptor. An area within this new >> framebuffer region could be directly mappable in QEMU while a >> non-mappable page, at a standard location with standardized format, >> provides a description of framebuffer and potentially even a >> communication channel to synchronize framebuffer captures. This would >> be new code for QEMU, but something we could share among all vGPU >> implementations. > > Now GVT-g already provides an interface to decode framebuffer information, > w/ an assumption that the framebuffer will be further composited into > OpenGL APIs. So the format is defined according to OpenGL definition. > Does that meet SPICE requirement? > > Another thing to be added. Framebuffers are frequently switched in > reality. So either Qemu needs to poll or a notification mechanism is required. > And since it's dynamic, having framebuffer page directly exposed in the > new region might be tricky. We can just expose framebuffer information > (including base, format, etc.) and let Qemu to map separately out of VFIO > interface. > > And... this works fine with vGPU model since software knows all the > detail about framebuffer. However in pass-through case, who do you expect > to provide that information? Is it OK to introduce vGPU specific APIs in > VFIO? > >> >> Another obvious area to be standardized would be how to discover, >> create, and destroy vGPU instances. SR-IOV has a standard mechanism to >> create VFs in sysfs and I would propose that vGPU vendors try to >> standardize on similar interfaces to enable libvirt to easily discover >> the vGPU capabilities of a given GPU and manage the lifecycle of a vGPU >> instance. > > Now there is no standard. We expose vGPU life-cycle mgmt. APIs through > sysfs (under i915 node), which is very Intel specific. In reality different > vendors have quite different capabilities for their own vGPUs, so not sure > how standard we can define such a mechanism. But this code should be > minor to be maintained in libvirt. > >> >> This is obviously a lot to digest, but I'd certainly be interested in >> hearing feedback on this proposal as well as try to clarify anything >> I've left out or misrepresented above. Another benefit to this >> mechanism is that direct GPU assignment and vGPU assignment use the same >> code within QEMU and same API to the kernel, which should make debugging >> and code support between the two easier. I'd really like to start a >> discussion around this proposal, and of course the first open source >> implementation of this sort of model will really help to drive the >> direction it takes. Thanks! >> > > Thanks for starting this discussion. Intel will definitely work with > community on this work. Based on earlier comments, I'm not sure > whether we can exactly same code for direct GPU assignment and > vGPU assignment, since even we extend VFIO some interfaces might > be vGPU specific. Does this way still achieve your end goal? > > Thanks > Kevin >