From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Wang Subject: Re: [RFC PATCH V3 0/3] basic device IOTLB support Date: Wed, 22 Jun 2016 17:53:39 +0800 Message-ID: <576A6023.1090308@redhat.com> References: <1464082585-13049-1-git-send-email-jasowang@redhat.com> <20160621164449.GC30949@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20160621164449.GC30949@redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: "Michael S. Tsirkin" Cc: kvm@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, peterx@redhat.com, virtualization@lists.linux-foundation.org, wexu@redhat.com, vkaplans@redhat.com List-Id: virtualization@lists.linuxfoundation.org CgpPbiAyMDE25bm0MDbmnIgyMuaXpSAwMDo0NCwgTWljaGFlbCBTLiBUc2lya2luIHdyb3RlOgo+ IE9uIFR1ZSwgTWF5IDI0LCAyMDE2IGF0IDA1OjM2OjIyUE0gKzA4MDAsIEphc29uIFdhbmcgd3Jv dGU6Cj4+IFRoaXMgcGF0Y2ggdHJpZXMgdG8gaW1wbGVtZW50IGFuIGRldmljZSBJT1RMQiBmb3Ig dmhvc3QuIFRoaXMgY291bGQgYmUKPj4gdXNlZCB3aXRoIGZvciBjby1vcGVyYXRpb24gd2l0aCB1 c2Vyc3BhY2UgSU9NTVUgaW1wbGVtZW50YXRpb24gKHFlbXUpCj4+IGZvciBhIHNlY3VyZSBETUEg ZW52aXJvbm1lbnQgKERNQVIpIGluIGd1ZXN0Lgo+Pgo+PiBUaGUgaWRlYSBpcyBzaW1wbGUuIFdo ZW4gdmhvc3QgbWVldHMgYW4gSU9UTEIgbWlzcywgaXQgd2lsbCByZXF1ZXN0Cj4+IHRoZSBhc3Np c3RhbmNlIG9mIHVzZXJzcGFjZSB0byBkbyB0aGUgdHJhbnNsYXRpb24sIHRoaXMgaXMgZG9uZQo+ PiB0aHJvdWdoOgo+Pgo+PiAtIHdoZW4gdGhlcmUncyBhIElPVExCIG1pc3MsIGl0IHdpbGwgbm90 aWZ5IHVzZXJzcGFjZSB0aHJvdWdoCj4+ICAgIHZob3N0X25ldCBmZCBhbmQgdGhlbiB1c2Vyc3Bh Y2UgcmVhZCB0aGUgZmF1bHQgYWRkcmVzcywgc2l6ZSBhbmQKPj4gICAgYWNjZXNzIGZyb20gdmhv c3QgZmQuCj4+IC0gdXNlcnNwYWNlIHdyaXRlIHRoZSB0cmFuc2xhdGlvbiByZXN1bHQgYmFjayB0 byB2aG9zdCBmZCwgdmhvc3QgY2FuCj4+ICAgIHRoZW4gdXBkYXRlIGl0cyBJT1RMQi4KPj4KPj4g VGhlIGNvZGVzIHdlcmUgb3B0aW1pemVkIGZvciBmaXhlZCBtYXBwaW5nIHVzZXJzIGUuZyBkcGRr IGluIGd1ZXN0LiBJdAo+PiB3aWxsIGJlIHNsb3cgaWYgZHluYW1pYyBtYXBwaW5ncyB3ZXJlIHVz ZWQgaW4gZ3Vlc3QuIFdlIGNvdWxkIGRvCj4+IG9wdGltaXphdGlvbnMgb24gdG9wLgo+Pgo+PiBU aGUgY29kZXMgd2VyZSBkZXNpZ25lZCB0byBiZSBhcmNoaXRlY3R1cmUgaW5kZXBlbmRlbnQuIEl0 IHNob3VsZCBiZQo+PiBlYXNpbHkgcG9ydGVkIHRvIGFueSBhcmNoaXRlY3R1cmUuCj4+Cj4+IFN0 cmVzcyB0ZXN0ZWQgd2l0aCBsMmZ3ZC92ZmlvIGluIGd1ZXN0IHdpdGggNEsvMk0vMUcgcGFnZSBz aXplLiBPbiAxRwo+PiBodWdlcGFnZSBjYXNlLCAxMDAlIFRMQiBoaXQgcmF0ZSB3ZXJlIG5vdGlj ZWQuCj4+Cj4+IENoYW5nZXMgZnJvbSBWMjoKPj4gLSBpbnRyb2R1Y2UgbWVtb3J5IGFjY2Vzc29y cyBmb3Igdmhvc3QKPj4gLSBzd2l0Y2ggZnJvbSBpb2N0bHMgdG8gb3JpZGluYXJ5IGZpbGUgcmVh ZC93cml0ZSBmb3IgaW90bGIgbWlzcyBhbmQKPj4gICAgdXBkYXRpbmcKPj4gLSBkbyBub3QgYXNz dW1lIHZpcnRxdWV1ZSB3ZXJlIHZpcnR1YWxseSBtYXBwZWQgY29udGlndW91c2x5LCBhbGwKPj4g ICAgdmlydHF1ZXVlIGFjY2VzcyB3ZXJlIGRvbmUgdGhyb3VnIElPVExCCj4+IC0gdmVyaWZ5IG1l bW9yeSBhY2Nlc3MgZHVyaW5nIElPVExCIHVwZGF0ZSBhbmQgZmFpbCBlYXJseQo+PiAtIGludHJv ZHVjZSBhIG1vZHVsZSBwYXJhbWV0ZXIgZm9yIHRoZSBzaXplIG9mIElPVExCCj4+Cj4+IENoYW5n ZXMgZnJvbSBWMToKPj4gLSBzdXBwb3J0IGFueSBzaXplL3JhbmdlIG9mIHVwZGF0aW5nIGFuZCBp bnZhbGlkYXRpb24gdGhyb3VnaAo+PiAgICBpbnRyb2R1Y2luZyB0aGUgaW50ZXJ2YWwgdHJlZS4K Pj4gLSBjb252ZXJ0IGZyb20gcGVyIGRldmljZSBpb3RsYiByZXF1ZXN0IHRvIHBlciB2aXJ0cXVl dWUgaW90bGIKPj4gICAgcmVxdWVzdCwgdGhpcyBzb2x2ZXMgdGhlIHBvc3NpYmxlIGRlYWRsb2Nr IGluIFYxLgo+PiAtIHJlYWQvd3JpdGUgcGVybWlzc2lvbiBjaGVjayBzdXBwb3J0Lgo+Pgo+PiBQ bGVhc2UgcmV2aWV3Lgo+IE5pY2UsIHRoaXMgbG9va3MgZ29vZCB0byBtZS4gQ2FuIHlvdSBwb3N0 IGEgbm9uLXJmYyBwbGVhc2U/Cj4KPgoKUG9zdGVkLCB0aGFua3MuCgo+PiBKYXNvbiBXYW5nICgz KToKPj4gICAgdmhvc3Q6IGludHJvZHVjZSB2aG9zdCBtZW1vcnkgYWNjZXNzb3JzCj4+ICAgIHZo b3N0OiBjb252ZXJ0IHByZSBzb3J0ZWQgdmhvc3QgbWVtb3J5IGFycmF5IHRvIGludGVydmFsIHRy ZWUKPj4gICAgdmhvc3Q6IGRldmljZSBJT1RMQiBBUEkKPj4KPj4gICBkcml2ZXJzL3Zob3N0L25l dC5jICAgICAgICB8ICA2MyArKystCj4+ICAgZHJpdmVycy92aG9zdC92aG9zdC5jICAgICAgfCA3 NjAgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tLS0tCj4+ICAgZHJp dmVycy92aG9zdC92aG9zdC5oICAgICAgfCAgNjAgKysrLQo+PiAgIGluY2x1ZGUvdWFwaS9saW51 eC92aG9zdC5oIHwgIDI4ICsrCj4+ICAgNCBmaWxlcyBjaGFuZ2VkLCA3OTAgaW5zZXJ0aW9ucygr KSwgMTIxIGRlbGV0aW9ucygtKQo+Pgo+PiAtLSAKPj4gMi43LjQKPiAtLQo+IFRvIHVuc3Vic2Ny aWJlIGZyb20gdGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5lICJ1bnN1YnNjcmliZSBrdm0iIGluCj4g dGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5vcmcKPiBNb3Jl IG1ham9yZG9tbyBpbmZvIGF0ICBodHRwOi8vdmdlci5rZXJuZWwub3JnL21ham9yZG9tby1pbmZv Lmh0bWwKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClZp cnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3Vu ZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0 aW5mby92aXJ0dWFsaXphdGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752306AbcFVKEK (ORCPT ); Wed, 22 Jun 2016 06:04:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59705 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751505AbcFVKDr (ORCPT ); Wed, 22 Jun 2016 06:03:47 -0400 Subject: Re: [RFC PATCH V3 0/3] basic device IOTLB support To: "Michael S. Tsirkin" References: <1464082585-13049-1-git-send-email-jasowang@redhat.com> <20160621164449.GC30949@redhat.com> Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, wexu@redhat.com, peterx@redhat.com, vkaplans@redhat.com From: Jason Wang Message-ID: <576A6023.1090308@redhat.com> Date: Wed, 22 Jun 2016 17:53:39 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <20160621164449.GC30949@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 22 Jun 2016 09:53:45 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2016年06月22日 00:44, Michael S. Tsirkin wrote: > On Tue, May 24, 2016 at 05:36:22PM +0800, Jason Wang wrote: >> This patch tries to implement an device IOTLB for vhost. This could be >> used with for co-operation with userspace IOMMU implementation (qemu) >> for a secure DMA environment (DMAR) in guest. >> >> The idea is simple. When vhost meets an IOTLB miss, it will request >> the assistance of userspace to do the translation, this is done >> through: >> >> - when there's a IOTLB miss, it will notify userspace through >> vhost_net fd and then userspace read the fault address, size and >> access from vhost fd. >> - userspace write the translation result back to vhost fd, vhost can >> then update its IOTLB. >> >> The codes were optimized for fixed mapping users e.g dpdk in guest. It >> will be slow if dynamic mappings were used in guest. We could do >> optimizations on top. >> >> The codes were designed to be architecture independent. It should be >> easily ported to any architecture. >> >> Stress tested with l2fwd/vfio in guest with 4K/2M/1G page size. On 1G >> hugepage case, 100% TLB hit rate were noticed. >> >> Changes from V2: >> - introduce memory accessors for vhost >> - switch from ioctls to oridinary file read/write for iotlb miss and >> updating >> - do not assume virtqueue were virtually mapped contiguously, all >> virtqueue access were done throug IOTLB >> - verify memory access during IOTLB update and fail early >> - introduce a module parameter for the size of IOTLB >> >> Changes from V1: >> - support any size/range of updating and invalidation through >> introducing the interval tree. >> - convert from per device iotlb request to per virtqueue iotlb >> request, this solves the possible deadlock in V1. >> - read/write permission check support. >> >> Please review. > Nice, this looks good to me. Can you post a non-rfc please? > > Posted, thanks. >> Jason Wang (3): >> vhost: introduce vhost memory accessors >> vhost: convert pre sorted vhost memory array to interval tree >> vhost: device IOTLB API >> >> drivers/vhost/net.c | 63 +++- >> drivers/vhost/vhost.c | 760 ++++++++++++++++++++++++++++++++++++++------- >> drivers/vhost/vhost.h | 60 +++- >> include/uapi/linux/vhost.h | 28 ++ >> 4 files changed, 790 insertions(+), 121 deletions(-) >> >> -- >> 2.7.4 > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html