From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCH V2] vhost: introduce O(1) vq metadata cache Date: Mon, 27 Feb 2017 20:35:47 +0200 Message-ID: <20170227203529-mutt-send-email-mst@kernel.org> References: <1481709237-19827-1-git-send-email-jasowang@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: Jason Wang Cc: kvm@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, maxime.coquelin@redhat.com, wexu@redhat.com, vkaplans@redhat.com List-Id: virtualization@lists.linuxfoundation.org T24gV2VkLCBGZWIgMTUsIDIwMTcgYXQgMDE6Mzc6MTdQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiAKPiAKPiBPbiAyMDE25bm0MTLmnIgxNOaXpSAxNzo1MywgSmFzb24gV2FuZyB3cm90ZToK PiA+IFdoZW4gZGV2aWNlIElPVExCIGlzIGVuYWJsZWQsIGFsbCBhZGRyZXNzIHRyYW5zbGF0aW9u cyB3ZXJlIHN0b3JlZCBpbgo+ID4gaW50ZXJ2YWwgdHJlZS4gTyhsZ04pIHNlYXJjaGluZyB0aW1l IGNvdWxkIGJlIHNsb3cgZm9yIHZpcnRxdWV1ZQo+ID4gbWV0YWRhdGEgKGF2YWlsLCB1c2VkIGFu ZCBkZXNjcmlwdG9ycykgc2luY2UgdGhleSB3ZXJlIGFjY2Vzc2VkIG11Y2gKPiA+IG9mdGVuIHRo YW4gb3RoZXIgYWRkcmVzc2VzLiBTbyB0aGlzIHBhdGNoIGludHJvZHVjZXMgYW4gTygxKSBhcnJh eQo+ID4gd2hpY2ggcG9pbnRzIHRvIHRoZSBpbnRlcnZhbCB0cmVlIG5vZGVzIHRoYXQgc3RvcmUg dGhlIHRyYW5zbGF0aW9ucyBvZgo+ID4gdnEgbWV0YWRhdGEuIFRob3NlIGFycmF5IHdlcmUgdXBk YXRlIGR1cmluZyB2cSBJT1RMQiBwcmVmZXRjaGluZyBhbmQKPiA+IHdlcmUgcmVzZXQgZHVyaW5n IGVhY2ggaW52YWxpZGF0aW9uIGFuZCB0bGIgdXBkYXRlLiBFYWNoIHRpbWUgd2Ugd2FudAo+ID4g dG8gYWNjZXNzIHZxIG1ldGFkYXRhLCB0aGlzIHNtYWxsIGFycmF5IHdlcmUgcXVlcmllZCBiZWZv cmUgaW50ZXJ2YWwKPiA+IHRyZWUuIFRoaXMgd291bGQgYmUgc3VmZmljaWVudCBmb3Igc3RhdGlj IG1hcHBpbmdzIGJ1dCBub3QgZHluYW1pYwo+ID4gbWFwcGluZ3MsIHdlIGNvdWxkIGRvIG9wdGlt aXphdGlvbnMgb24gdG9wLgo+ID4gCj4gPiBUZXN0IHdlcmUgZG9uZSB3aXRoIGwyZndkIGluIGd1 ZXN0ICgyTSBodWdlcGFnZSk6Cj4gPiAKPiA+ICAgICBub2lvbW11ICB8IGJlZm9yZSAgICAgICAg fCBhZnRlcgo+ID4gdHggMS4zMk1wcHMgfCAxLjA2TXBwcyg4MiUpIHwgMS4zME1wcHMoOTglKQo+ ID4gcnggMi4zM01wcHMgfCAxLjQ2TXBwcyg2MyUpIHwgMi4yOU1wcHMoOTglKQo+ID4gCj4gPiBX ZSBjYW4gYWxtb3N0IHJlYWNoIHRoZSBzYW1lIHBlcmZvcm1hbmNlIGFzIG5vaW9tbXUgbW9kZS4K PiA+IAo+ID4gU2lnbmVkLW9mZi1ieTogSmFzb24gV2FuZzxqYXNvd2FuZ0ByZWRoYXQuY29tPgo+ ID4gLS0tCj4gPiBDaGFuZ2VzIGZyb20gVjE6Cj4gPiAtIHNpbGVudCAzMmJpdCBidWlsZCB3YXJu aW5nCj4gCj4gcGluZwoKQ291bGQgeW91IHJlYmFzZSBwbHM/CkkgcHVzaGVkIG15IHRyZWUgaW50 byBsaW51eCBuZXh0LgoKLS0gCk1TVApfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fXwpWaXJ0dWFsaXphdGlvbiBtYWlsaW5nIGxpc3QKVmlydHVhbGl6YXRpb25A bGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24u b3JnL21haWxtYW4vbGlzdGluZm8vdmlydHVhbGl6YXRpb24= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751584AbdB0TC2 (ORCPT ); Mon, 27 Feb 2017 14:02:28 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59196 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751422AbdB0TC1 (ORCPT ); Mon, 27 Feb 2017 14:02:27 -0500 Date: Mon, 27 Feb 2017 20:35:47 +0200 From: "Michael S. Tsirkin" To: Jason Wang Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, wexu@redhat.com, peterx@redhat.com, vkaplans@redhat.com, maxime.coquelin@redhat.com Subject: Re: [PATCH V2] vhost: introduce O(1) vq metadata cache Message-ID: <20170227203529-mutt-send-email-mst@kernel.org> References: <1481709237-19827-1-git-send-email-jasowang@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 27 Feb 2017 18:35:49 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Feb 15, 2017 at 01:37:17PM +0800, Jason Wang wrote: > > > On 2016年12月14日 17:53, Jason Wang wrote: > > When device IOTLB is enabled, all address translations were stored in > > interval tree. O(lgN) searching time could be slow for virtqueue > > metadata (avail, used and descriptors) since they were accessed much > > often than other addresses. So this patch introduces an O(1) array > > which points to the interval tree nodes that store the translations of > > vq metadata. Those array were update during vq IOTLB prefetching and > > were reset during each invalidation and tlb update. Each time we want > > to access vq metadata, this small array were queried before interval > > tree. This would be sufficient for static mappings but not dynamic > > mappings, we could do optimizations on top. > > > > Test were done with l2fwd in guest (2M hugepage): > > > > noiommu | before | after > > tx 1.32Mpps | 1.06Mpps(82%) | 1.30Mpps(98%) > > rx 2.33Mpps | 1.46Mpps(63%) | 2.29Mpps(98%) > > > > We can almost reach the same performance as noiommu mode. > > > > Signed-off-by: Jason Wang > > --- > > Changes from V1: > > - silent 32bit build warning > > ping Could you rebase pls? I pushed my tree into linux next. -- MST