From mboxrd@z Thu Jan 1 00:00:00 1970 From: Xiao Guangrong Subject: Re: [PATCH 1/2] KVM: MMU: Mark sp mmio cached when creating mmio spte Date: Thu, 14 Mar 2013 13:36:24 +0800 Message-ID: <514161D8.6020603@linux.vnet.ibm.com> References: <20130312174440.5d5199ee.yoshikawa_takuya_b1@lab.ntt.co.jp> <5140094F.5080700@linux.vnet.ibm.com> <20130313162816.c62899dc.yoshikawa_takuya_b1@lab.ntt.co.jp> <51402DDA.607@linux.vnet.ibm.com> <20130313123358.GM11223@redhat.com> <51407441.4020200@linux.vnet.ibm.com> <20130313224056.8c9c87f4d95b332d2273a685@gmail.com> <514087A0.1000704@linux.vnet.ibm.com> <20130314015821.GA13261@amt.cnet> <20130314112641.e2ccbc6b.yoshikawa_takuya_b1@lab.ntt.co.jp> <20130314023953.GB18111@amt.cnet> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Takuya Yoshikawa , Takuya Yoshikawa , Gleb Natapov , kvm@vger.kernel.org To: Marcelo Tosatti Return-path: Received: from e28smtp09.in.ibm.com ([122.248.162.9]:39191 "EHLO e28smtp09.in.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751609Ab3CNFgg (ORCPT ); Thu, 14 Mar 2013 01:36:36 -0400 Received: from /spool/local by e28smtp09.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 14 Mar 2013 11:04:12 +0530 Received: from d28relay02.in.ibm.com (d28relay02.in.ibm.com [9.184.220.59]) by d28dlp03.in.ibm.com (Postfix) with ESMTP id 23A7B125804E for ; Thu, 14 Mar 2013 11:07:34 +0530 (IST) Received: from d28av02.in.ibm.com (d28av02.in.ibm.com [9.184.220.64]) by d28relay02.in.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r2E5aQn924838256 for ; Thu, 14 Mar 2013 11:06:27 +0530 Received: from d28av02.in.ibm.com (loopback [127.0.0.1]) by d28av02.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r2E5aSSL017172 for ; Thu, 14 Mar 2013 16:36:28 +1100 In-Reply-To: <20130314023953.GB18111@amt.cnet> Sender: kvm-owner@vger.kernel.org List-ID: On 03/14/2013 10:39 AM, Marcelo Tosatti wrote: > On Thu, Mar 14, 2013 at 11:26:41AM +0900, Takuya Yoshikawa wrote: >> On Wed, 13 Mar 2013 22:58:21 -0300 >> Marcelo Tosatti wrote: >> >>>>> In zap_spte, don't we need to search the pointer to be removed from the >>>>> global mmio-rmap list? How long can that list be? >>>> >>>> It is not bad. On softmmu, the rmap list has already been long more than 300. >>>> On hardmmu, normally the mmio spte is not frequently zapped (just set not clear). >> >> mmu_shrink() is an exception. >> >>>> >>>> The worst case is zap-all-mmio-spte that removes all mmio-spte. This operation >>>> can be speed up after applying my previous patch: >>>> KVM: MMU: fast drop all spte on the pte_list >> >> My point is other code may need to care more about latency. >> >> Zapping all mmio sptes can happen only when changing memory regions: >> not so latency severe but should be reasonably fast not to hold >> mmu_lock for a (too) long time. >> >> Compared to that, mmu_shrink() may be called any time and adding >> more work to it should be avoided IMO. It should return ASAP. Hmm? How frequently is of mmu_shrink? Well, it would be heavy sometimes, but is not the case on normal running. How many mmio shdow pages we got in the system? Not many, especially on the virtio supported guest. And, if it is a real problem, it is worthwhile to optimize it since it is more worse for normal page rmap on shadow mmu. I have a idea to avoid holding mmu-lock that i mentioned in the previous mail that is cache generation-number into mmio spte. When zap mmio spte is needed, we can just simply increase the global generation-number. > > Good point. > >> In general, we should try hard to keep ourselves from affecting >> unrelated code path for optimizing something. The global pte >> list is something which can affect many code paths in the future. >> >> >> So, I'm fine with trying mmio-rmap once we can actually measure >> very long mmu_lock hold time by traversing shadow pages. >> >> How about applying this first and then see the effect on big guests? > > Works for me. Xiao? Marcelo, I do not persist in it. ;)