From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:54625) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TYmTK-0005Uq-TU for qemu-devel@nongnu.org; Wed, 14 Nov 2012 18:34:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TYmTH-0006Fw-RB for qemu-devel@nongnu.org; Wed, 14 Nov 2012 18:34:18 -0500 Received: from e23smtp09.au.ibm.com ([202.81.31.142]:37462) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TYmTH-0006Ff-9R for qemu-devel@nongnu.org; Wed, 14 Nov 2012 18:34:15 -0500 Received: from /spool/local by e23smtp09.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Nov 2012 09:30:06 +1000 Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id qAENNX4m55050448 for ; Thu, 15 Nov 2012 10:23:35 +1100 Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id qAENXuMC032722 for ; Thu, 15 Nov 2012 10:33:56 +1100 Message-ID: <50A42A5E.5070905@linux.vnet.ibm.com> Date: Thu, 15 Nov 2012 07:33:50 +0800 From: Xiao Guangrong MIME-Version: 1.0 References: <50978DFE.1000005@linux.vnet.ibm.com> <20121112231032.GB5798@amt.cnet> <20121114003350.d6e8ff85658fccbf41183f05@gmail.com> <20121114144410.GB7054@amt.cnet> In-Reply-To: <20121114144410.GB7054@amt.cnet> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH] KVM: MMU: lazily drop large spte List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Marcelo Tosatti Cc: kvm@vger.kernel.org, quintela@redhat.com, qemu-devel@nongnu.org, linux-kernel@vger.kernel.org, owasserm@redhat.com, avi@redhat.com, pbonzini@redhat.com, chegu_vinod@hp.com, Takuya Yoshikawa , yamahata@valinux.co.jp On 11/14/2012 10:44 PM, Marcelo Tosatti wrote: > On Wed, Nov 14, 2012 at 12:33:50AM +0900, Takuya Yoshikawa wrote: >> Ccing live migration developers who should be interested in this work, >> >> On Mon, 12 Nov 2012 21:10:32 -0200 >> Marcelo Tosatti wrote: >> >>> On Mon, Nov 05, 2012 at 05:59:26PM +0800, Xiao Guangrong wrote: >>>> Do not drop large spte until it can be insteaded by small pages so that >>>> the guest can happliy read memory through it >>>> >>>> The idea is from Avi: >>>> | As I mentioned before, write-protecting a large spte is a good idea, >>>> | since it moves some work from protect-time to fault-time, so it reduces >>>> | jitter. This removes the need for the return value. >>>> >>>> Signed-off-by: Xiao Guangrong >>>> --- >>>> arch/x86/kvm/mmu.c | 34 +++++++++------------------------- >>>> 1 files changed, 9 insertions(+), 25 deletions(-) >>> >>> Its likely that other 4k pages are mapped read-write in the 2mb range >>> covered by a read-only 2mb map. Therefore its not entirely useful to >>> map read-only. >>> >>> Can you measure an improvement with this change? >> >> What we discussed at KVM Forum last week was about the jitter we could >> measure right after starting live migration: both Isaku and Chegu reported >> such jitter. >> >> So if this patch reduces such jitter for some real workloads, by lazily >> dropping largepage mappings and saving read faults until that point, that >> would be very nice! >> >> But sadly, what they measured included interactions with the outside of the >> guest, and the main cause was due to the big QEMU lock problem, they guessed. >> The order is so different that an improvement by a kernel side effort may not >> be seen easily. >> >> FWIW: I am now changing the initial write protection by >> kvm_mmu_slot_remove_write_access() to rmap based as I proposed at KVM Forum. >> ftrace said that 1ms was improved to 250-350us by the change for 10GB guest. >> My code still drops largepage mappings, so the initial write protection time >> itself may not be a such big issue here, I think. >> >> Again, if we can eliminate read faults to such an extent that guests can see >> measurable improvement, that should be very nice! >> >> Any thoughts? >> >> Thanks, >> Takuya > > OK, makes sense. I'm worried about shadow / oos interactions > with large read-only mappings (trying to remember what was the > case exactly, it might be non-existant now). Marcelo, i guess commit 38187c830cab84daecb41169948467f1f19317e3 is what you mentioned, but i do not know how it can "Simplifies out of sync shadow." :(