From mboxrd@z Thu Jan 1 00:00:00 1970 From: Xiao Guangrong Subject: Re: [PATCH 0/2] Simplify RCU freeing of shadow pages Date: Mon, 23 Apr 2012 15:28:29 +0800 Message-ID: <4F95049D.2050304@linux.vnet.ibm.com> References: <1334852797-28167-1-git-send-email-avi@redhat.com> <4F90E08D.7000903@linux.vnet.ibm.com> <4F940673.1030407@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org, Marcelo Tosatti To: Avi Kivity Return-path: Received: from e28smtp05.in.ibm.com ([122.248.162.5]:52294 "EHLO e28smtp05.in.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753684Ab2DWH2h (ORCPT ); Mon, 23 Apr 2012 03:28:37 -0400 Received: from /spool/local by e28smtp05.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 23 Apr 2012 12:58:34 +0530 Received: from d28av05.in.ibm.com (d28av05.in.ibm.com [9.184.220.67]) by d28relay03.in.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q3N7SVrG3928194 for ; Mon, 23 Apr 2012 12:58:31 +0530 Received: from d28av05.in.ibm.com (loopback [127.0.0.1]) by d28av05.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q3NCx0Rm025804 for ; Mon, 23 Apr 2012 22:59:00 +1000 In-Reply-To: <4F940673.1030407@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On 04/22/2012 09:24 PM, Avi Kivity wrote: > On 04/20/2012 07:05 AM, Xiao Guangrong wrote: >> On 04/20/2012 12:26 AM, Avi Kivity wrote: >> >>> This patchset simplifies the freeing by RCU of mmu pages. >>> >>> Xiao, I'm sure you thought of always freeing by RCU. Why didn't you choose >>> this way? I saves a couple of atomics in the fast path. >>> >> >> >> Avi, we have discussed it last year: >> >> https://lkml.org/lkml/2011/6/29/177 > > Here are my own measurements (4 way guest on 24-way host): > > time make -j8 > > ept=0, with patch > > real 3m54.526s > user 11m7.074s > sys 3m29.678s > > real 3m53.806s > user 11m13.654s > sys 3m26.577s > > real 3m53.578s > user 11m13.491s > sys 3m26.095s > > real 3m53.956s > user 11m14.866s > sys 3m25.075s > > ept=1, with patch > > > real 2m36.510s > user 8m18.905s > sys 1m31.712s > > real 2m36.642s > user 8m18.706s > sys 1m31.898s > > real 2m36.952s > user 8m20.041s > sys 1m31.929s > > real 2m37.036s > user 8m20.204s > sys 1m31.947s > > ept=0, unpatched > > real 3m51.299s > user 11m23.770s > sys 3m28.049s > > real 3m52.982s > user 11m28.419s > sys 3m29.645s > > real 3m52.634s > user 11m27.325s > sys 3m29.752s > > real 3m53.696s > user 11m30.215s > sys 3m30.146s > > ept=1, unpatched > > real 2m37.134s > user 8m26.707s > sys 1m33.152s > > real 2m36.838s > user 8m26.255s > sys 1m33.078s > > real 2m36.826s > user 8m26.917s > sys 1m32.449s > > real 2m36.877s > user 8m26.226s > sys 1m32.074s > > > So there is a small degradation with ept=0 and an even smaller > improvement with ept=1. But certainly we should give more weight to > ept=1, and also factor in the code simplification. > I agree. >> I have optimized/simplified for "write flood" a lot, but, unfortunately, >> the zapping sp is still frequently. >> >> Maybe we can cache the zapped sp in a invalid_sp_list to reduce the >> frequency. > > To recover the time for re-initializing sp->spt? Batch free shadow page, for example, list 10 sp in the invalid_list, then just need one rcu_free to free them. > >> Or, we may use SLAB_DESTROY_BY_RCU to free the shadow page. > > IIUC a kmem_cache_alloc() can reuse an allocation freed using > SLAB_DESTROY_BY_RCU before a grace period has elapsed. > Yes. So we need a refcount out of mmu-lock, and see the shadow page is stil pointed the upper level (prevent not re-alloced). This way may make code more complex.