From mboxrd@z Thu Jan 1 00:00:00 1970 From: "David S. Ahern" Subject: Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3) Date: Sun, 18 May 2008 22:14:05 -0600 Message-ID: <4830FE8D.6010006@cisco.com> References: <48054518.3000104@cisco.com> <4805BCF1.6040605@qumranet.com> <4807BD53.6020304@cisco.com> <48085485.3090205@qumranet.com> <480C188F.3020101@cisco.com> <480C5C39.4040300@qumranet.com> <480E492B.3060500@cisco.com> <480EEDA0.3080209@qumranet.com> <480F546C.2030608@cisco.com> <481215DE.3000302@cisco.com> <20080428181550.GA3965@dmt> <4816617F.3080403@cisco.com> <4817F30C.6050308@cisco.com> <48184228.2020701@qumranet.com> <481876A9.1010806@cisco.com> <48187903.2070409@qumranet.com> <4826E744.1080107@qumranet.com> <4826F668.6030305@qumranet.com> <48290FC2.4070505@cisco.com> <48294272.5020801@qumranet.com> <482B4D29.7010202@cisco.com> <482C1633.5070302@qumranet.com> <482E5F9C.6000207@cisco.com> <482FCEE1.5040306@qumranet.com> <4830F90A.1020809@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org To: Avi Kivity Return-path: Received: from sj-iport-3.cisco.com ([171.71.176.72]:20797 "EHLO sj-iport-3.cisco.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751177AbYESEOQ (ORCPT ); Mon, 19 May 2008 00:14:16 -0400 In-Reply-To: <4830F90A.1020809@cisco.com> Sender: kvm-owner@vger.kernel.org List-ID: [resend to new list]. David S. Ahern wrote: > I was just digging through the sysstat history files, and I was not > imagining it: I did have an excellent overnight run on 5/13-5/14 with > your patch and the standard RHEL3U8 smp kernel in the guest. I have no > idea why I cannot get anywhere close to that again. I have updated quite > a few variables since then (such as going from 2.6.25-rc8 to 2.6.25.3 > kernel in the host), but backing them out (i.e., resetting the test to > my recollection of all the details of 5/14) has not helped. baffling and > frustrating. > > more in-line below. > > > Avi Kivity wrote: >> David S. Ahern wrote: >>> Avi Kivity wrote: >>> >>>> Okay, I committed the patch without the flood count == 5. >>>> >>>> >>> I've continued testing the RHEL3 guests with the flood count at 3, and I >>> am right back to where I started. With the patch and the flood count at >>> 3, I had 2 runs totaling around 24 hours that looked really good. Now, I >>> am back to square one. I guess the short of it is that I am not sure if >>> the patch resolves this issue or not. >>> >>> >> What about with the flood count at 5? Does it reliably improve >> performance? >> > > [dsa] No. I saw the same problem with the flood count at 5. The > attachment in the last email shows kvm_stat data during a kscand event. > The data was collected with the patch you posted. With the flood count > at 3 the mmu cache/flood counters are in the 18,000/sec and pte updates > at ~50,000/sec and writes at 70,000/sec. With the flood count at 5 > mmu_cache/flood drops to 0 and pte updates and writes both hit > 180,000+/second. In both cases these last for 30 seconds or more. I only > included data for the onset as it's pretty flat during the kscand activity. > >>> Also, in a prior e-mail I mentioned guest time advancing rapidly. I've >>> noticed that with the -no-kvm-pit option the guest time is much better >>> and typically stays within 3 seconds or so of the host, even through the >>> high kscand activity which is one instance of when I've noticed time >>> jumps with the kernel pit. Yes, this result has been repeatable through >>> 6 or so runs. :-) >>> >> Strange. The in-kernel PIT was supposed to improve accuracy. >> > > [dsa] I started a run with the RHEL4 guest 8 hours ago and it is showing > the same kind of success. With the in-kernel PIT, time in the guest > advanced ~120 seconds over real time after just 2 days of up time. With > the userspace PIT, time in the guest is behind real time by only 1 > second after 8 hours of uptime. Note that I am running the RHEL4.6 > kernel recompiled with HZ at 250 instead of the usual 1000. > > david >