From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56231) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a7Igd-0001ah-Up for qemu-devel@nongnu.org; Fri, 11 Dec 2015 03:04:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a7Iga-0006Id-Lz for qemu-devel@nongnu.org; Fri, 11 Dec 2015 03:04:19 -0500 Received: from vps01.wiesinger.com ([46.36.37.179]:43640) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a7Iga-00060k-Al for qemu-devel@nongnu.org; Fri, 11 Dec 2015 03:04:16 -0500 Received: from wiesinger.com (wiesinger.com [62.178.19.14]) by vps01.wiesinger.com (Postfix) with ESMTPS id B49D89F283 for ; Fri, 11 Dec 2015 09:03:11 +0100 (CET) Received: from [192.168.2.151] (80-110-43-200.static.surfer.at [80.110.43.200]) (authenticated bits=0) by wiesinger.com (8.15.2/8.15.2) with ESMTPSA id tBB834Ul013172 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 11 Dec 2015 09:03:05 +0100 References: <5666A553.8070009@wiesinger.com> From: Gerhard Wiesinger Message-ID: <566A8338.1040208@wiesinger.com> Date: Fri, 11 Dec 2015 09:03:04 +0100 MIME-Version: 1.0 In-Reply-To: <5666A553.8070009@wiesinger.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] QEMU/KVM performance gets worser - high load - high interrupts - high context switches List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Any comments? Ciao, Gerhard On 08.12.2015 10:39, Gerhard Wiesinger wrote: > Hello, > > Yesterday I looked at my munin statistics on my KVM host and I swar > that performance gets worser: load is getting higher, interrupts are > getting higher and are high as well as context switches. VMs and > applications didn't change that way. > > You can find graphics at: http://www.wiesinger.com/tmp/kvm/ > Last spike I guess was upgrade from FC22 to FC23 or a kernel update. > And it was even lower on older versions > > For me it looks like the high interrupt load and context switches are > the root cause. Interrupts inside the VM are <100, so with 10 VMs I'm > expecting 1000+baseload => <2000, see statistics below. > > All VMs are virtio on disk/network except one (IDE/rtl8139). > > # Host as well as all guests (except 2 VMs): > uname -a > Linux kvm 4.2.6-301.fc23.x86_64 #1 SMP Fri Nov 20 22:22:41 UTC 2015 > x86_64 x86_64 x86_64 GNU/Linux > > qemu-system-x86-2.4.1-1.fc23.x86_64 > > Platform: > > All VMs have the pc-i440fx-2.4 profile (I upgraded yesterday from > pc-i440fx-2.3 without any change). > > Any ideas, anyone having same issues? > > Ciao, > Gerhard > > kvm: no VM running > r b swpd free buff cache si so bi bo in cs us sy > id wa st > 0 0 0 3308516 102408 3798568 0 0 0 12 197 679 0 > 0 99 0 0 > 0 0 0 3308516 102416 3798564 0 0 0 42 197 914 0 > 0 99 1 0 > 0 0 0 3308516 102416 3798568 0 0 0 0 190 791 0 > 0 100 0 0 > 2 0 0 3308484 102416 3798568 0 0 0 0 129 440 0 > 0 100 0 0 > > kvm: 2 VMs running > procs -----------memory---------- ---swap-- -----io---- -system-- > ------cpu----- > r b swpd free buff cache si so bi bo in cs us sy > id wa st > 1 0 0 2641464 103052 3814700 0 0 0 0 2715 5648 > 3 2 95 0 0 > 0 0 0 2641340 103052 3814700 0 0 0 0 2601 5555 > 1 2 97 0 0 > 1 0 0 2641308 103052 3814700 0 0 0 5 2687 5708 > 3 2 95 0 0 > 0 0 0 2640620 103060 3814628 0 0 0 30 2779 5756 > 4 3 93 1 0 > 0 0 0 2640644 103060 3814636 0 0 0 0 2436 5364 > 1 2 97 0 0 > 1 0 0 2640520 103060 3814636 0 0 0 119 2734 5975 > 3 2 95 0 0 > > kvm: all 10 VMs running > procs -----------memory---------- ---swap-- -----io---- -system-- > ------cpu----- > r b swpd free buff cache si so bi bo in cs us sy > id wa st > 1 0 0 60408 78892 3371984 0 0 0 85 9015 17357 > 4 9 87 0 0 > 2 0 0 60408 78892 3371968 0 0 0 47 9375 17797 > 9 9 82 0 0 > 0 0 0 60472 78892 3372092 0 0 40 60 8882 17343 > 4 8 86 1 0 > 1 0 0 60316 78892 3372080 0 0 0 59 8863 17517 > 4 8 87 0 0 > 0 0 0 59540 78900 3372092 0 0 0 55 9135 17796 > 8 9 81 1 0 > 0 0 0 59168 78900 3372112 0 0 0 51 8931 17484 > 4 9 87 0 0 > > cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 15 > model name : Intel(R) Core(TM)2 Quad CPU @ 2.66GHz > stepping : 7 > >