From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59040) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Y0eO9-000524-KP for qemu-devel@nongnu.org; Mon, 15 Dec 2014 17:45:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Y0eNq-0000D6-4z for qemu-devel@nongnu.org; Mon, 15 Dec 2014 17:45:13 -0500 Received: from mail.choopa.net ([216.155.136.52]:32937) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Y0eNq-0000D0-1j for qemu-devel@nongnu.org; Mon, 15 Dec 2014 17:44:54 -0500 Received: from localhost (unknown [127.0.0.1]) by mail.choopa.net (iRedMail) with ESMTP id A95084AF484 for ; Mon, 15 Dec 2014 17:44:53 -0500 (EST) Received: from mail.choopa.net ([127.0.0.1]) by localhost (mail.choopa.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cWjgRQevEyiv for ; Mon, 15 Dec 2014 17:44:53 -0500 (EST) Received: from [10.5.5.117] (office-nat.choopa.net [208.167.225.40]) by mail.choopa.net (iRedMail) with ESMTPSA id 3821F4AF458 for ; Mon, 15 Dec 2014 17:44:53 -0500 (EST) Message-ID: <548F6465.8080706@gameservers.com> Date: Mon, 15 Dec 2014 17:44:53 -0500 From: Brian Rak MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Qemu-devel] Server 2012R2 guest poor performance List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org I'm trying to track down some CPU usage issues we're having with some virtual machines. Basically, when we run our application under KVM, we see about 2x the CPU usage that we would see if we were running it on bare metal. kvm_stat shows me the following: efer_reload 0 0 exits 120145461 54146 fpu_reload 23548805 10735 halt_exits 21716863 9688 halt_wakeup 21674988 9696 host_state_reload 30264874 14106 hypercalls 0 0 insn_emulation 4459351 2087 insn_emulation_fail 0 0 invlpg 0 0 io_exits 8608533 4178 irq_exits 4170903 1026 irq_injections 32551242 15819 irq_window 7779935 5332 I'm trying to figure out why I'm seeing such high numbers for VM exits, but I haven't been able to track down the cause yet. If I do: # echo 1 >/sys/kernel/debug/tracing/events/kvm/enable; cat /sys/kernel/debug/tracing/trace_pipe | head -100000 > trace.log ; echo 0 >/sys/kernel/debug/tracing/events/kvm/enable # cat trace.log | grep kvm_exit | awk '{print $7}' | sort | uniq -c 4518 EXCEPTION_NMI 427 EXTERNAL_INTERRUPT 4048 HLT 1870 IO_INSTRUCTION 6819 MSR_WRITE 188 PAUSE_INSTRUCTION 1756 PENDING_INTERRUPT 1775 TPR_BELOW_THRESHOLD I've uploaded a copy of the trace log to https://dl.dropboxusercontent.com/u/50986796/trace.log.gz What is causing the MSR_WRITE exits? Is there anything I can do to optimize this? I'm running: qemu 2.2.0 '-cpu Haswell,+invtsc,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -no-hpet' virtio storage + network Is there anything else that I can do to make this run more efficiently?