From: Andrew Theurer <habanero@linux.vnet.ibm.com>
To: KVM list <kvm@vger.kernel.org>
Subject: Windows Server 2008 VM performance
Date: Tue, 02 Jun 2009 16:05:47 -0500 [thread overview]
Message-ID: <4A25942B.7010503@linux.vnet.ibm.com> (raw)
I've been looking at how KVM handles windows guests, and I am a little
concerned with the CPU overhead. My test case is as follows:
I am running 4 instances of a J2EE benchmark. Each instance needs one
application server and one DB server. 8 VMs in total are used.
I have the same App and DB software for Linux and Windows (and same
versions) so I can compare between Linux and Windows. I also have
another hypervisor which I can test both Windows and Linux VMs.
The host has EPT capable processors. VMs in KVM are backed with large
pages.
Test results:
Config CPU utilization
------ -----
KVM-85
Windows Server 2008 64-bit VMs 44.84
RedHat 5.3 w/ 2.6.29 64-bit VMs 24.56
Other-Hypervisor
Windows Server 2008 64-bit VMs 30.63
RedHat 5.3 w/ 2.6.18 64-bit VMs 27.13
-KVM running Windows VMs uses 46% more CPU than the Other-Hypervisor
-The Other-Hypervisor provides an optimized virtual network driver
-KVM results listed above did not use virtio_net or virtio_disk for
Windows, but do for Linux
-One extra KVM run (not listed above) was made with virtio_net for
Windows VMs but only reduced CPU by 2%
-Most of the CPU overhead could be attributed to the DB VMs, where there
is about 5 MB/sec writes per VM
-I don't have a virtio_block driver for Windows to test. Does one exist?
-All tests above had 2 vCPUS per VM
Here's a comparison of kvm_stat between Windows (run1) and Linux (run2):
run1 run2 run1/run2
---- ---- ---------
efer_relo: 0 0 1
exits : 1206880 121916 9.899
fpu_reloa: 210969 20863 10.112
halt_exit: 15092 13222 1.141
halt_wake: 14466 9294 1.556
host_stat: 211066 45117 4.678
hypercall: 0 0 1
insn_emul: 119582 38126 3.136
insn_emul: 0 0 1
invlpg : 0 0 1
io_exits : 131051 26349 4.974
irq_exits: 8128 12937 0.628
irq_injec: 29955 21825 1.373
irq_windo: 2504 2022 1.238
kvm_reque: 0 0 1
largepage: 1 64 0.009
mmio_exit: 59224 0 Inf
mmu_cache: 0 3 0.000
mmu_flood: 0 0 1
mmu_pde_z: 0 0 1
mmu_pte_u: 0 0 1
mmu_pte_w: 0 0 1
mmu_recyc: 0 0 1
mmu_shado: 0 0 1
mmu_unsyn: 0 0 1
mmu_unsyn: 0 0 1
nmi_injec: 0 0 1
nmi_windo: 0 0 1
pf_fixed : 1 67 0.009
pf_guest : 0 0 1
remote_tl: 0 0 1
request_n: 0 0 1
signal_ex: 0 0 1
tlb_flush: 220 14037 0.016
10x the number of exits, a problem?
I happened to try just one vCPU per VM for KVM/Windows VMs, and I was
surprised how much of a difference it made:
Config CPU utilization
------ -----
KVM-85
Windows Server 2008 64-bit VMs, 2 vCPU per VM 44.84
Windows Server 2008 64-bit VMs, 1 vCPU per VM 36.44
A 19% reduction in CPU utilization vs KVM/Windows-2vCPU! Does not
explain all the overhead (vs Other-Hypervisor, 2 vCPUs per VM) but, that
sure seems like a lot between 1 to 2 vCPUs for KVM/Windows-VMs. I have
not run with 1 vCPU per VM with Other-Hypervisor, but I will soon.
Anyway, I also collected kvm_stat for the 1 vCPU case, and here it is
compared to KVM/Linux VMs with 2 vCPUs:
run1 run2 run1/run2
---- ---- ---------
efer_relo: 0 0 1
exits : 1184471 121916 9.715
fpu_reloa: 192766 20863 9.240
halt_exit: 4697 13222 0.355
halt_wake: 4360 9294 0.469
host_stat: 192828 45117 4.274
hypercall: 0 0 1
insn_emul: 130487 38126 3.422
insn_emul: 0 0 1
invlpg : 0 0 1
io_exits : 114430 26349 4.343
irq_exits: 7075 12937 0.547
irq_injec: 29930 21825 1.371
irq_windo: 2391 2022 1.182
kvm_reque: 0 0 1
largepage: 0 64 0.001
mmio_exit: 69028 0 Inf
mmu_cache: 0 3 0.000
mmu_flood: 0 0 1
mmu_pde_z: 0 0 1
mmu_pte_u: 0 0 1
mmu_pte_w: 0 0 1
mmu_recyc: 0 0 1
mmu_shado: 0 0 1
mmu_unsyn: 0 0 1
mmu_unsyn: 0 0 1
nmi_injec: 0 0 1
nmi_windo: 0 0 1
pf_fixed : 0 67 0.001
pf_guest : 0 0 1
remote_tl: 0 0 1
request_n: 0 0 1
signal_ex: 0 0 1
tlb_flush: 124 14037 0.009
Still see the huge difference in vm_exits, so I guess not all is great yet.
So, what would be the best course of action for this? Is there a
virtio_block driver to test? Can we find the root cause of the exits
(is there a way to get stack dump or something that can show where there
are coming from)?
Thanks!
-Andrew
P.S. Here is the qemu cmd line for the windows VMs:
/usr/local/bin/qemu-system-x86_64 -name newcastle-xdbt01 -hda
/dev/disk/by-id/scsi-3600a0b80000f1eb10000074f4a02b08a -net
nic,model=e1000,vlan=0,macaddr=00:50:56:00:00:03 -m 1024 -mempath
/hugetlbfs -net tap,vlan=0,ifname=tap3,script=/etc/qemu-ifup -vnc
127.0.0.1:3 -smp 2 -daemonize
next reply other threads:[~2009-06-02 21:05 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-06-02 21:05 Andrew Theurer [this message]
2009-06-02 21:38 ` Windows Server 2008 VM performance Avi Kivity
2009-06-02 21:45 ` Javier Guerra
2009-06-02 21:48 ` Avi Kivity
2009-06-02 22:15 ` Ryan Harper
2009-06-03 14:21 ` Andrew Theurer
2009-06-03 17:38 ` Marcelo Tosatti
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4A25942B.7010503@linux.vnet.ibm.com \
--to=habanero@linux.vnet.ibm.com \
--cc=kvm@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.