* [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
@ 2012-02-20 18:17 Peter Lieven
2012-02-20 18:40 ` Gleb Natapov
0 siblings, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-20 18:17 UTC (permalink / raw)
To: qemu-devel, kvm
Hi,
I came a across an issue with a Windows 7 (32-bit) as well as with a
Windows 2008 R2 (64-bit) guest.
If I transfer a file from the VM via CIFS or FTP to a remote machine,
i get very poor read performance (around 13MB/s). The VM peaks at 100%
cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
efer_reload 0 0
exits 2260976 79620
fpu_reload 6197 11
halt_exits 114734 5011
halt_wakeup 111195 4876
host_state_reload 1499659 60962
hypercalls 0 0
insn_emulation 1577325 58488
insn_emulation_fail 0 0
invlpg 0 0
io_exits 943949 40249
irq_exits 108679 5434
irq_injections 236545 10788
irq_window 7606 246
largepages 672 5
mmio_exits 460020 16082
mmu_cache_miss 119 0
mmu_flooded 0 0
mmu_pde_zapped 0 0
mmu_pte_updated 0 0
mmu_pte_write 13474 9
mmu_recycled 0 0
mmu_shadow_zapped 141 0
mmu_unsync 0 0
nmi_injections 0 0
nmi_window 0 0
pf_fixed 22803 35
pf_guest 0 0
remote_tlb_flush 239 2
request_irq 0 0
signal_exits 0 0
tlb_flush 20933 0
If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
throughput. The kvm_stats look a lot more sane.
efer_reload 0 0
exits 6132004 17931
fpu_reload 19863 3
halt_exits 264961 3083
halt_wakeup 236468 2959
host_state_reload 1104468 3104
hypercalls 0 0
insn_emulation 1417443 7518
insn_emulation_fail 0 0
invlpg 0 0
io_exits 869380 2795
irq_exits 253501 2362
irq_injections 616967 6804
irq_window 201186 2161
largepages 1019 0
mmio_exits 205268 0
mmu_cache_miss 192 0
mmu_flooded 0 0
mmu_pde_zapped 0 0
mmu_pte_updated 0 0
mmu_pte_write 7440546 0
mmu_recycled 0 0
mmu_shadow_zapped 259 0
mmu_unsync 0 0
nmi_injections 0 0
nmi_window 0 0
pf_fixed 38529 30
pf_guest 0 0
remote_tlb_flush 761 1
request_irq 0 0
signal_exits 0 0
tlb_flush 0 0
I use virtio-net (with vhost-net) and virtio-blk. I tried disabling hpet
(which basically illiminated the mmio_exits, but does not increase
performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
) - no improvement.
My commandline:
/usr/bin/qemu-kvm-1.0 -netdev
type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on -device
virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive
format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
-m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
/var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu host
-rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
What further information is needed to debug this further?
Thanks,
Peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 18:17 [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists Peter Lieven
@ 2012-02-20 18:40 ` Gleb Natapov
2012-02-20 19:04 ` Gleb Natapov
2012-02-20 19:15 ` Peter Lieven
0 siblings, 2 replies; 19+ messages in thread
From: Gleb Natapov @ 2012-02-20 18:40 UTC (permalink / raw)
To: Peter Lieven; +Cc: qemu-devel, kvm
On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
> Hi,
>
> I came a across an issue with a Windows 7 (32-bit) as well as with a
> Windows 2008 R2 (64-bit) guest.
>
> If I transfer a file from the VM via CIFS or FTP to a remote machine,
> i get very poor read performance (around 13MB/s). The VM peaks at 100%
> cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
>
> efer_reload 0 0
> exits 2260976 79620
> fpu_reload 6197 11
> halt_exits 114734 5011
> halt_wakeup 111195 4876
> host_state_reload 1499659 60962
> hypercalls 0 0
> insn_emulation 1577325 58488
> insn_emulation_fail 0 0
> invlpg 0 0
> io_exits 943949 40249
Hmm, too many of those.
> irq_exits 108679 5434
> irq_injections 236545 10788
> irq_window 7606 246
> largepages 672 5
> mmio_exits 460020 16082
> mmu_cache_miss 119 0
> mmu_flooded 0 0
> mmu_pde_zapped 0 0
> mmu_pte_updated 0 0
> mmu_pte_write 13474 9
> mmu_recycled 0 0
> mmu_shadow_zapped 141 0
> mmu_unsync 0 0
> nmi_injections 0 0
> nmi_window 0 0
> pf_fixed 22803 35
> pf_guest 0 0
> remote_tlb_flush 239 2
> request_irq 0 0
> signal_exits 0 0
> tlb_flush 20933 0
>
> If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
> throughput. The kvm_stats look a lot more sane.
>
> efer_reload 0 0
> exits 6132004 17931
> fpu_reload 19863 3
> halt_exits 264961 3083
> halt_wakeup 236468 2959
> host_state_reload 1104468 3104
> hypercalls 0 0
> insn_emulation 1417443 7518
> insn_emulation_fail 0 0
> invlpg 0 0
> io_exits 869380 2795
> irq_exits 253501 2362
> irq_injections 616967 6804
> irq_window 201186 2161
> largepages 1019 0
> mmio_exits 205268 0
> mmu_cache_miss 192 0
> mmu_flooded 0 0
> mmu_pde_zapped 0 0
> mmu_pte_updated 0 0
> mmu_pte_write 7440546 0
> mmu_recycled 0 0
> mmu_shadow_zapped 259 0
> mmu_unsync 0 0
> nmi_injections 0 0
> nmi_window 0 0
> pf_fixed 38529 30
> pf_guest 0 0
> remote_tlb_flush 761 1
> request_irq 0 0
> signal_exits 0 0
> tlb_flush 0 0
>
> I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
> hpet (which basically illiminated the mmio_exits, but does not
> increase
> performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
> ) - no improvement.
>
> My commandline:
> /usr/bin/qemu-kvm-1.0 -netdev
> type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
> -device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
> -m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
> lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
> /var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
> host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
>
> What further information is needed to debug this further?
>
Which kernel version (looks like something recent)?
Which host CPU (looks like something old)?
Which Windows' virtio drivers are you using?
Take a trace like described here http://www.linux-kvm.org/page/Tracing
(with -no-hpet please).
Try to use -cpu host,+x2apic. It may help Linux guest performance.
--
Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 18:40 ` Gleb Natapov
@ 2012-02-20 19:04 ` Gleb Natapov
2012-02-20 19:24 ` Peter Lieven
2012-02-20 19:59 ` Peter Lieven
2012-02-20 19:15 ` Peter Lieven
1 sibling, 2 replies; 19+ messages in thread
From: Gleb Natapov @ 2012-02-20 19:04 UTC (permalink / raw)
To: Peter Lieven; +Cc: qemu-devel, kvm
On Mon, Feb 20, 2012 at 08:40:08PM +0200, Gleb Natapov wrote:
> On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
> > Hi,
> >
> > I came a across an issue with a Windows 7 (32-bit) as well as with a
> > Windows 2008 R2 (64-bit) guest.
> >
> > If I transfer a file from the VM via CIFS or FTP to a remote machine,
> > i get very poor read performance (around 13MB/s). The VM peaks at 100%
> > cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
> >
> > efer_reload 0 0
> > exits 2260976 79620
> > fpu_reload 6197 11
> > halt_exits 114734 5011
> > halt_wakeup 111195 4876
> > host_state_reload 1499659 60962
> > hypercalls 0 0
> > insn_emulation 1577325 58488
> > insn_emulation_fail 0 0
> > invlpg 0 0
> > io_exits 943949 40249
> Hmm, too many of those.
>
> > irq_exits 108679 5434
> > irq_injections 236545 10788
> > irq_window 7606 246
> > largepages 672 5
> > mmio_exits 460020 16082
> > mmu_cache_miss 119 0
> > mmu_flooded 0 0
> > mmu_pde_zapped 0 0
> > mmu_pte_updated 0 0
> > mmu_pte_write 13474 9
> > mmu_recycled 0 0
> > mmu_shadow_zapped 141 0
> > mmu_unsync 0 0
> > nmi_injections 0 0
> > nmi_window 0 0
> > pf_fixed 22803 35
> > pf_guest 0 0
> > remote_tlb_flush 239 2
> > request_irq 0 0
> > signal_exits 0 0
> > tlb_flush 20933 0
> >
> > If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
> > throughput. The kvm_stats look a lot more sane.
> >
> > efer_reload 0 0
> > exits 6132004 17931
> > fpu_reload 19863 3
> > halt_exits 264961 3083
> > halt_wakeup 236468 2959
> > host_state_reload 1104468 3104
> > hypercalls 0 0
> > insn_emulation 1417443 7518
> > insn_emulation_fail 0 0
> > invlpg 0 0
> > io_exits 869380 2795
> > irq_exits 253501 2362
> > irq_injections 616967 6804
> > irq_window 201186 2161
> > largepages 1019 0
> > mmio_exits 205268 0
> > mmu_cache_miss 192 0
> > mmu_flooded 0 0
> > mmu_pde_zapped 0 0
> > mmu_pte_updated 0 0
> > mmu_pte_write 7440546 0
> > mmu_recycled 0 0
> > mmu_shadow_zapped 259 0
> > mmu_unsync 0 0
> > nmi_injections 0 0
> > nmi_window 0 0
> > pf_fixed 38529 30
> > pf_guest 0 0
> > remote_tlb_flush 761 1
> > request_irq 0 0
> > signal_exits 0 0
> > tlb_flush 0 0
> >
> > I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
> > hpet (which basically illiminated the mmio_exits, but does not
> > increase
> > performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
> > ) - no improvement.
> >
> > My commandline:
> > /usr/bin/qemu-kvm-1.0 -netdev
> > type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
> > -device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
> > -m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
> > lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
> > /var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
> > host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
> >
> > What further information is needed to debug this further?
> >
> Which kernel version (looks like something recent)?
> Which host CPU (looks like something old)?
Output of cat /proc/cpuinfo
> Which Windows' virtio drivers are you using?
>
> Take a trace like described here http://www.linux-kvm.org/page/Tracing
> (with -no-hpet please).
>
And also "info pci" output from qemu monitor while we are at it.
> Try to use -cpu host,+x2apic. It may help Linux guest performance.
>
> --
> Gleb.
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 18:40 ` Gleb Natapov
2012-02-20 19:04 ` Gleb Natapov
@ 2012-02-20 19:15 ` Peter Lieven
2012-02-20 20:42 ` Gleb Natapov
1 sibling, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-20 19:15 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel, kvm
On 20.02.2012 19:40, Gleb Natapov wrote:
> On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
>> Hi,
>>
>> I came a across an issue with a Windows 7 (32-bit) as well as with a
>> Windows 2008 R2 (64-bit) guest.
>>
>> If I transfer a file from the VM via CIFS or FTP to a remote machine,
>> i get very poor read performance (around 13MB/s). The VM peaks at 100%
>> cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
>>
>> efer_reload 0 0
>> exits 2260976 79620
>> fpu_reload 6197 11
>> halt_exits 114734 5011
>> halt_wakeup 111195 4876
>> host_state_reload 1499659 60962
>> hypercalls 0 0
>> insn_emulation 1577325 58488
>> insn_emulation_fail 0 0
>> invlpg 0 0
>> io_exits 943949 40249
> Hmm, too many of those.
>
>> irq_exits 108679 5434
>> irq_injections 236545 10788
>> irq_window 7606 246
>> largepages 672 5
>> mmio_exits 460020 16082
>> mmu_cache_miss 119 0
>> mmu_flooded 0 0
>> mmu_pde_zapped 0 0
>> mmu_pte_updated 0 0
>> mmu_pte_write 13474 9
>> mmu_recycled 0 0
>> mmu_shadow_zapped 141 0
>> mmu_unsync 0 0
>> nmi_injections 0 0
>> nmi_window 0 0
>> pf_fixed 22803 35
>> pf_guest 0 0
>> remote_tlb_flush 239 2
>> request_irq 0 0
>> signal_exits 0 0
>> tlb_flush 20933 0
>>
>> If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
>> throughput. The kvm_stats look a lot more sane.
>>
>> efer_reload 0 0
>> exits 6132004 17931
>> fpu_reload 19863 3
>> halt_exits 264961 3083
>> halt_wakeup 236468 2959
>> host_state_reload 1104468 3104
>> hypercalls 0 0
>> insn_emulation 1417443 7518
>> insn_emulation_fail 0 0
>> invlpg 0 0
>> io_exits 869380 2795
>> irq_exits 253501 2362
>> irq_injections 616967 6804
>> irq_window 201186 2161
>> largepages 1019 0
>> mmio_exits 205268 0
>> mmu_cache_miss 192 0
>> mmu_flooded 0 0
>> mmu_pde_zapped 0 0
>> mmu_pte_updated 0 0
>> mmu_pte_write 7440546 0
>> mmu_recycled 0 0
>> mmu_shadow_zapped 259 0
>> mmu_unsync 0 0
>> nmi_injections 0 0
>> nmi_window 0 0
>> pf_fixed 38529 30
>> pf_guest 0 0
>> remote_tlb_flush 761 1
>> request_irq 0 0
>> signal_exits 0 0
>> tlb_flush 0 0
>>
>> I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
>> hpet (which basically illiminated the mmio_exits, but does not
>> increase
>> performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
>> ) - no improvement.
>>
>> My commandline:
>> /usr/bin/qemu-kvm-1.0 -netdev
>> type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
>> -device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
>> -m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
>> lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
>> /var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
>> host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
>>
>> What further information is needed to debug this further?
>>
> Which kernel version (looks like something recent)?
2.6.38 with kvm-kmod 3.2
> Which host CPU (looks like something old)?
why? i guess its (quite) new.
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU L5640 @ 2.27GHz
stepping : 2
cpu MHz : 1596.000
cache size : 12288 KB
physical id : 1
siblings : 6
core id : 10
cpu cores : 6
apicid : 52
initial apicid : 52
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx
smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm
arat dts tpr_shadow vnmi flexpriority ept vpid
bogomips : 2254.43
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
> Which Windows' virtio drivers are you using?
i used to use 0.1-16 and today also tried 0.1-22 from
http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/
> Take a trace like described here http://www.linux-kvm.org/page/Tracing
> (with -no-hpet please).
will prepare this.
> Try to use -cpu host,+x2apic. It may help Linux guest performance.
Thanks, it improved throughput a little while lowering the
cpu usage. Windows does not support this?
Thanks
Peter
> --
> Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 19:04 ` Gleb Natapov
@ 2012-02-20 19:24 ` Peter Lieven
2012-02-20 19:59 ` Peter Lieven
1 sibling, 0 replies; 19+ messages in thread
From: Peter Lieven @ 2012-02-20 19:24 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel, kvm
On 20.02.2012 20:04, Gleb Natapov wrote:
> On Mon, Feb 20, 2012 at 08:40:08PM +0200, Gleb Natapov wrote:
>> On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
>>> Hi,
>>>
>>> I came a across an issue with a Windows 7 (32-bit) as well as with a
>>> Windows 2008 R2 (64-bit) guest.
>>>
>>> If I transfer a file from the VM via CIFS or FTP to a remote machine,
>>> i get very poor read performance (around 13MB/s). The VM peaks at 100%
>>> cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
>>>
>>> efer_reload 0 0
>>> exits 2260976 79620
>>> fpu_reload 6197 11
>>> halt_exits 114734 5011
>>> halt_wakeup 111195 4876
>>> host_state_reload 1499659 60962
>>> hypercalls 0 0
>>> insn_emulation 1577325 58488
>>> insn_emulation_fail 0 0
>>> invlpg 0 0
>>> io_exits 943949 40249
>> Hmm, too many of those.
>>
>>> irq_exits 108679 5434
>>> irq_injections 236545 10788
>>> irq_window 7606 246
>>> largepages 672 5
>>> mmio_exits 460020 16082
>>> mmu_cache_miss 119 0
>>> mmu_flooded 0 0
>>> mmu_pde_zapped 0 0
>>> mmu_pte_updated 0 0
>>> mmu_pte_write 13474 9
>>> mmu_recycled 0 0
>>> mmu_shadow_zapped 141 0
>>> mmu_unsync 0 0
>>> nmi_injections 0 0
>>> nmi_window 0 0
>>> pf_fixed 22803 35
>>> pf_guest 0 0
>>> remote_tlb_flush 239 2
>>> request_irq 0 0
>>> signal_exits 0 0
>>> tlb_flush 20933 0
>>>
>>> If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
>>> throughput. The kvm_stats look a lot more sane.
>>>
>>> efer_reload 0 0
>>> exits 6132004 17931
>>> fpu_reload 19863 3
>>> halt_exits 264961 3083
>>> halt_wakeup 236468 2959
>>> host_state_reload 1104468 3104
>>> hypercalls 0 0
>>> insn_emulation 1417443 7518
>>> insn_emulation_fail 0 0
>>> invlpg 0 0
>>> io_exits 869380 2795
>>> irq_exits 253501 2362
>>> irq_injections 616967 6804
>>> irq_window 201186 2161
>>> largepages 1019 0
>>> mmio_exits 205268 0
>>> mmu_cache_miss 192 0
>>> mmu_flooded 0 0
>>> mmu_pde_zapped 0 0
>>> mmu_pte_updated 0 0
>>> mmu_pte_write 7440546 0
>>> mmu_recycled 0 0
>>> mmu_shadow_zapped 259 0
>>> mmu_unsync 0 0
>>> nmi_injections 0 0
>>> nmi_window 0 0
>>> pf_fixed 38529 30
>>> pf_guest 0 0
>>> remote_tlb_flush 761 1
>>> request_irq 0 0
>>> signal_exits 0 0
>>> tlb_flush 0 0
>>>
>>> I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
>>> hpet (which basically illiminated the mmio_exits, but does not
>>> increase
>>> performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
>>> ) - no improvement.
>>>
>>> My commandline:
>>> /usr/bin/qemu-kvm-1.0 -netdev
>>> type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
>>> -device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
>>> -m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
>>> lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
>>> /var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
>>> host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
>>>
>>> What further information is needed to debug this further?
>>>
>> Which kernel version (looks like something recent)?
>> Which host CPU (looks like something old)?
> Output of cat /proc/cpuinfo
>
>> Which Windows' virtio drivers are you using?
>>
>> Take a trace like described here http://www.linux-kvm.org/page/Tracing
>> (with -no-hpet please).
>>
> And also "info pci" output from qemu monitor while we are at it.
here we go. i stumbled across IRQ 0 ?
QEMU 1.0 monitor - type 'help' for more information
(qemu) info pci
info pci
Bus 0, device 0, function 0:
Host bridge: PCI device 8086:1237
id ""
Bus 0, device 1, function 0:
ISA bridge: PCI device 8086:7000
id ""
Bus 0, device 1, function 1:
IDE controller: PCI device 8086:7010
BAR4: I/O at 0xc080 [0xc08f].
id ""
Bus 0, device 1, function 2:
USB controller: PCI device 8086:7020
IRQ 5.
BAR4: I/O at 0xc040 [0xc05f].
id ""
Bus 0, device 1, function 3:
Bridge: PCI device 8086:7113
IRQ 9.
id ""
Bus 0, device 2, function 0:
VGA controller: PCI device 1234:1111
BAR0: 32 bit prefetchable memory at 0xfd000000 [0xfdffffff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 3, function 0:
Ethernet controller: PCI device 1af4:1000
IRQ 0.
BAR0: I/O at 0xc060 [0xc07f].
BAR1: 32 bit memory at 0xfebf0000 [0xfebf0fff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 4, function 0:
SCSI controller: PCI device 1af4:1001
IRQ 0.
BAR0: I/O at 0xc000 [0xc03f].
BAR1: 32 bit memory at 0xfebf1000 [0xfebf1fff].
id ""
>> Try to use -cpu host,+x2apic. It may help Linux guest performance.
>>
>> --
>> Gleb.
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 19:04 ` Gleb Natapov
2012-02-20 19:24 ` Peter Lieven
@ 2012-02-20 19:59 ` Peter Lieven
2012-02-20 20:45 ` Gleb Natapov
1 sibling, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-20 19:59 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel, kvm
On 20.02.2012 20:04, Gleb Natapov wrote:
> On Mon, Feb 20, 2012 at 08:40:08PM +0200, Gleb Natapov wrote:
>> On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
>>> Hi,
>>>
>>> I came a across an issue with a Windows 7 (32-bit) as well as with a
>>> Windows 2008 R2 (64-bit) guest.
>>>
>>> If I transfer a file from the VM via CIFS or FTP to a remote machine,
>>> i get very poor read performance (around 13MB/s). The VM peaks at 100%
>>> cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
>>>
>>> efer_reload 0 0
>>> exits 2260976 79620
>>> fpu_reload 6197 11
>>> halt_exits 114734 5011
>>> halt_wakeup 111195 4876
>>> host_state_reload 1499659 60962
>>> hypercalls 0 0
>>> insn_emulation 1577325 58488
>>> insn_emulation_fail 0 0
>>> invlpg 0 0
>>> io_exits 943949 40249
>> Hmm, too many of those.
>>
>>> irq_exits 108679 5434
>>> irq_injections 236545 10788
>>> irq_window 7606 246
>>> largepages 672 5
>>> mmio_exits 460020 16082
>>> mmu_cache_miss 119 0
>>> mmu_flooded 0 0
>>> mmu_pde_zapped 0 0
>>> mmu_pte_updated 0 0
>>> mmu_pte_write 13474 9
>>> mmu_recycled 0 0
>>> mmu_shadow_zapped 141 0
>>> mmu_unsync 0 0
>>> nmi_injections 0 0
>>> nmi_window 0 0
>>> pf_fixed 22803 35
>>> pf_guest 0 0
>>> remote_tlb_flush 239 2
>>> request_irq 0 0
>>> signal_exits 0 0
>>> tlb_flush 20933 0
>>>
>>> If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
>>> throughput. The kvm_stats look a lot more sane.
>>>
>>> efer_reload 0 0
>>> exits 6132004 17931
>>> fpu_reload 19863 3
>>> halt_exits 264961 3083
>>> halt_wakeup 236468 2959
>>> host_state_reload 1104468 3104
>>> hypercalls 0 0
>>> insn_emulation 1417443 7518
>>> insn_emulation_fail 0 0
>>> invlpg 0 0
>>> io_exits 869380 2795
>>> irq_exits 253501 2362
>>> irq_injections 616967 6804
>>> irq_window 201186 2161
>>> largepages 1019 0
>>> mmio_exits 205268 0
>>> mmu_cache_miss 192 0
>>> mmu_flooded 0 0
>>> mmu_pde_zapped 0 0
>>> mmu_pte_updated 0 0
>>> mmu_pte_write 7440546 0
>>> mmu_recycled 0 0
>>> mmu_shadow_zapped 259 0
>>> mmu_unsync 0 0
>>> nmi_injections 0 0
>>> nmi_window 0 0
>>> pf_fixed 38529 30
>>> pf_guest 0 0
>>> remote_tlb_flush 761 1
>>> request_irq 0 0
>>> signal_exits 0 0
>>> tlb_flush 0 0
>>>
>>> I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
>>> hpet (which basically illiminated the mmio_exits, but does not
>>> increase
>>> performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
>>> ) - no improvement.
>>>
>>> My commandline:
>>> /usr/bin/qemu-kvm-1.0 -netdev
>>> type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
>>> -device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
>>> -m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
>>> lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
>>> /var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
>>> host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
>>>
>>> What further information is needed to debug this further?
>>>
>> Which kernel version (looks like something recent)?
>> Which host CPU (looks like something old)?
> Output of cat /proc/cpuinfo
>
>> Which Windows' virtio drivers are you using?
>>
>> Take a trace like described here http://www.linux-kvm.org/page/Tracing
>> (with -no-hpet please).
>>
> And also "info pci" output from qemu monitor while we are at it.
here is the output while i was tracing. you can download the trace
i took while i did a ftp transfer from the vm:
-> http://82.141.21.156/report.txt.gz
QEMU 1.0 monitor - type 'help' for more information
(qemu) info pci
info pci
Bus 0, device 0, function 0:
Host bridge: PCI device 8086:1237
id ""
Bus 0, device 1, function 0:
ISA bridge: PCI device 8086:7000
id ""
Bus 0, device 1, function 1:
IDE controller: PCI device 8086:7010
BAR4: I/O at 0xc080 [0xc08f].
id ""
Bus 0, device 1, function 2:
USB controller: PCI device 8086:7020
IRQ 5.
BAR4: I/O at 0xc040 [0xc05f].
id ""
Bus 0, device 1, function 3:
Bridge: PCI device 8086:7113
IRQ 9.
id ""
Bus 0, device 2, function 0:
VGA controller: PCI device 1234:1111
BAR0: 32 bit prefetchable memory at 0xfd000000 [0xfdffffff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 3, function 0:
Ethernet controller: PCI device 1af4:1000
IRQ 0.
BAR0: I/O at 0xc060 [0xc07f].
BAR1: 32 bit memory at 0xfebf0000 [0xfebf0fff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 4, function 0:
SCSI controller: PCI device 1af4:1001
IRQ 0.
BAR0: I/O at 0xc000 [0xc03f].
BAR1: 32 bit memory at 0xfebf1000 [0xfebf1fff].
id ""
thanks for your help,
peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 19:15 ` Peter Lieven
@ 2012-02-20 20:42 ` Gleb Natapov
0 siblings, 0 replies; 19+ messages in thread
From: Gleb Natapov @ 2012-02-20 20:42 UTC (permalink / raw)
To: Peter Lieven; +Cc: qemu-devel, kvm
On Mon, Feb 20, 2012 at 08:15:15PM +0100, Peter Lieven wrote:
> On 20.02.2012 19:40, Gleb Natapov wrote:
> >On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
> >>Hi,
> >>
> >>I came a across an issue with a Windows 7 (32-bit) as well as with a
> >>Windows 2008 R2 (64-bit) guest.
> >>
> >>If I transfer a file from the VM via CIFS or FTP to a remote machine,
> >>i get very poor read performance (around 13MB/s). The VM peaks at 100%
> >>cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
> >>
> >>efer_reload 0 0
> >>exits 2260976 79620
> >>fpu_reload 6197 11
> >>halt_exits 114734 5011
> >>halt_wakeup 111195 4876
> >>host_state_reload 1499659 60962
> >>hypercalls 0 0
> >>insn_emulation 1577325 58488
> >>insn_emulation_fail 0 0
> >>invlpg 0 0
> >>io_exits 943949 40249
> >Hmm, too many of those.
> >
> >>irq_exits 108679 5434
> >>irq_injections 236545 10788
> >>irq_window 7606 246
> >>largepages 672 5
> >>mmio_exits 460020 16082
> >>mmu_cache_miss 119 0
> >>mmu_flooded 0 0
> >>mmu_pde_zapped 0 0
> >>mmu_pte_updated 0 0
> >>mmu_pte_write 13474 9
> >>mmu_recycled 0 0
> >>mmu_shadow_zapped 141 0
> >>mmu_unsync 0 0
> >>nmi_injections 0 0
> >>nmi_window 0 0
> >>pf_fixed 22803 35
> >>pf_guest 0 0
> >>remote_tlb_flush 239 2
> >>request_irq 0 0
> >>signal_exits 0 0
> >>tlb_flush 20933 0
> >>
> >>If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
> >>throughput. The kvm_stats look a lot more sane.
> >>
> >>efer_reload 0 0
> >>exits 6132004 17931
> >>fpu_reload 19863 3
> >>halt_exits 264961 3083
> >>halt_wakeup 236468 2959
> >>host_state_reload 1104468 3104
> >>hypercalls 0 0
> >>insn_emulation 1417443 7518
> >>insn_emulation_fail 0 0
> >>invlpg 0 0
> >>io_exits 869380 2795
> >>irq_exits 253501 2362
> >>irq_injections 616967 6804
> >>irq_window 201186 2161
> >>largepages 1019 0
> >>mmio_exits 205268 0
> >>mmu_cache_miss 192 0
> >>mmu_flooded 0 0
> >>mmu_pde_zapped 0 0
> >>mmu_pte_updated 0 0
> >>mmu_pte_write 7440546 0
> >>mmu_recycled 0 0
> >>mmu_shadow_zapped 259 0
> >>mmu_unsync 0 0
> >>nmi_injections 0 0
> >>nmi_window 0 0
> >>pf_fixed 38529 30
> >>pf_guest 0 0
> >>remote_tlb_flush 761 1
> >>request_irq 0 0
> >>signal_exits 0 0
> >>tlb_flush 0 0
> >>
> >>I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
> >>hpet (which basically illiminated the mmio_exits, but does not
> >>increase
> >>performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
> >>) - no improvement.
> >>
> >>My commandline:
> >>/usr/bin/qemu-kvm-1.0 -netdev
> >>type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
> >>-device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
> >>-m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
> >>lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
> >>/var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
> >>host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
> >>
> >>What further information is needed to debug this further?
> >>
> >Which kernel version (looks like something recent)?
> 2.6.38 with kvm-kmod 3.2
> >Which host CPU (looks like something old)?
> why? i guess its (quite) new.
>
Yes, it is. I misread the kvm_stat data and thought that CPU does not
have ept, but it does.
> vendor_id : GenuineIntel
> cpu family : 6
> model : 44
> model name : Intel(R) Xeon(R) CPU L5640 @ 2.27GHz
> stepping : 2
> cpu MHz : 1596.000
> cache size : 12288 KB
> physical id : 1
> siblings : 6
> core id : 10
> cpu cores : 6
> apicid : 52
> initial apicid : 52
> fpu : yes
> fpu_exception : yes
> cpuid level : 11
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts
> rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64
> monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1
> sse4_2 popcnt aes lahf_lm arat dts tpr_shadow vnmi flexpriority ept
> vpid
> bogomips : 2254.43
> clflush size : 64
> cache_alignment : 64
> address sizes : 40 bits physical, 48 bits virtual
>
> >Which Windows' virtio drivers are you using?
> i used to use 0.1-16 and today also tried 0.1-22 from
> http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/
> >Take a trace like described here http://www.linux-kvm.org/page/Tracing
> >(with -no-hpet please).
> will prepare this.
> >Try to use -cpu host,+x2apic. It may help Linux guest performance.
> Thanks, it improved throughput a little while lowering the
> cpu usage. Windows does not support this?
>
Not yet. It will support something similar when qemu will have hyper-v
support.
--
Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 19:59 ` Peter Lieven
@ 2012-02-20 20:45 ` Gleb Natapov
2012-02-21 10:50 ` Peter Lieven
0 siblings, 1 reply; 19+ messages in thread
From: Gleb Natapov @ 2012-02-20 20:45 UTC (permalink / raw)
To: Peter Lieven; +Cc: qemu-devel, kvm
On Mon, Feb 20, 2012 at 08:59:38PM +0100, Peter Lieven wrote:
> On 20.02.2012 20:04, Gleb Natapov wrote:
> >On Mon, Feb 20, 2012 at 08:40:08PM +0200, Gleb Natapov wrote:
> >>On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
> >>>Hi,
> >>>
> >>>I came a across an issue with a Windows 7 (32-bit) as well as with a
> >>>Windows 2008 R2 (64-bit) guest.
> >>>
> >>>If I transfer a file from the VM via CIFS or FTP to a remote machine,
> >>>i get very poor read performance (around 13MB/s). The VM peaks at 100%
> >>>cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
> >>>
> >>>efer_reload 0 0
> >>>exits 2260976 79620
> >>>fpu_reload 6197 11
> >>>halt_exits 114734 5011
> >>>halt_wakeup 111195 4876
> >>>host_state_reload 1499659 60962
> >>>hypercalls 0 0
> >>>insn_emulation 1577325 58488
> >>>insn_emulation_fail 0 0
> >>>invlpg 0 0
> >>>io_exits 943949 40249
> >>Hmm, too many of those.
> >>
> >>>irq_exits 108679 5434
> >>>irq_injections 236545 10788
> >>>irq_window 7606 246
> >>>largepages 672 5
> >>>mmio_exits 460020 16082
> >>>mmu_cache_miss 119 0
> >>>mmu_flooded 0 0
> >>>mmu_pde_zapped 0 0
> >>>mmu_pte_updated 0 0
> >>>mmu_pte_write 13474 9
> >>>mmu_recycled 0 0
> >>>mmu_shadow_zapped 141 0
> >>>mmu_unsync 0 0
> >>>nmi_injections 0 0
> >>>nmi_window 0 0
> >>>pf_fixed 22803 35
> >>>pf_guest 0 0
> >>>remote_tlb_flush 239 2
> >>>request_irq 0 0
> >>>signal_exits 0 0
> >>>tlb_flush 20933 0
> >>>
> >>>If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
> >>>throughput. The kvm_stats look a lot more sane.
> >>>
> >>>efer_reload 0 0
> >>>exits 6132004 17931
> >>>fpu_reload 19863 3
> >>>halt_exits 264961 3083
> >>>halt_wakeup 236468 2959
> >>>host_state_reload 1104468 3104
> >>>hypercalls 0 0
> >>>insn_emulation 1417443 7518
> >>>insn_emulation_fail 0 0
> >>>invlpg 0 0
> >>>io_exits 869380 2795
> >>>irq_exits 253501 2362
> >>>irq_injections 616967 6804
> >>>irq_window 201186 2161
> >>>largepages 1019 0
> >>>mmio_exits 205268 0
> >>>mmu_cache_miss 192 0
> >>>mmu_flooded 0 0
> >>>mmu_pde_zapped 0 0
> >>>mmu_pte_updated 0 0
> >>>mmu_pte_write 7440546 0
> >>>mmu_recycled 0 0
> >>>mmu_shadow_zapped 259 0
> >>>mmu_unsync 0 0
> >>>nmi_injections 0 0
> >>>nmi_window 0 0
> >>>pf_fixed 38529 30
> >>>pf_guest 0 0
> >>>remote_tlb_flush 761 1
> >>>request_irq 0 0
> >>>signal_exits 0 0
> >>>tlb_flush 0 0
> >>>
> >>>I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
> >>>hpet (which basically illiminated the mmio_exits, but does not
> >>>increase
> >>>performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
> >>>) - no improvement.
> >>>
> >>>My commandline:
> >>>/usr/bin/qemu-kvm-1.0 -netdev
> >>>type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
> >>>-device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
> >>>-m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
> >>>lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
> >>>/var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
> >>>host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
> >>>
> >>>What further information is needed to debug this further?
> >>>
> >>Which kernel version (looks like something recent)?
> >>Which host CPU (looks like something old)?
> >Output of cat /proc/cpuinfo
> >
> >>Which Windows' virtio drivers are you using?
> >>
> >>Take a trace like described here http://www.linux-kvm.org/page/Tracing
> >>(with -no-hpet please).
> >>
> >And also "info pci" output from qemu monitor while we are at it.
> here is the output while i was tracing. you can download the trace
> i took while i did a ftp transfer from the vm:
>
> -> http://82.141.21.156/report.txt.gz
>
Windows reads PM timer. A lot. 15152 times per second.
Can you try to run this command in Windows guest:
bcdedit /set {default} useplatformclock false
I hope it will make Windows use TSC instead, but you can't be sure
about anything with Windows :(
> QEMU 1.0 monitor - type 'help' for more information
> (qemu) info pci
> info pci
> Bus 0, device 0, function 0:
> Host bridge: PCI device 8086:1237
> id ""
> Bus 0, device 1, function 0:
> ISA bridge: PCI device 8086:7000
> id ""
> Bus 0, device 1, function 1:
> IDE controller: PCI device 8086:7010
> BAR4: I/O at 0xc080 [0xc08f].
> id ""
> Bus 0, device 1, function 2:
> USB controller: PCI device 8086:7020
> IRQ 5.
> BAR4: I/O at 0xc040 [0xc05f].
> id ""
> Bus 0, device 1, function 3:
> Bridge: PCI device 8086:7113
> IRQ 9.
> id ""
> Bus 0, device 2, function 0:
> VGA controller: PCI device 1234:1111
> BAR0: 32 bit prefetchable memory at 0xfd000000 [0xfdffffff].
> BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
> id ""
> Bus 0, device 3, function 0:
> Ethernet controller: PCI device 1af4:1000
> IRQ 0.
> BAR0: I/O at 0xc060 [0xc07f].
> BAR1: 32 bit memory at 0xfebf0000 [0xfebf0fff].
> BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
> id ""
> Bus 0, device 4, function 0:
> SCSI controller: PCI device 1af4:1001
> IRQ 0.
> BAR0: I/O at 0xc000 [0xc03f].
> BAR1: 32 bit memory at 0xfebf1000 [0xfebf1fff].
> id ""
>
> thanks for your help,
> peter
--
Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-20 20:45 ` Gleb Natapov
@ 2012-02-21 10:50 ` Peter Lieven
2012-02-21 10:56 ` Gleb Natapov
0 siblings, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-21 10:50 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel, kvm
On 20.02.2012 21:45, Gleb Natapov wrote:
> On Mon, Feb 20, 2012 at 08:59:38PM +0100, Peter Lieven wrote:
>> On 20.02.2012 20:04, Gleb Natapov wrote:
>>> On Mon, Feb 20, 2012 at 08:40:08PM +0200, Gleb Natapov wrote:
>>>> On Mon, Feb 20, 2012 at 07:17:55PM +0100, Peter Lieven wrote:
>>>>> Hi,
>>>>>
>>>>> I came a across an issue with a Windows 7 (32-bit) as well as with a
>>>>> Windows 2008 R2 (64-bit) guest.
>>>>>
>>>>> If I transfer a file from the VM via CIFS or FTP to a remote machine,
>>>>> i get very poor read performance (around 13MB/s). The VM peaks at 100%
>>>>> cpu and I see a lot of insn_emulations and all kinds of exists in kvm_stat
>>>>>
>>>>> efer_reload 0 0
>>>>> exits 2260976 79620
>>>>> fpu_reload 6197 11
>>>>> halt_exits 114734 5011
>>>>> halt_wakeup 111195 4876
>>>>> host_state_reload 1499659 60962
>>>>> hypercalls 0 0
>>>>> insn_emulation 1577325 58488
>>>>> insn_emulation_fail 0 0
>>>>> invlpg 0 0
>>>>> io_exits 943949 40249
>>>> Hmm, too many of those.
>>>>
>>>>> irq_exits 108679 5434
>>>>> irq_injections 236545 10788
>>>>> irq_window 7606 246
>>>>> largepages 672 5
>>>>> mmio_exits 460020 16082
>>>>> mmu_cache_miss 119 0
>>>>> mmu_flooded 0 0
>>>>> mmu_pde_zapped 0 0
>>>>> mmu_pte_updated 0 0
>>>>> mmu_pte_write 13474 9
>>>>> mmu_recycled 0 0
>>>>> mmu_shadow_zapped 141 0
>>>>> mmu_unsync 0 0
>>>>> nmi_injections 0 0
>>>>> nmi_window 0 0
>>>>> pf_fixed 22803 35
>>>>> pf_guest 0 0
>>>>> remote_tlb_flush 239 2
>>>>> request_irq 0 0
>>>>> signal_exits 0 0
>>>>> tlb_flush 20933 0
>>>>>
>>>>> If I run the same VM with a Ubuntu 10.04.4 guest I get around 60MB/s
>>>>> throughput. The kvm_stats look a lot more sane.
>>>>>
>>>>> efer_reload 0 0
>>>>> exits 6132004 17931
>>>>> fpu_reload 19863 3
>>>>> halt_exits 264961 3083
>>>>> halt_wakeup 236468 2959
>>>>> host_state_reload 1104468 3104
>>>>> hypercalls 0 0
>>>>> insn_emulation 1417443 7518
>>>>> insn_emulation_fail 0 0
>>>>> invlpg 0 0
>>>>> io_exits 869380 2795
>>>>> irq_exits 253501 2362
>>>>> irq_injections 616967 6804
>>>>> irq_window 201186 2161
>>>>> largepages 1019 0
>>>>> mmio_exits 205268 0
>>>>> mmu_cache_miss 192 0
>>>>> mmu_flooded 0 0
>>>>> mmu_pde_zapped 0 0
>>>>> mmu_pte_updated 0 0
>>>>> mmu_pte_write 7440546 0
>>>>> mmu_recycled 0 0
>>>>> mmu_shadow_zapped 259 0
>>>>> mmu_unsync 0 0
>>>>> nmi_injections 0 0
>>>>> nmi_window 0 0
>>>>> pf_fixed 38529 30
>>>>> pf_guest 0 0
>>>>> remote_tlb_flush 761 1
>>>>> request_irq 0 0
>>>>> signal_exits 0 0
>>>>> tlb_flush 0 0
>>>>>
>>>>> I use virtio-net (with vhost-net) and virtio-blk. I tried disabling
>>>>> hpet (which basically illiminated the mmio_exits, but does not
>>>>> increase
>>>>> performance) and also commit (39a7a362e16bb27e98738d63f24d1ab5811e26a8
>>>>> ) - no improvement.
>>>>>
>>>>> My commandline:
>>>>> /usr/bin/qemu-kvm-1.0 -netdev
>>>>> type=tap,id=guest8,script=no,downscript=no,ifname=tap0,vhost=on
>>>>> -device virtio-net-pci,netdev=guest8,mac=52:54:00:ff:00:d3 -drive format=host_device,file=/dev/mapper/iqn.2001-05.com.equallogic:0-8a0906-eeef4e007-a8a9f3818674f2fc-lieven-windows7-vc-r80788,if=virtio,cache=none,aio=native
>>>>> -m 2048 -smp 2 -monitor tcp:0:4001,server,nowait -vnc :1 -name
>>>>> lieven-win7-vc -boot order=dc,menu=off -k de -pidfile
>>>>> /var/run/qemu/vm-187.pid -mem-path /hugepages -mem-prealloc -cpu
>>>>> host -rtc base=localtime -vga std -usb -usbdevice tablet -no-hpet
>>>>>
>>>>> What further information is needed to debug this further?
>>>>>
>>>> Which kernel version (looks like something recent)?
>>>> Which host CPU (looks like something old)?
>>> Output of cat /proc/cpuinfo
>>>
>>>> Which Windows' virtio drivers are you using?
>>>>
>>>> Take a trace like described here http://www.linux-kvm.org/page/Tracing
>>>> (with -no-hpet please).
>>>>
>>> And also "info pci" output from qemu monitor while we are at it.
>> here is the output while i was tracing. you can download the trace
>> i took while i did a ftp transfer from the vm:
>>
>> -> http://82.141.21.156/report.txt.gz
>>
> Windows reads PM timer. A lot. 15152 times per second.
>
> Can you try to run this command in Windows guest:
>
> bcdedit /set {default} useplatformclock false
>
> I hope it will make Windows use TSC instead, but you can't be sure
> about anything with Windows :(
Whatever it does now it eates more CPU has almost equal
number of exits and throughput is about the same (15MB/s).
If pmtimer is at 0xb008 it still reads it like hell.
I checked with bcedit /v that useplatformclock is set to "No".
I still wonder why both virtio devices are on IRQ0 ?
New Trace:
http://82.141.21.156/report2.txt.gz
efer_reload 0 0
exits 1510993 59343
fpu_reload 6729 10
halt_exits 93603 5913
halt_wakeup 95698 5849
host_state_reload 738523 24727
hypercalls 0 0
insn_emulation 678416 20107
insn_emulation_fail 0 0
invlpg 0 0
io_exits 703291 28436
irq_exits 102117 7527
irq_injections 217335 14344
irq_window 9926 650
largepages 573 8
mmio_exits 27 0
mmu_cache_miss 148 0
mmu_flooded 0 0
mmu_pde_zapped 0 0
mmu_pte_updated 0 0
mmu_pte_write 0 0
mmu_recycled 0 0
mmu_shadow_zapped 190 0
mmu_unsync 0 0
nmi_injections 0 0
nmi_window 0 0
pf_fixed 21938 38
pf_guest 0 0
remote_tlb_flush 20 0
request_irq 0 0
signal_exits 0 0
tlb_flush 11711 0
QEMU 1.0 monitor - type 'help' for more information
(qemu) info pci
info pci
Bus 0, device 0, function 0:
Host bridge: PCI device 8086:1237
id ""
Bus 0, device 1, function 0:
ISA bridge: PCI device 8086:7000
id ""
Bus 0, device 1, function 1:
IDE controller: PCI device 8086:7010
BAR4: I/O at 0xc080 [0xc08f].
id ""
Bus 0, device 1, function 2:
USB controller: PCI device 8086:7020
IRQ 5.
BAR4: I/O at 0xc040 [0xc05f].
id ""
Bus 0, device 1, function 3:
Bridge: PCI device 8086:7113
IRQ 9.
id ""
Bus 0, device 2, function 0:
VGA controller: PCI device 1234:1111
BAR0: 32 bit prefetchable memory at 0xfd000000 [0xfdffffff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 3, function 0:
Ethernet controller: PCI device 1af4:1000
IRQ 0.
BAR0: I/O at 0xc060 [0xc07f].
BAR1: 32 bit memory at 0xfebf0000 [0xfebf0fff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 4, function 0:
SCSI controller: PCI device 1af4:1001
IRQ 0.
BAR0: I/O at 0xc000 [0xc03f].
BAR1: 32 bit memory at 0xfebf1000 [0xfebf1fff].
id ""
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 10:50 ` Peter Lieven
@ 2012-02-21 10:56 ` Gleb Natapov
2012-02-21 10:59 ` Peter Lieven
0 siblings, 1 reply; 19+ messages in thread
From: Gleb Natapov @ 2012-02-21 10:56 UTC (permalink / raw)
To: Peter Lieven; +Cc: qemu-devel, kvm, vrozenfe
On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
> >I hope it will make Windows use TSC instead, but you can't be sure
> >about anything with Windows :(
> Whatever it does now it eates more CPU has almost equal
> number of exits and throughput is about the same (15MB/s).
> If pmtimer is at 0xb008 it still reads it like hell.
>
> I checked with bcedit /v that useplatformclock is set to "No".
Yeah, today I noticed that it is likely virtio drivers that hammer
on PM timer (at least rip of the instruction that access it is
very close to rip of the instruction that access virtio pio).
Vadim, Windows driver developer, is CCed.
>
> I still wonder why both virtio devices are on IRQ0 ?
>
They use MSI like they should.
--
Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 10:56 ` Gleb Natapov
@ 2012-02-21 10:59 ` Peter Lieven
2012-02-21 11:00 ` Gleb Natapov
0 siblings, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-21 10:59 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel, kvm, vrozenfe
On 21.02.2012 11:56, Gleb Natapov wrote:
> On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
>>> I hope it will make Windows use TSC instead, but you can't be sure
>>> about anything with Windows :(
>> Whatever it does now it eates more CPU has almost equal
>> number of exits and throughput is about the same (15MB/s).
>> If pmtimer is at 0xb008 it still reads it like hell.
>>
>> I checked with bcedit /v that useplatformclock is set to "No".
> Yeah, today I noticed that it is likely virtio drivers that hammer
> on PM timer (at least rip of the instruction that access it is
> very close to rip of the instruction that access virtio pio).
> Vadim, Windows driver developer, is CCed.
Ok, I will switch to IDE and e1000 to confirm this? Or does it not make
sense?
Peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 10:59 ` Peter Lieven
@ 2012-02-21 11:00 ` Gleb Natapov
2012-02-21 11:16 ` Peter Lieven
0 siblings, 1 reply; 19+ messages in thread
From: Gleb Natapov @ 2012-02-21 11:00 UTC (permalink / raw)
To: Peter Lieven; +Cc: qemu-devel, kvm, vrozenfe
On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
> On 21.02.2012 11:56, Gleb Natapov wrote:
> >On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
> >>>I hope it will make Windows use TSC instead, but you can't be sure
> >>>about anything with Windows :(
> >>Whatever it does now it eates more CPU has almost equal
> >>number of exits and throughput is about the same (15MB/s).
> >>If pmtimer is at 0xb008 it still reads it like hell.
> >>
> >>I checked with bcedit /v that useplatformclock is set to "No".
> >Yeah, today I noticed that it is likely virtio drivers that hammer
> >on PM timer (at least rip of the instruction that access it is
> >very close to rip of the instruction that access virtio pio).
> >Vadim, Windows driver developer, is CCed.
> Ok, I will switch to IDE and e1000 to confirm this? Or does it not
> make sense?
>
It make perfect sense! Please try it.
--
Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 11:00 ` Gleb Natapov
@ 2012-02-21 11:16 ` Peter Lieven
2012-02-21 11:46 ` Gleb Natapov
0 siblings, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-21 11:16 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel, kvm, vrozenfe
On 21.02.2012 12:00, Gleb Natapov wrote:
> On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
>> On 21.02.2012 11:56, Gleb Natapov wrote:
>>> On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
>>>>> I hope it will make Windows use TSC instead, but you can't be sure
>>>>> about anything with Windows :(
>>>> Whatever it does now it eates more CPU has almost equal
>>>> number of exits and throughput is about the same (15MB/s).
>>>> If pmtimer is at 0xb008 it still reads it like hell.
>>>>
>>>> I checked with bcedit /v that useplatformclock is set to "No".
>>> Yeah, today I noticed that it is likely virtio drivers that hammer
>>> on PM timer (at least rip of the instruction that access it is
>>> very close to rip of the instruction that access virtio pio).
>>> Vadim, Windows driver developer, is CCed.
>> Ok, I will switch to IDE and e1000 to confirm this? Or does it not
>> make sense?
>>
> It make perfect sense! Please try it.
~10MB/s. still a lot of 0xb008 reads.
efer_reload 0 0
exits 4389875 72341
fpu_reload 36729 342
halt_exits 206204 3451
halt_wakeup 212953 3474
host_state_reload 2976799 59043
hypercalls 0 0
insn_emulation 2936091 54921
insn_emulation_fail 0 0
invlpg 0 0
io_exits 1821386 14108
irq_exits 81999 2798
irq_injections 343720 8560
irq_window 12712 153
largepages 754 2
mmio_exits 37 0
mmu_cache_miss 148 0
mmu_flooded 0 0
mmu_pde_zapped 0 0
mmu_pte_updated 0 0
mmu_pte_write 0 0
mmu_recycled 0 0
mmu_shadow_zapped 189 0
mmu_unsync 0 0
nmi_injections 0 0
nmi_window 0 0
pf_fixed 139461 21
pf_guest 0 0
remote_tlb_flush 248 0
request_irq 0 0
signal_exits 0 0
tlb_flush 15366 0
trace at http://82.141.21.156/report3.txt.gz
Peter
> --
> Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 11:16 ` Peter Lieven
@ 2012-02-21 11:46 ` Gleb Natapov
2012-02-21 12:05 ` Peter Lieven
0 siblings, 1 reply; 19+ messages in thread
From: Gleb Natapov @ 2012-02-21 11:46 UTC (permalink / raw)
To: Peter Lieven; +Cc: qemu-devel, kvm, vrozenfe
On Tue, Feb 21, 2012 at 12:16:16PM +0100, Peter Lieven wrote:
> On 21.02.2012 12:00, Gleb Natapov wrote:
> >On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
> >>On 21.02.2012 11:56, Gleb Natapov wrote:
> >>>On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
> >>>>>I hope it will make Windows use TSC instead, but you can't be sure
> >>>>>about anything with Windows :(
> >>>>Whatever it does now it eates more CPU has almost equal
> >>>>number of exits and throughput is about the same (15MB/s).
> >>>>If pmtimer is at 0xb008 it still reads it like hell.
> >>>>
> >>>>I checked with bcedit /v that useplatformclock is set to "No".
> >>>Yeah, today I noticed that it is likely virtio drivers that hammer
> >>>on PM timer (at least rip of the instruction that access it is
> >>>very close to rip of the instruction that access virtio pio).
> >>>Vadim, Windows driver developer, is CCed.
> >>Ok, I will switch to IDE and e1000 to confirm this? Or does it not
> >>make sense?
> >>
> >It make perfect sense! Please try it.
> ~10MB/s. still a lot of 0xb008 reads.
>
The same amount of reads essentially. So my theory is incorrect. Virtio
driver probably calls Windows function to do IO and the function
happens to be near the function that access PM timer.
I wonder why time stamps in your traces are so coarse-grained. What do
you see in /sys/bus/clocksource/devices/clocksource0/current_clocksource ?
--
Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 11:46 ` Gleb Natapov
@ 2012-02-21 12:05 ` Peter Lieven
2012-02-21 13:56 ` Vadim Rozenfeld
0 siblings, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-21 12:05 UTC (permalink / raw)
To: Gleb Natapov; +Cc: qemu-devel, kvm, vrozenfe
On 21.02.2012 12:46, Gleb Natapov wrote:
> On Tue, Feb 21, 2012 at 12:16:16PM +0100, Peter Lieven wrote:
>> On 21.02.2012 12:00, Gleb Natapov wrote:
>>> On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
>>>> On 21.02.2012 11:56, Gleb Natapov wrote:
>>>>> On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
>>>>>>> I hope it will make Windows use TSC instead, but you can't be sure
>>>>>>> about anything with Windows :(
>>>>>> Whatever it does now it eates more CPU has almost equal
>>>>>> number of exits and throughput is about the same (15MB/s).
>>>>>> If pmtimer is at 0xb008 it still reads it like hell.
>>>>>>
>>>>>> I checked with bcedit /v that useplatformclock is set to "No".
>>>>> Yeah, today I noticed that it is likely virtio drivers that hammer
>>>>> on PM timer (at least rip of the instruction that access it is
>>>>> very close to rip of the instruction that access virtio pio).
>>>>> Vadim, Windows driver developer, is CCed.
>>>> Ok, I will switch to IDE and e1000 to confirm this? Or does it not
>>>> make sense?
>>>>
>>> It make perfect sense! Please try it.
>> ~10MB/s. still a lot of 0xb008 reads.
>>
> The same amount of reads essentially. So my theory is incorrect. Virtio
> driver probably calls Windows function to do IO and the function
> happens to be near the function that access PM timer.
>
> I wonder why time stamps in your traces are so coarse-grained. What do
> you see in /sys/bus/clocksource/devices/clocksource0/current_clocksource ?
its set to acpi_pm on the host. we changed that from tsc (choosen by
kernel) after we encountered
a kernel bug which ooops all hosts after approx. 270 days uptime.
(https://lkml.org/lkml/2011/7/21/343).
i am not sure if this is fixed in 2.6.38 or later kernels and we could
go back to tsc.
for testing i already checked this, but it doesn't give better performance.
peter
> --
> Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 12:05 ` Peter Lieven
@ 2012-02-21 13:56 ` Vadim Rozenfeld
2012-02-21 14:10 ` Peter Lieven
0 siblings, 1 reply; 19+ messages in thread
From: Vadim Rozenfeld @ 2012-02-21 13:56 UTC (permalink / raw)
To: Peter Lieven; +Cc: Gleb Natapov, qemu-devel, kvm
----- Original Message -----
From: "Peter Lieven" <pl@dlh.net>
To: "Gleb Natapov" <gleb@redhat.com>
Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, vrozenfe@redhat.com
Sent: Tuesday, February 21, 2012 2:05:25 PM
Subject: Re: win7 bad i/o performance, high insn_emulation and exists
On 21.02.2012 12:46, Gleb Natapov wrote:
> On Tue, Feb 21, 2012 at 12:16:16PM +0100, Peter Lieven wrote:
>> On 21.02.2012 12:00, Gleb Natapov wrote:
>>> On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
>>>> On 21.02.2012 11:56, Gleb Natapov wrote:
>>>>> On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
>>>>>>> I hope it will make Windows use TSC instead, but you can't be sure
>>>>>>> about anything with Windows :(
>>>>>> Whatever it does now it eates more CPU has almost equal
>>>>>> number of exits and throughput is about the same (15MB/s).
>>>>>> If pmtimer is at 0xb008 it still reads it like hell.
>>>>>>
>>>>>> I checked with bcedit /v that useplatformclock is set to "No".
>>>>> Yeah, today I noticed that it is likely virtio drivers that hammer
>>>>> on PM timer (at least rip of the instruction that access it is
>>>>> very close to rip of the instruction that access virtio pio).
>>>>> Vadim, Windows driver developer, is CCed.
>>>> Ok, I will switch to IDE and e1000 to confirm this? Or does it not
>>>> make sense?
>>>>
>>> It make perfect sense! Please try it.
>> ~10MB/s. still a lot of 0xb008 reads.
>>
[VR]
Could it be that you have Driver Verifier running in you system?
> The same amount of reads essentially. So my theory is incorrect. Virtio
> driver probably calls Windows function to do IO and the function
> happens to be near the function that access PM timer.
>
> I wonder why time stamps in your traces are so coarse-grained. What do
> you see in /sys/bus/clocksource/devices/clocksource0/current_clocksource ?
its set to acpi_pm on the host. we changed that from tsc (choosen by
kernel) after we encountered
a kernel bug which ooops all hosts after approx. 270 days uptime.
(https://lkml.org/lkml/2011/7/21/343).
i am not sure if this is fixed in 2.6.38 or later kernels and we could
go back to tsc.
for testing i already checked this, but it doesn't give better performance.
peter
> --
> Gleb.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 13:56 ` Vadim Rozenfeld
@ 2012-02-21 14:10 ` Peter Lieven
2012-02-21 16:48 ` Vadim Rozenfeld
0 siblings, 1 reply; 19+ messages in thread
From: Peter Lieven @ 2012-02-21 14:10 UTC (permalink / raw)
To: Vadim Rozenfeld; +Cc: Gleb Natapov, qemu-devel, kvm
On 21.02.2012 14:56, Vadim Rozenfeld wrote:
>
> ----- Original Message -----
> From: "Peter Lieven"<pl@dlh.net>
> To: "Gleb Natapov"<gleb@redhat.com>
> Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, vrozenfe@redhat.com
> Sent: Tuesday, February 21, 2012 2:05:25 PM
> Subject: Re: win7 bad i/o performance, high insn_emulation and exists
>
> On 21.02.2012 12:46, Gleb Natapov wrote:
>> On Tue, Feb 21, 2012 at 12:16:16PM +0100, Peter Lieven wrote:
>>> On 21.02.2012 12:00, Gleb Natapov wrote:
>>>> On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
>>>>> On 21.02.2012 11:56, Gleb Natapov wrote:
>>>>>> On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
>>>>>>>> I hope it will make Windows use TSC instead, but you can't be sure
>>>>>>>> about anything with Windows :(
>>>>>>> Whatever it does now it eates more CPU has almost equal
>>>>>>> number of exits and throughput is about the same (15MB/s).
>>>>>>> If pmtimer is at 0xb008 it still reads it like hell.
>>>>>>>
>>>>>>> I checked with bcedit /v that useplatformclock is set to "No".
>>>>>> Yeah, today I noticed that it is likely virtio drivers that hammer
>>>>>> on PM timer (at least rip of the instruction that access it is
>>>>>> very close to rip of the instruction that access virtio pio).
>>>>>> Vadim, Windows driver developer, is CCed.
>>>>> Ok, I will switch to IDE and e1000 to confirm this? Or does it not
>>>>> make sense?
>>>>>
>>>> It make perfect sense! Please try it.
>>> ~10MB/s. still a lot of 0xb008 reads.
>>>
> [VR]
> Could it be that you have Driver Verifier running in you system?
>
unfortunately not.
i found the following in an old knowledge base article
(http://support.microsoft.com/kb/938448):
"Only Windows Server 2003 with Service Pack 2 uniprocessor ACPI HALs use
*PMTIMER* for QPC by default. Multiprocessor ACPI HALs will use
*PMTIMER* only if *USE_PLATFORM_CLOCK *flag is set by the BIOS or if the
*/usepmtimer *boot.ini option is used. Other HAL types don’t support
*PMTIMER* and will use *TSC* by default for QPC
By default, Windows Server 2003 Service Pack 2 (SP2) uses the PM timer
for all Advanced Configuration and Power Interface (ACPI) HALs unless
one of the following conditions aretrue:
* The check process to determine whether the BIOS supports the APIC
or ACPI HALs fails.
* *
Note:* If the BIOS does not support the ACPI HAL, contact the
original equipment manufacturer to determine whether a BIOS update
is available that will resolve the problem. If a BIOS update is
not available, you must use the PM timer by using the
*/usepmtimer* switch.
If you are not running Windows Server 2003 SP2, you must force the AMD
computer to use the PM timer by using the */usepmtimer* switch.
*Note* The decision to use the PM timer or the TSC timer is made during
a check that is performed at startup to query the BIOS and to determine
whether the BIOS will support the PM timer functions. This check is not
completely accurate on AMD chipsets. Therefore, you must use the
*/usepmtimer* switch.
In Windows Server 2003 SP2, this section of code was rewritten.
Therefore, the correct performance monitor data appears on AMD chipsets
that have Windows Server 2003 SP2 installed, and you do not have to use
the */usepmtimer* switch.
For more information about ACPI and APCI hardware support, click the
following article number to view the article in the Microsoft Knowledge
Base:
309283 <http://support.microsoft.com/kb/309283> HAL options after
Windows XP or Windows Server 2003 Setup
The third-party products that this article discusses are manufactured by
companies that are independent of Microsoft. Microsoft makes no
warranty, implied or otherwise, about the performance or reliability of
these products."
-
so it seems windows prefers pmtimer over tsc. has anyone an idea/hack to
make the acpi_pm timer fail without disabling acpi completely?
thanks,
peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 14:10 ` Peter Lieven
@ 2012-02-21 16:48 ` Vadim Rozenfeld
2012-02-21 18:21 ` Peter Lieven
0 siblings, 1 reply; 19+ messages in thread
From: Vadim Rozenfeld @ 2012-02-21 16:48 UTC (permalink / raw)
To: Peter Lieven; +Cc: Gleb Natapov, qemu-devel, kvm
----- Original Message -----
From: "Peter Lieven" <pl@dlh.net>
To: "Vadim Rozenfeld" <vrozenfe@redhat.com>
Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, "Gleb Natapov" <gleb@redhat.com>
Sent: Tuesday, February 21, 2012 4:10:22 PM
Subject: Re: win7 bad i/o performance, high insn_emulation and exists
On 21.02.2012 14:56, Vadim Rozenfeld wrote:
>
> ----- Original Message -----
> From: "Peter Lieven"<pl@dlh.net>
> To: "Gleb Natapov"<gleb@redhat.com>
> Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, vrozenfe@redhat.com
> Sent: Tuesday, February 21, 2012 2:05:25 PM
> Subject: Re: win7 bad i/o performance, high insn_emulation and exists
>
> On 21.02.2012 12:46, Gleb Natapov wrote:
>> On Tue, Feb 21, 2012 at 12:16:16PM +0100, Peter Lieven wrote:
>>> On 21.02.2012 12:00, Gleb Natapov wrote:
>>>> On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
>>>>> On 21.02.2012 11:56, Gleb Natapov wrote:
>>>>>> On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
>>>>>>>> I hope it will make Windows use TSC instead, but you can't be sure
>>>>>>>> about anything with Windows :(
>>>>>>> Whatever it does now it eates more CPU has almost equal
>>>>>>> number of exits and throughput is about the same (15MB/s).
>>>>>>> If pmtimer is at 0xb008 it still reads it like hell.
>>>>>>>
>>>>>>> I checked with bcedit /v that useplatformclock is set to "No".
>>>>>> Yeah, today I noticed that it is likely virtio drivers that hammer
>>>>>> on PM timer (at least rip of the instruction that access it is
>>>>>> very close to rip of the instruction that access virtio pio).
>>>>>> Vadim, Windows driver developer, is CCed.
>>>>> Ok, I will switch to IDE and e1000 to confirm this? Or does it not
>>>>> make sense?
>>>>>
>>>> It make perfect sense! Please try it.
>>> ~10MB/s. still a lot of 0xb008 reads.
>>>
> [VR]
> Could it be that you have Driver Verifier running in you system?
>
unfortunately not.
[VR]
Then could you try booting into "Safe Mode"?
i found the following in an old knowledge base article
(http://support.microsoft.com/kb/938448):
"Only Windows Server 2003 with Service Pack 2 uniprocessor ACPI HALs use
*PMTIMER* for QPC by default. Multiprocessor ACPI HALs will use
*PMTIMER* only if *USE_PLATFORM_CLOCK *flag is set by the BIOS or if the
*/usepmtimer *boot.ini option is used. Other HAL types don’t support
*PMTIMER* and will use *TSC* by default for QPC
By default, Windows Server 2003 Service Pack 2 (SP2) uses the PM timer
for all Advanced Configuration and Power Interface (ACPI) HALs unless
one of the following conditions aretrue:
* The check process to determine whether the BIOS supports the APIC
or ACPI HALs fails.
* *
Note:* If the BIOS does not support the ACPI HAL, contact the
original equipment manufacturer to determine whether a BIOS update
is available that will resolve the problem. If a BIOS update is
not available, you must use the PM timer by using the
*/usepmtimer* switch.
If you are not running Windows Server 2003 SP2, you must force the AMD
computer to use the PM timer by using the */usepmtimer* switch.
*Note* The decision to use the PM timer or the TSC timer is made during
a check that is performed at startup to query the BIOS and to determine
whether the BIOS will support the PM timer functions. This check is not
completely accurate on AMD chipsets. Therefore, you must use the
*/usepmtimer* switch.
In Windows Server 2003 SP2, this section of code was rewritten.
Therefore, the correct performance monitor data appears on AMD chipsets
that have Windows Server 2003 SP2 installed, and you do not have to use
the */usepmtimer* switch.
For more information about ACPI and APCI hardware support, click the
following article number to view the article in the Microsoft Knowledge
Base:
309283 <http://support.microsoft.com/kb/309283> HAL options after
Windows XP or Windows Server 2003 Setup
The third-party products that this article discusses are manufactured by
companies that are independent of Microsoft. Microsoft makes no
warranty, implied or otherwise, about the performance or reliability of
these products."
-
so it seems windows prefers pmtimer over tsc. has anyone an idea/hack to
make the acpi_pm timer fail without disabling acpi completely?
thanks,
peter
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists
2012-02-21 16:48 ` Vadim Rozenfeld
@ 2012-02-21 18:21 ` Peter Lieven
0 siblings, 0 replies; 19+ messages in thread
From: Peter Lieven @ 2012-02-21 18:21 UTC (permalink / raw)
To: Vadim Rozenfeld; +Cc: Gleb Natapov, qemu-devel, kvm
On 21.02.2012 17:48, Vadim Rozenfeld wrote:
>
> ----- Original Message -----
> From: "Peter Lieven"<pl@dlh.net>
> To: "Vadim Rozenfeld"<vrozenfe@redhat.com>
> Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, "Gleb Natapov"<gleb@redhat.com>
> Sent: Tuesday, February 21, 2012 4:10:22 PM
> Subject: Re: win7 bad i/o performance, high insn_emulation and exists
>
> On 21.02.2012 14:56, Vadim Rozenfeld wrote:
>> ----- Original Message -----
>> From: "Peter Lieven"<pl@dlh.net>
>> To: "Gleb Natapov"<gleb@redhat.com>
>> Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, vrozenfe@redhat.com
>> Sent: Tuesday, February 21, 2012 2:05:25 PM
>> Subject: Re: win7 bad i/o performance, high insn_emulation and exists
>>
>> On 21.02.2012 12:46, Gleb Natapov wrote:
>>> On Tue, Feb 21, 2012 at 12:16:16PM +0100, Peter Lieven wrote:
>>>> On 21.02.2012 12:00, Gleb Natapov wrote:
>>>>> On Tue, Feb 21, 2012 at 11:59:23AM +0100, Peter Lieven wrote:
>>>>>> On 21.02.2012 11:56, Gleb Natapov wrote:
>>>>>>> On Tue, Feb 21, 2012 at 11:50:47AM +0100, Peter Lieven wrote:
>>>>>>>>> I hope it will make Windows use TSC instead, but you can't be sure
>>>>>>>>> about anything with Windows :(
>>>>>>>> Whatever it does now it eates more CPU has almost equal
>>>>>>>> number of exits and throughput is about the same (15MB/s).
>>>>>>>> If pmtimer is at 0xb008 it still reads it like hell.
>>>>>>>>
>>>>>>>> I checked with bcedit /v that useplatformclock is set to "No".
>>>>>>> Yeah, today I noticed that it is likely virtio drivers that hammer
>>>>>>> on PM timer (at least rip of the instruction that access it is
>>>>>>> very close to rip of the instruction that access virtio pio).
>>>>>>> Vadim, Windows driver developer, is CCed.
>>>>>> Ok, I will switch to IDE and e1000 to confirm this? Or does it not
>>>>>> make sense?
>>>>>>
>>>>> It make perfect sense! Please try it.
>>>> ~10MB/s. still a lot of 0xb008 reads.
>>>>
>> [VR]
>> Could it be that you have Driver Verifier running in you system?
>>
> unfortunately not.
>
> [VR]
> Then could you try booting into "Safe Mode"?
same picture.
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2012-02-21 18:21 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-02-20 18:17 [Qemu-devel] win7 bad i/o performance, high insn_emulation and exists Peter Lieven
2012-02-20 18:40 ` Gleb Natapov
2012-02-20 19:04 ` Gleb Natapov
2012-02-20 19:24 ` Peter Lieven
2012-02-20 19:59 ` Peter Lieven
2012-02-20 20:45 ` Gleb Natapov
2012-02-21 10:50 ` Peter Lieven
2012-02-21 10:56 ` Gleb Natapov
2012-02-21 10:59 ` Peter Lieven
2012-02-21 11:00 ` Gleb Natapov
2012-02-21 11:16 ` Peter Lieven
2012-02-21 11:46 ` Gleb Natapov
2012-02-21 12:05 ` Peter Lieven
2012-02-21 13:56 ` Vadim Rozenfeld
2012-02-21 14:10 ` Peter Lieven
2012-02-21 16:48 ` Vadim Rozenfeld
2012-02-21 18:21 ` Peter Lieven
2012-02-20 19:15 ` Peter Lieven
2012-02-20 20:42 ` Gleb Natapov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).