From: Paolo Bonzini <pbonzini@redhat.com>
To: Zhang Haoyu <zhanghy@sangfor.com>, kvm <kvm@vger.kernel.org>
Cc: Alex Williamson <alex.williamson@redhat.com>,
"Zhang, Yang Z" <yang.z.zhang@intel.com>,
"Michael S.Tsirkin" <mst@redhat.com>,
Gleb Natapov <gleb@cloudius-systems.com>
Subject: Re: [question] lots of interrupts injected to vm when pressing somekey w/o releasing
Date: Tue, 25 Nov 2014 14:52:42 +0100 [thread overview]
Message-ID: <547489AA.3080509@redhat.com> (raw)
In-Reply-To: <201411251920494491538@sangfor.com>
On 25/11/2014 12:20, Zhang Haoyu wrote:
>>> I tested win-server-2008 with "-cpu core2duo,hv_spinlocks=0xffff,hv_relaxed,hv_time",
>>> this problem still happened, about 200,000 vmexits per-second,
>>> bringing very bad experience, just like being stuck.
>>
>> Please upload a full trace somewhere, or at least the "perf report" output.
>>
> # perf kvm stat report --event=vmexit
> Warning:
> Processed 4912765 events and lost 81 chunks!
> Check IO/CPU overload!
>
> Analyze events for all VCPUs:
>
> VM-EXIT Samples Samples% Time% Avg time
>
> PAUSE_INSTRUCTION 1248620 60.74% 23.33% 0.83us ( +- 0.02% )
> IO_INSTRUCTION 799619 38.89% 74.49% 4.13us ( +- 6.93% )
> EXTERNAL_INTERRUPT 2629 0.13% 0.47% 7.93us ( +- 1.27% )
> VMCALL 1785 0.09% 0.04% 0.98us ( +- 0.30% )
> APIC_ACCESS 1495 0.07% 0.11% 3.38us ( +- 1.65% )
> EXCEPTION_NMI 1188 0.06% 0.03% 1.26us ( +- 1.66% )
> TPR_BELOW_THRESHOLD 273 0.01% 0.01% 1.57us ( +- 1.32% )
> PENDING_INTERRUPT 226 0.01% 0.01% 1.38us ( +- 2.59% )
> HLT 6 0.00% 1.51% 11153.81us ( +- 15.24% )
> EPT_MISCONFIG 2 0.00% 0.00% 15.46us ( +- 23.93% )
>
> Total Samples:2055843, Total events handled time:4438794.59us.
>
> # perf kvm stat report --event=ioport
> Warning:
> Processed 4912765 events and lost 81 chunks!
> Check IO/CPU overload!
>
> Analyze events for all VCPUs:
>
> IO Port Access Samples Samples% Time% Avg time
>
> 0x64:PIN 799032 99.93% 99.84% 3.42us ( +- 8.35% )
> 0x3fd:PIN 300 0.04% 0.08% 6.88us ( +- 3.45% )
> 0x60:PIN 274 0.03% 0.09% 8.53us ( +- 3.07% )
> 0xb000:POUT 2 0.00% 0.00% 7.96us ( +- 3.16% )
> 0xb000:PIN 2 0.00% 0.00% 8.16us ( +- 2.31% )
> 0xb008:PIN 2 0.00% 0.00% 1.53us ( +- 13.66% )
> 0xafe0:PIN 2 0.00% 0.00% 3.62us ( +- 0.84% )
> 0xafe1:PIN 2 0.00% 0.00% 2.76us ( +- 2.99% )
>
> Total Samples:799616, Total events handled time:2739038.53us.
>
> # trace-cmd report
> version = 6
> CPU 4 is empty
> CPU 5 is empty
> CPU 6 is empty
> CPU 7 is empty
> CPU 9 is empty
> CPU 11 is empty
> CPU 12 is empty
> CPU 13 is empty
> cpus=16
> kvm-22783 [003] 721.255618: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255619: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255622: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22784 [014] 721.255623: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255623: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255623: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255623: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255625: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255626: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22783 [003] 721.255626: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255627: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255628: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255628: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255628: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22783 [003] 721.255629: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255630: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255631: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22784 [014] 721.255632: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22783 [003] 721.255632: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255632: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255632: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255634: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22784 [014] 721.255634: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255635: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255636: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255636: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255636: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255637: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22783 [003] 721.255637: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255638: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255640: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22783 [003] 721.255640: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255640: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255640: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255641: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255642: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22784 [014] 721.255643: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255643: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255645: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255645: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255645: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255645: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22783 [003] 721.255646: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255647: kvm_entry: vcpu 1
> kvm-22784 [014] 721.255649: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255649: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255649: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255649: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22783 [003] 721.255650: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255651: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255652: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22783 [003] 721.255653: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255653: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255653: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22784 [014] 721.255654: kvm_userspace_exit: reason KVM_EXIT_IO (2)
> kvm-22783 [003] 721.255655: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> kvm-22784 [014] 721.255655: kvm_entry: vcpu 1
> kvm-22783 [003] 721.255655: kvm_entry: vcpu 0
> kvm-22784 [014] 721.255657: kvm_exit: reason IO_INSTRUCTION rip 0xfffff88000cdd23b info 640008 0
> kvm-22784 [014] 721.255658: kvm_pio: pio_read at 0x64 size 1 count 1
> kvm-22783 [003] 721.255658: kvm_exit: reason PAUSE_INSTRUCTION rip 0xfffff80001688a9e info 0 0
> ...
>
> And, if I remove the commit of 0bc830b0, the problem disappeared.
Please send the full trace file. If you compress it, it should be small.
Paolo
next prev parent reply other threads:[~2014-11-25 13:53 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-20 2:20 [question] lots of interrupts injected to vm when pressing some key w/o releasing Zhang Haoyu
2014-11-20 6:19 ` Paolo Bonzini
2014-11-20 6:59 ` [question] lots of interrupts injected to vm when pressing somekey " Zhang Haoyu
2014-11-20 7:06 ` Gleb Natapov
2014-11-24 8:58 ` [question] lots of interrupts injected to vm when pressingsomekey " Zhang Haoyu
[not found] ` <A9667DDFB95DB7438FA9D7D576C3D87E0ABEF25B@SHSMSX104.ccr.corp.intel.com>
2014-11-21 22:31 ` [question] lots of interrupts injected to vm when pressing some key " Paolo Bonzini
[not found] ` <A9667DDFB95DB7438FA9D7D576C3D87E0ABF0FE3@SHSMSX104.ccr.corp.intel.com>
2014-11-25 9:49 ` Paolo Bonzini
[not found] ` <A9667DDFB95DB7438FA9D7D576C3D87E0ABF396A@SHSMSX104.ccr.corp.intel.com>
2014-12-03 15:42 ` Paolo Bonzini
2014-12-15 9:39 ` Zhang Haoyu
2014-12-15 9:40 ` Paolo Bonzini
2014-11-24 12:17 ` Zhang Haoyu
2014-11-24 12:25 ` Paolo Bonzini
2014-11-25 1:54 ` [question] lots of interrupts injected to vm when pressing somekey " Zhang Haoyu
2014-11-25 7:02 ` Paolo Bonzini
2014-11-25 11:20 ` Zhang Haoyu
2014-11-25 13:52 ` Paolo Bonzini [this message]
[not found] ` <201411261511537876936@sangfor.com>
[not found] ` <A9667DDFB95DB7438FA9D7D576C3D87E0ABF2A95@SHSMSX104.ccr.corp.intel.com>
2014-11-27 7:20 ` [question] lots of interrupts injected to vm when pressing some key " Zhang Haoyu
[not found] ` <A9667DDFB95DB7438FA9D7D576C3D87E0ABF2C24@SHSMSX104.ccr.corp.intel.com>
2014-11-27 8:54 ` Zhang Haoyu
2014-11-28 8:27 ` Wanpeng Li
2014-11-28 9:44 ` Zhang Haoyu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=547489AA.3080509@redhat.com \
--to=pbonzini@redhat.com \
--cc=alex.williamson@redhat.com \
--cc=gleb@cloudius-systems.com \
--cc=kvm@vger.kernel.org \
--cc=mst@redhat.com \
--cc=yang.z.zhang@intel.com \
--cc=zhanghy@sangfor.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.