From: Zdenek Kaspar <zkaspar82@gmail.com>
To: "kvm@vger.kernel.org" <kvm@vger.kernel.org>
Subject: Re: Bad performance since 5.9-rc1
Date: Tue, 1 Dec 2020 07:35:37 +0100 [thread overview]
Message-ID: <20201201073537.6749e2d7.zkaspar82@gmail.com> (raw)
In-Reply-To: <20201119040526.5263f557.zkaspar82@gmail.com>
On Thu, 19 Nov 2020 04:05:26 +0100
Zdenek Kaspar <zkaspar82@gmail.com> wrote:
> Hi,
>
> in my initial report (https://marc.info/?l=kvm&m=160502183220080&w=2 -
> now fixed by c887c9b9ca62c051d339b1c7b796edf2724029ed) I saw degraded
> performance going back somewhere between v5.8 - v5.9-rc1.
>
> OpenBSD 6.8 (GENERIC.MP) guest performance (time ./test-build.sh)
> good: 0m13.54s real 0m10.51s user 0m10.96s system
> bad : 6m20.07s real 11m42.93s user 0m13.57s system
>
> bisected to first bad commit: 6b82ef2c9cf18a48726e4bb359aa9014632f6466
>
> git bisect log:
> # bad: [e47c4aee5bde03e7018f4fde45ba21028a8f8438] KVM: x86/mmu: Rename
> page_header() to to_shadow_page() # good:
> [01c3b2b5cdae39af8dfcf6e40fdf484ae0e812c5] KVM: SVM: Rename
> svm_nested_virtualize_tpr() to nested_svm_virtualize_tpr() git bisect
> start 'e47c4aee5bde' '01c3b2b5cdae' # bad:
> [ebdb292dac7993425c8e31e2c21c9978e914a676] KVM: x86/mmu: Batch zap MMU
> pages when shrinking the slab git bisect bad
> ebdb292dac7993425c8e31e2c21c9978e914a676 # good:
> [fb58a9c345f645f1774dcf6a36fda169253008ae] KVM: x86/mmu: Optimize MMU
> page cache lookup for fully direct MMUs git bisect good
> fb58a9c345f645f1774dcf6a36fda169253008ae # bad:
> [6b82ef2c9cf18a48726e4bb359aa9014632f6466] KVM: x86/mmu: Batch zap MMU
> pages when recycling oldest pages git bisect bad
> 6b82ef2c9cf18a48726e4bb359aa9014632f6466 # good:
> [f95eec9bed76d42194c23153cb1cc8f186bf91cb] KVM: x86/mmu: Don't put
> invalid SPs back on the list of active pages git bisect good
> f95eec9bed76d42194c23153cb1cc8f186bf91cb # first bad commit:
> [6b82ef2c9cf18a48726e4bb359aa9014632f6466] KVM: x86/mmu: Batch zap MMU
> pages when recycling oldest pages
>
> Host machine is old Intel Core2 without EPT (TDP).
>
> TIA, Z.
Hi, with v5.10-rc6:
get_mmio_spte: detect reserved bits on spte, addr 0xfe00d000, dump hierarchy:
------ spte 0x8000030e level 3.
------ spte 0xaf82027 level 2.
------ spte 0x2038001ffe00d407 level 1.
------------[ cut here ]------------
WARNING: CPU: 1 PID: 355 at kvm_mmu_page_fault.cold+0x42/0x4f [kvm]
...
CPU: 1 PID: 355 Comm: qemu-build Not tainted 5.10.0-rc6-amd64 #1
Hardware name: /DG35EC, BIOS ECG3510M.86A.0118.2010.0113.1426 01/13/2010
RIP: 0010:kvm_mmu_page_fault.cold+0x42/0x4f [kvm]
Code: e2 ec 44 8b 04 24 8b 5c 24 0c 44 89 c5 89 da 83 eb 01 48 c7 c7 20 b2 65 c0 48 63 c3 48 8b 74 c4 30 e8 dd 74 e2 ec 39 dd 7e e3 <0f> 0b 41 b8 ea ff ff ff e9 27 99 ff ff 0f 0b 48 8b 54 24 10 48 83
RSP: 0018:ffffb67400653d30 EFLAGS: 00010202
RAX: 0000000000000027 RBX: 0000000000000000 RCX: ffffa271ff2976f8
RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffffa271ff2976f0
RBP: 0000000000000001 R08: ffffffffadd02ae8 R09: 0000000000000003
R10: 00000000ffffe000 R11: 3fffffffffffffff R12: 00000000fe00d000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
FS: 00007fc10ae3d640(0000) GS:ffffa271ff280000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000002dc2000 CR4: 00000000000026e0
Call Trace:
kvm_arch_vcpu_ioctl_run+0xbaf/0x18f0 [kvm]
? do_futex+0x7c4/0xb80
kvm_vcpu_ioctl+0x203/0x520 [kvm]
? set_next_entity+0x5b/0x80
? __switch_to_asm+0x32/0x60
? finish_task_switch+0x70/0x260
__x64_sys_ioctl+0x338/0x720
? __x64_sys_futex+0x120/0x190
do_syscall_64+0x33/0x40
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7fc10c389f6b
Code: 89 d8 49 8d 3c 1c 48 f7 d8 49 39 c4 72 b5 e8 1c ff ff ff 85 c0 78 ba 4c 89 e0 5b 5d 41 5c c3 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 ae 0c 00 f7 d8 64 89 01 48
RSP: 002b:00007fc10ae3c628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007fc10c389f6b
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000012
RBP: 000055ad3767baf0 R08: 000055ad36be4850 R09: 00000000000000ff
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
R13: 000055ad371d9800 R14: 0000000000000001 R15: 0000000000000002
---[ end trace c5f7ae690f5abcc4 ]---
without: kvm: x86/mmu: Fix get_mmio_spte() on CPUs supporting 5-level PT
I can run guest again, but with degraded performance as before.
Z.
next prev parent reply other threads:[~2020-12-01 6:36 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-19 3:05 Bad performance since 5.9-rc1 Zdenek Kaspar
2020-12-01 6:35 ` Zdenek Kaspar [this message]
2020-12-18 19:33 ` Zdenek Kaspar
2020-12-21 19:41 ` Sean Christopherson
2020-12-21 21:13 ` Zdenek Kaspar
2020-12-22 17:07 ` Sean Christopherson
2020-12-22 21:26 ` Zdenek Kaspar
2021-01-12 11:18 ` Zdenek Kaspar
2021-01-13 20:17 ` Sean Christopherson
2021-01-13 22:17 ` Zdenek Kaspar
2020-12-02 0:31 ` Sean Christopherson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201201073537.6749e2d7.zkaspar82@gmail.com \
--to=zkaspar82@gmail.com \
--cc=kvm@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.