From: David Matlack <dmatlack@google.com>
To: kvm@vger.kernel.org
Cc: seanjc@google.com, pbonzini@redhat.com, vkuznets@redhat.com,
laijs@linux.alibaba.com
Subject: VM_BUG_ON in vmx_prepare_switch_to_guest->__get_current_cr3_fast at kvm/queue
Date: Fri, 10 Dec 2021 17:57:24 +0000 [thread overview]
Message-ID: <YbOVBDCcpuwtXD/7@google.com> (raw)
While testing some patches I ran into a VM_BUG_ON that I have been able to
reproduce at kvm/queue commit 45af1bb99b72 ("KVM: VMX: Clean up PI
pre/post-block WARNs").
To repro run the kvm-unit-tests on a kernel built from kvm/queue with
CONFIG_DEBUG_VM=y. I was testing on an Intel Cascade Lake host and have not
tested in any other environments yet. The repro is not 100% reliable, although
it's fairly easy to trigger and always during a vmx* kvm-unit-tests
Given the details of the crash, commit 15ad9762d69f ("KVM: VMX: Save HOST_CR3
in vmx_prepare_switch_to_guest()") and surrounding commits look most suspect.
The splat:
[ 698.724442] ------------[ cut here ]------------
[ 698.729095] kernel BUG at arch/x86/mm/tlb.c:1082!
[ 698.733838] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC NOPTI
[ 698.740475] CPU: 29 PID: 63256 Comm: qemu-kvm-system Tainted: G S O 5.16.0-dbg-DEV #1
[ 698.756882] RIP: 0010:__get_current_cr3_fast+0xe6/0x110
[ 698.762134] Code: 3b 4d f8 75 27 48 83 c4 10 5d c3 0f 0b eb df 0f 0b eb 98 0f 0b eb a2 66 85 c9 75 15 48 39 d0 76 17 48 8b 0d dc a7 b9 01 eb 1c <0f> 0b e8 23 8c ba 00 0f 0b 48 39 d0 77 e9 48 c7 c1 00 00 00 80 48
[ 698.780967] RSP: 0018:ffffc90039c6fa50 EFLAGS: 00010297
[ 698.786209] RAX: 00000060674f2005 RBX: ffff88e0911d5380 RCX: 00000060674f2006
[ 698.793366] RDX: 0000000000000004 RSI: 0000000000000000 RDI: ffff88e0985eec18
[ 698.800524] RBP: ffffc90039c6fa60 R08: ffff893e5bf40000 R09: 0000000000000000
[ 698.807682] R10: 00000000000206dd R11: 0000000000000000 R12: ffff88e0985ec8c0
[ 698.814838] R13: 0000000000000000 R14: ffff88e0985eec18 R15: ffff893e5bf40000
[ 698.821997] FS: 00007f5b823ff700(0000) GS:ffff893e5bf40000(0000) knlGS:0000000000000000
[ 698.830114] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 698.835877] CR2: 0000000000000000 CR3: 00000060674f2006 CR4: 00000000003726e0
[ 698.843034] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 698.850192] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 698.857349] Call Trace:
[ 698.859801] <TASK>
[ 698.861907] vmx_prepare_switch_to_guest+0x11f/0x290 [kvm_intel]
[ 698.867945] vcpu_enter_guest+0x128b/0x24b0 [kvm]
[ 698.872719] ? __this_cpu_preempt_check+0x13/0x20
[ 698.877446] ? lock_is_held_type+0xff/0x170
[ 698.881646] ? __this_cpu_preempt_check+0x13/0x20
[ 698.886371] ? lock_is_held_type+0xff/0x170
[ 698.890568] ? __lock_acquire+0x91e/0xf00
[ 698.894599] ? __lock_acquire+0x91e/0xf00
[ 698.898622] ? __this_cpu_preempt_check+0x13/0x20
[ 698.903348] ? lock_acquire+0xda/0x210
[ 698.907111] ? trace_kvm_pio+0x2c/0xd0 [kvm]
[ 698.911422] vcpu_run+0x90/0x370 [kvm]
[ 698.915211] kvm_arch_vcpu_ioctl_run+0x173/0x330 [kvm]
[ 698.920394] kvm_vcpu_ioctl+0x5e3/0x6b0 [kvm]
[ 698.924792] ? rcu_lock_release+0x10/0x20
[ 698.928824] ? __fget_files+0x1bb/0x1d0
[ 698.932672] __se_sys_ioctl+0x77/0xc0
[ 698.936355] __x64_sys_ioctl+0x1d/0x20
[ 698.940116] do_syscall_64+0x44/0xa0
[ 698.943702] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 698.948769] RIP: 0033:0x7f5b8b60b947
[ 698.952355] Code: 73 01 c3 48 8b 0d 31 f5 16 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 01 f5 16 00 f7 d8 64 89 01 48
[ 698.971187] RSP: 002b:00007f5b823fe4d8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[ 698.978780] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f5b8b60b947
[ 698.985937] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 000000000000000e
[ 698.993093] RBP: 00007f5b823fe5c0 R08: 0000000000000400 R09: 00000000000000ff
[ 699.000251] R10: 0000550e7e92ed00 R11: 0000000000000246 R12: 00007f5b8a4a6000
[ 699.007410] R13: 0000550e7f95c000 R14: 0000000000000000 R15: 0000550e7f95c000
[ 699.014571] </TASK>
[ 699.034357] ---[ end trace ee35b3363814d971 ]---
... which is the following VM_BUG_ON:
1074 unsigned long __get_current_cr3_fast(void)
1075 {
1076 unsigned long cr3 = build_cr3(this_cpu_read(cpu_tlbstate.loaded_mm)->pgd,
1077 this_cpu_read(cpu_tlbstate.loaded_mm_asid));
1078
1079 /* For now, be very restrictive about when this can be called. */
1080 VM_WARN_ON(in_nmi() || preemptible());
1081
1082 VM_BUG_ON(cr3 != __read_cr3()); <------------
1083 return cr3;
1084 }
next reply other threads:[~2021-12-10 17:57 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-10 17:57 David Matlack [this message]
2021-12-10 23:54 ` VM_BUG_ON in vmx_prepare_switch_to_guest->__get_current_cr3_fast at kvm/queue Lai Jiangshan
2021-12-11 1:04 ` Lai Jiangshan
2021-12-11 0:11 ` Paolo Bonzini
2021-12-11 2:01 ` Paolo Bonzini
2021-12-11 3:14 ` Lai Jiangshan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YbOVBDCcpuwtXD/7@google.com \
--to=dmatlack@google.com \
--cc=kvm@vger.kernel.org \
--cc=laijs@linux.alibaba.com \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=vkuznets@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.