Re: kernel BUG at arch/x86/kvm/x86.c:361! on AMD CPU

Borislav Petkov
2019-06-18T17:52:09Z

On Mon, Jun 17, 2019 at 11:13:22AM -0400, George Kennedy wrote:
> 319f3: 0f 01 da vmload <--- svm_vcpu_run+0xa83

Hmm, so VMLOAD can fault for a bunch of reasons if you look at its
description in the APM.

Looking at your Code: section and building with your config, rIP and the
insns around it point to:

All code
========
0: 00 55 89 add %dl,-0x77(%rbp)
3: d2 45 89 rolb %cl,-0x77(%rbp)
6: c9 leaveq
7: 48 89 e5 mov %rsp,%rbp
a: 8b 45 18 mov 0x18(%rbp),%eax
d: 50 push %rax
e: 8b 45 10 mov 0x10(%rbp),%eax
11: 50 push %rax
12: e8 8a 42 6b 00 callq 0x6b42a1
17: 58 pop %rax
18: 5a pop %rdx
19: c9 leaveq
1a: c3 retq
1b: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1)
21: 66 66 66 66 90 data16 data16 data16 xchg %ax,%ax
26: 55 push %rbp
27: 48 89 e5 mov %rsp,%rbp
2a:* 0f 0b ud2 <-- trapping instruction
2c: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
31: 66 66 66 66 90 data16 data16 data16 xchg %ax,%ax
36: 55 push %rbp
37: 48 89 e5 mov %rsp,%rbp
3a: 41 55 push %r13
3c: 41 54 push %r12
3e: 41 rex.B
3f: 89 .byte 0x89

0000000000017f30 <__bpf_trace_kvm_nested_vmexit>:
17f30: 55 push %rbp
17f31: 89 d2 mov %edx,%edx
17f33: 45 89 c9 mov %r9d,%r9d
17f36: 48 89 e5 mov %rsp,%rbp
17f39: 8b 45 18 mov 0x18(%rbp),%eax
17f3c: 50 push %rax
17f3d: 8b 45 10 mov 0x10(%rbp),%eax
17f40: 50 push %rax
17f41: e8 00 00 00 00 callq 17f46 <__bpf_trace_kvm_nested_vmexit+0x16>
17f46: 58 pop %rax
17f47: 5a pop %rdx
17f48: c9 leaveq
17f49: c3 retq
17f4a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1)

0000000000017f50 <kvm_spurious_fault>:
17f50: e8 00 00 00 00 callq 17f55 <kvm_spurious_fault+0x5>
17f55: 55 push %rbp
17f56: 48 89 e5 mov %rsp,%rbp
17f59: 0f 0b ud2
17f5b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)

so the invalid opcode splat is dumping the insn bytes around rIP the
UD2 happens but in order to see which of the VMLOAD conditions causes
the fault, you'd probably need to intercept the fault handler and dump
registers in the hypervisor to check.

There's something else that's bothering me though: your splat is from
the guest yet there is svm_vcpu_run() mentioned there which should be
called by the hypervisor and not by the guest. Unless you're doing
nested virt stuff...

Anyway, sorry that I cannot be of more help - I'm sure KVM guys would
make much more sense of it.