From: Paolo Bonzini <pbonzini@redhat.com>
To: edk2-devel@lists.sourceforge.net
Cc: Laszlo Ersek <lersek@redhat.com>,
KVM devel mailing list <kvm@vger.kernel.org>
Subject: Re: [edk2] apparent KVM problem with LRET in TianoCore S3 resume trampoline
Date: Fri, 06 Dec 2013 13:03:10 +0100 [thread overview]
Message-ID: <52A1BCFE.4020100@redhat.com> (raw)
In-Reply-To: <52A0C5F4.3040603@redhat.com>
Il 05/12/2013 19:29, Laszlo Ersek ha scritto:
> On 12/05/13 18:42, Paolo Bonzini wrote:
>> Il 05/12/2013 17:12, Laszlo Ersek ha scritto:
>>> Hi,
>>>
>>> I'm working on S3 suspend/resume in OVMF. The problem is that I'm getting an
>>> unexpected guest reboot for code (LRET) that works on physical hardware. I
>>> tried to trace the problem with ftrace, but I didn't get any mentions of
>>> em_ret_far(). (Maybe I was looking in the wrong place.)
>>
>> What does ftrace say anyway?
>
> (pls. see in the next msg I sent)
Actually I meant the ftrace without any patches.
Thanks to your binary I now reproduced the issue and it looks like the
64-bit->16-bit switch works:
qemu-system-x86-4081 [001] 62650.335040: kvm_exit: reason CR_ACCESS rip 0x3cf7ae45 info 0 0
qemu-system-x86-4081 [001] 62650.335041: kvm_cr: cr_write 0 = 0x32
qemu-system-x86-4081 [001] 62650.335046: kvm_entry: vcpu 0
This is the "mov %rax, %cr0". PE and PG are turned off.
qemu-system-x86-4081 [001] 62650.335047: kvm_exit: reason MSR_READ rip 0x3cf7ae4e info 0 0
qemu-system-x86-4081 [001] 62650.335048: kvm_msr: msr_read c0000080 = 0x100
qemu-system-x86-4081 [001] 62650.335048: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335048: kvm_exit: reason MSR_WRITE rip 0x3cf7ae53 info 0 0
qemu-system-x86-4081 [001] 62650.335049: kvm_msr: msr_write c0000080 = 0x0
qemu-system-x86-4081 [001] 62650.335050: kvm_entry: vcpu 0
LME is turned off.
qemu-system-x86-4081 [001] 62650.335050: kvm_exit: reason CR_ACCESS rip 0x3cf7ae55 info 304 0
qemu-system-x86-4081 [001] 62650.335050: kvm_cr: cr_write 4 = 0x640
qemu-system-x86-4081 [001] 62650.335053: kvm_entry: vcpu 0
PAE is turned off.
qemu-system-x86-4081 [001] 62650.335054: kvm_exit: reason CR_ACCESS rip 0x11e6 info 0 0
qemu-system-x86-4081 [001] 62650.335054: kvm_cr: cr_write 0 = 0x33
qemu-system-x86-4081 [001] 62650.335054: kvm_entry: vcpu 0
Here we're already in real mode. The weird RIP is explained by
the first few bytes after the FACS resume vector:
0x9a1d:0000: cli
0x9a1d:0001: cld
0x9a1d:0002: ljmp $9900,$11d7
9900:11d7 is the same physical address as 9a1d:0007. Fast forward a bit:
qemu-system-x86-4081 [001] 62650.335071: kvm_exit: reason CR_ACCESS rip 0x9aec7 info 0 0
qemu-system-x86-4081 [001] 62650.335071: kvm_cr: cr_write 0 = 0x80010001
qemu-system-x86-4081 [001] 62650.335074: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335076: kvm_exit: reason TRIPLE_FAULT rip 0x0 info 0 0
Disassembling mixed 16-/32-/64-bit is a pain, so I ran QEMU with -no-shutdown
-no-reboot and dumped the memory with
(qemu) dump-guest-memory tramp.dmp 0x90000 0x10000
Lets look at the rest of the trace now. After the previous cr0 access we have:
qemu-system-x86-4081 [001] 62650.335055: kvm_exit: reason CR_ACCESS rip 0x11fa info 0 0
qemu-system-x86-4081 [001] 62650.335055: kvm_cr: cr_write 0 = 0x32
qemu-system-x86-4081 [001] 62650.335055: kvm_entry: vcpu 0
It gets out to real mode again. No idea why. It does some setup.
qemu-system-x86-4081 [001] 62650.335056: kvm_exit: reason MSR_WRITE rip 0x1258 info 0 0
qemu-system-x86-4081 [001] 62650.335056: kvm_msr: msr_write 1a0 = 0x1
qemu-system-x86-4081 [001] 62650.335057: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335057: kvm_exit: reason WBINVD rip 0x1001 info 0 0
qemu-system-x86-4081 [001] 62650.335057: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335058: kvm_exit: reason CPUID rip 0x1073 info 0 0
qemu-system-x86-4081 [001] 62650.335058: kvm_cpuid: func 0 rax 4 rbx 756e6547 rcx 6c65746e rdx 49656e69
qemu-system-x86-4081 [001] 62650.335059: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335059: kvm_exit: reason CPUID rip 0x10c0 info 0 0
qemu-system-x86-4081 [001] 62650.335059: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd
qemu-system-x86-4081 [001] 62650.335059: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335060: kvm_exit: reason CPUID rip 0x10ff info 0 0
qemu-system-x86-4081 [001] 62650.335060: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd
qemu-system-x86-4081 [001] 62650.335060: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335061: kvm_exit: reason CPUID rip 0x1117 info 0 0
qemu-system-x86-4081 [001] 62650.335061: kvm_cpuid: func 80000000 rax 8000000a rbx 756e6547 rcx 6c65746e rdx 49656e69
qemu-system-x86-4081 [001] 62650.335061: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335062: kvm_exit: reason CPUID rip 0x1127 info 0 0
qemu-system-x86-4081 [001] 62650.335062: kvm_cpuid: func 80000001 rax 663 rbx 0 rcx 1 rdx 2191abfd
qemu-system-x86-4081 [001] 62650.335062: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335063: kvm_exit: reason CPUID rip 0x113f info 0 0
qemu-system-x86-4081 [001] 62650.335063: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd
qemu-system-x86-4081 [001] 62650.335063: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335064: kvm_exit: reason CR_ACCESS rip 0x103c info 0 0
qemu-system-x86-4081 [001] 62650.335064: kvm_cr: cr_write 0 = 0x1
qemu-system-x86-4081 [001] 62650.335064: kvm_entry: vcpu 0
Enabling protected mode:
0009A036 66B801000000 mov eax,0x1
0009A03C 0F22C0 mov cr0,eax
0009A03F 66EA90AE09000800 jmp dword 0x8:0x9ae90
This is a 32-bit selector.
0009AE90 8ED2 mov ss,edx
0009AE92 81C400900900 add esp,0x99000
0009AE98 8EDA mov ds,edx
0009AE9A 8EC2 mov es,edx
0009AE9C 8EE2 mov fs,edx
0009AE9E 8EEA mov gs,edx
qemu-system-x86-4081 [001] 62650.335065: kvm_exit: reason CR_ACCESS rip 0x9aea5 info 4 0
qemu-system-x86-4081 [001] 62650.335065: kvm_cr: cr_write 4 = 0x6f0
qemu-system-x86-4081 [001] 62650.335066: kvm_entry: vcpu 0
Enabling PAE:
0009AEA0 A110D00900 mov eax,[0x9d010]
0009AEA5 0F22E0 mov cr4,eax
qemu-system-x86-4081 [001] 62650.335067: kvm_exit: reason CR_ACCESS rip 0x9aead info 3 0
qemu-system-x86-4081 [001] 62650.335067: kvm_cr: cr_write 3 = 0x9c000
qemu-system-x86-4081 [001] 62650.335068: kvm_entry: vcpu 0
Setting CR3
0009AEA8 B800C00900 mov eax,0x9c000
0009AEAD 0F22D8 mov cr3,eax
qemu-system-x86-4081 [001] 62650.335068: kvm_exit: reason MSR_WRITE rip 0x9aec0 info 0 0
qemu-system-x86-4081 [001] 62650.335070: kvm_msr: msr_write c0000080 = 0x901
qemu-system-x86-4081 [001] 62650.335070: kvm_entry: vcpu 0
Enabling LME
0009AEB0 A108D00900 mov eax,[0x9d008]
0009AEB5 8B150CD00900 mov edx,[dword 0x9d00c]
0009AEBB B9800000C0 mov ecx,0xc0000080
0009AEC0 0F30 wrmsr
qemu-system-x86-4081 [001] 62650.335071: kvm_exit: reason CR_ACCESS rip 0x9aec7 info 0 0
qemu-system-x86-4081 [001] 62650.335071: kvm_cr: cr_write 0 = 0x80010001
qemu-system-x86-4081 [001] 62650.335074: kvm_entry: vcpu 0
Enabling paging
0009AEC2 B801000180 mov eax,0x80010001
0009AEC7 0F22C0 mov cr0,eax
But before we get here:
0009AECA EA30AF09001000 jmp dword 0x10:0x9af30
... kaboom:
qemu-system-x86-4081 [001] 62650.335076: kvm_exit: reason TRIPLE_FAULT rip 0x0 info 0 0
The page tables are, ahem, crap:
000c000: 6750 fe01 0000 0000 0000 0000 0000 0000 gP..............
000c010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
This is 0x9c000. Strikes any bell?
Paolo
next prev parent reply other threads:[~2013-12-06 12:03 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-05 16:12 [edk2] apparent KVM problem with LRET in TianoCore S3 resume trampoline Laszlo Ersek
2013-12-05 16:50 ` Laszlo Ersek
2013-12-05 17:42 ` Paolo Bonzini
2013-12-05 18:29 ` Laszlo Ersek
2013-12-06 12:03 ` Paolo Bonzini [this message]
2013-12-06 13:31 ` Paolo Bonzini
2013-12-06 13:46 ` Yao, Jiewen
2013-12-06 14:29 ` Paolo Bonzini
2013-12-06 14:47 ` Yao, Jiewen
2013-12-06 14:51 ` Paolo Bonzini
2013-12-06 13:31 ` Yao, Jiewen
2013-12-08 17:43 ` Laszlo Ersek
2013-12-08 22:15 ` Laszlo Ersek
2013-12-05 22:38 ` Laszlo Ersek
2013-12-05 22:53 ` Andrew Fish
2013-12-07 16:25 ` David Woodhouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52A1BCFE.4020100@redhat.com \
--to=pbonzini@redhat.com \
--cc=edk2-devel@lists.sourceforge.net \
--cc=kvm@vger.kernel.org \
--cc=lersek@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox