From: Paolo Bonzini <pbonzini@redhat.com>
To: edk2-devel@lists.sourceforge.net
Cc: Laszlo Ersek <lersek@redhat.com>,
KVM devel mailing list <kvm@vger.kernel.org>
Subject: Re: [edk2] apparent KVM problem with LRET in TianoCore S3 resume trampoline
Date: Fri, 06 Dec 2013 13:03:10 +0100 [thread overview]
Message-ID: <52A1BCFE.4020100@redhat.com> (raw)
In-Reply-To: <52A0C5F4.3040603@redhat.com>
Il 05/12/2013 19:29, Laszlo Ersek ha scritto:
> On 12/05/13 18:42, Paolo Bonzini wrote:
>> Il 05/12/2013 17:12, Laszlo Ersek ha scritto:
>>> Hi,
>>>
>>> I'm working on S3 suspend/resume in OVMF. The problem is that I'm getting an
>>> unexpected guest reboot for code (LRET) that works on physical hardware. I
>>> tried to trace the problem with ftrace, but I didn't get any mentions of
>>> em_ret_far(). (Maybe I was looking in the wrong place.)
>>
>> What does ftrace say anyway?
>
> (pls. see in the next msg I sent)
Actually I meant the ftrace without any patches.
Thanks to your binary I now reproduced the issue and it looks like the
64-bit->16-bit switch works:
qemu-system-x86-4081 [001] 62650.335040: kvm_exit: reason CR_ACCESS rip 0x3cf7ae45 info 0 0
qemu-system-x86-4081 [001] 62650.335041: kvm_cr: cr_write 0 = 0x32
qemu-system-x86-4081 [001] 62650.335046: kvm_entry: vcpu 0
This is the "mov %rax, %cr0". PE and PG are turned off.
qemu-system-x86-4081 [001] 62650.335047: kvm_exit: reason MSR_READ rip 0x3cf7ae4e info 0 0
qemu-system-x86-4081 [001] 62650.335048: kvm_msr: msr_read c0000080 = 0x100
qemu-system-x86-4081 [001] 62650.335048: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335048: kvm_exit: reason MSR_WRITE rip 0x3cf7ae53 info 0 0
qemu-system-x86-4081 [001] 62650.335049: kvm_msr: msr_write c0000080 = 0x0
qemu-system-x86-4081 [001] 62650.335050: kvm_entry: vcpu 0
LME is turned off.
qemu-system-x86-4081 [001] 62650.335050: kvm_exit: reason CR_ACCESS rip 0x3cf7ae55 info 304 0
qemu-system-x86-4081 [001] 62650.335050: kvm_cr: cr_write 4 = 0x640
qemu-system-x86-4081 [001] 62650.335053: kvm_entry: vcpu 0
PAE is turned off.
qemu-system-x86-4081 [001] 62650.335054: kvm_exit: reason CR_ACCESS rip 0x11e6 info 0 0
qemu-system-x86-4081 [001] 62650.335054: kvm_cr: cr_write 0 = 0x33
qemu-system-x86-4081 [001] 62650.335054: kvm_entry: vcpu 0
Here we're already in real mode. The weird RIP is explained by
the first few bytes after the FACS resume vector:
0x9a1d:0000: cli
0x9a1d:0001: cld
0x9a1d:0002: ljmp $9900,$11d7
9900:11d7 is the same physical address as 9a1d:0007. Fast forward a bit:
qemu-system-x86-4081 [001] 62650.335071: kvm_exit: reason CR_ACCESS rip 0x9aec7 info 0 0
qemu-system-x86-4081 [001] 62650.335071: kvm_cr: cr_write 0 = 0x80010001
qemu-system-x86-4081 [001] 62650.335074: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335076: kvm_exit: reason TRIPLE_FAULT rip 0x0 info 0 0
Disassembling mixed 16-/32-/64-bit is a pain, so I ran QEMU with -no-shutdown
-no-reboot and dumped the memory with
(qemu) dump-guest-memory tramp.dmp 0x90000 0x10000
Lets look at the rest of the trace now. After the previous cr0 access we have:
qemu-system-x86-4081 [001] 62650.335055: kvm_exit: reason CR_ACCESS rip 0x11fa info 0 0
qemu-system-x86-4081 [001] 62650.335055: kvm_cr: cr_write 0 = 0x32
qemu-system-x86-4081 [001] 62650.335055: kvm_entry: vcpu 0
It gets out to real mode again. No idea why. It does some setup.
qemu-system-x86-4081 [001] 62650.335056: kvm_exit: reason MSR_WRITE rip 0x1258 info 0 0
qemu-system-x86-4081 [001] 62650.335056: kvm_msr: msr_write 1a0 = 0x1
qemu-system-x86-4081 [001] 62650.335057: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335057: kvm_exit: reason WBINVD rip 0x1001 info 0 0
qemu-system-x86-4081 [001] 62650.335057: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335058: kvm_exit: reason CPUID rip 0x1073 info 0 0
qemu-system-x86-4081 [001] 62650.335058: kvm_cpuid: func 0 rax 4 rbx 756e6547 rcx 6c65746e rdx 49656e69
qemu-system-x86-4081 [001] 62650.335059: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335059: kvm_exit: reason CPUID rip 0x10c0 info 0 0
qemu-system-x86-4081 [001] 62650.335059: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd
qemu-system-x86-4081 [001] 62650.335059: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335060: kvm_exit: reason CPUID rip 0x10ff info 0 0
qemu-system-x86-4081 [001] 62650.335060: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd
qemu-system-x86-4081 [001] 62650.335060: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335061: kvm_exit: reason CPUID rip 0x1117 info 0 0
qemu-system-x86-4081 [001] 62650.335061: kvm_cpuid: func 80000000 rax 8000000a rbx 756e6547 rcx 6c65746e rdx 49656e69
qemu-system-x86-4081 [001] 62650.335061: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335062: kvm_exit: reason CPUID rip 0x1127 info 0 0
qemu-system-x86-4081 [001] 62650.335062: kvm_cpuid: func 80000001 rax 663 rbx 0 rcx 1 rdx 2191abfd
qemu-system-x86-4081 [001] 62650.335062: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335063: kvm_exit: reason CPUID rip 0x113f info 0 0
qemu-system-x86-4081 [001] 62650.335063: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd
qemu-system-x86-4081 [001] 62650.335063: kvm_entry: vcpu 0
qemu-system-x86-4081 [001] 62650.335064: kvm_exit: reason CR_ACCESS rip 0x103c info 0 0
qemu-system-x86-4081 [001] 62650.335064: kvm_cr: cr_write 0 = 0x1
qemu-system-x86-4081 [001] 62650.335064: kvm_entry: vcpu 0
Enabling protected mode:
0009A036 66B801000000 mov eax,0x1
0009A03C 0F22C0 mov cr0,eax
0009A03F 66EA90AE09000800 jmp dword 0x8:0x9ae90
This is a 32-bit selector.
0009AE90 8ED2 mov ss,edx
0009AE92 81C400900900 add esp,0x99000
0009AE98 8EDA mov ds,edx
0009AE9A 8EC2 mov es,edx
0009AE9C 8EE2 mov fs,edx
0009AE9E 8EEA mov gs,edx
qemu-system-x86-4081 [001] 62650.335065: kvm_exit: reason CR_ACCESS rip 0x9aea5 info 4 0
qemu-system-x86-4081 [001] 62650.335065: kvm_cr: cr_write 4 = 0x6f0
qemu-system-x86-4081 [001] 62650.335066: kvm_entry: vcpu 0
Enabling PAE:
0009AEA0 A110D00900 mov eax,[0x9d010]
0009AEA5 0F22E0 mov cr4,eax
qemu-system-x86-4081 [001] 62650.335067: kvm_exit: reason CR_ACCESS rip 0x9aead info 3 0
qemu-system-x86-4081 [001] 62650.335067: kvm_cr: cr_write 3 = 0x9c000
qemu-system-x86-4081 [001] 62650.335068: kvm_entry: vcpu 0
Setting CR3
0009AEA8 B800C00900 mov eax,0x9c000
0009AEAD 0F22D8 mov cr3,eax
qemu-system-x86-4081 [001] 62650.335068: kvm_exit: reason MSR_WRITE rip 0x9aec0 info 0 0
qemu-system-x86-4081 [001] 62650.335070: kvm_msr: msr_write c0000080 = 0x901
qemu-system-x86-4081 [001] 62650.335070: kvm_entry: vcpu 0
Enabling LME
0009AEB0 A108D00900 mov eax,[0x9d008]
0009AEB5 8B150CD00900 mov edx,[dword 0x9d00c]
0009AEBB B9800000C0 mov ecx,0xc0000080
0009AEC0 0F30 wrmsr
qemu-system-x86-4081 [001] 62650.335071: kvm_exit: reason CR_ACCESS rip 0x9aec7 info 0 0
qemu-system-x86-4081 [001] 62650.335071: kvm_cr: cr_write 0 = 0x80010001
qemu-system-x86-4081 [001] 62650.335074: kvm_entry: vcpu 0
Enabling paging
0009AEC2 B801000180 mov eax,0x80010001
0009AEC7 0F22C0 mov cr0,eax
But before we get here:
0009AECA EA30AF09001000 jmp dword 0x10:0x9af30
... kaboom:
qemu-system-x86-4081 [001] 62650.335076: kvm_exit: reason TRIPLE_FAULT rip 0x0 info 0 0
The page tables are, ahem, crap:
000c000: 6750 fe01 0000 0000 0000 0000 0000 0000 gP..............
000c010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000c0f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
This is 0x9c000. Strikes any bell?
Paolo
next prev parent reply other threads:[~2013-12-06 12:03 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-05 16:12 [edk2] apparent KVM problem with LRET in TianoCore S3 resume trampoline Laszlo Ersek
2013-12-05 16:50 ` Laszlo Ersek
2013-12-05 17:42 ` Paolo Bonzini
2013-12-05 18:29 ` Laszlo Ersek
2013-12-06 12:03 ` Paolo Bonzini [this message]
2013-12-06 13:31 ` Paolo Bonzini
2013-12-06 13:46 ` Yao, Jiewen
2013-12-06 14:29 ` Paolo Bonzini
2013-12-06 14:47 ` Yao, Jiewen
2013-12-06 14:51 ` Paolo Bonzini
2013-12-06 13:31 ` Yao, Jiewen
2013-12-08 17:43 ` Laszlo Ersek
2013-12-08 22:15 ` Laszlo Ersek
2013-12-05 22:38 ` Laszlo Ersek
2013-12-05 22:53 ` Andrew Fish
2013-12-07 16:25 ` David Woodhouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52A1BCFE.4020100@redhat.com \
--to=pbonzini@redhat.com \
--cc=edk2-devel@lists.sourceforge.net \
--cc=kvm@vger.kernel.org \
--cc=lersek@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.