From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Egger, Christoph" Subject: Re: x86/AMD: Nested hvm crashes in 4.3 Date: Fri, 28 Jun 2013 17:05:58 +0200 Message-ID: <51CDA656.9000905@amazon.de> References: <51CB863B.6020405@amd.com> <51CC125E02000078000E10CE@nat28.tlf.novell.com> <51CC03C9.60800@amd.com> <51CC2B2702000078000E1178@nat28.tlf.novell.com> <51CC12EC.7050608@amd.com> <51CC150B.8040001@amazon.de> <51CC1EAB.1060103@amd.com> <51CCDC72.5070008@amd.com> <51CD5E4802000078000E174F@nat28.tlf.novell.com> <51CD9BA3.2060908@amd.com> <51CDBF4802000078000E19D6@nat28.tlf.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <51CDBF4802000078000E19D6@nat28.tlf.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: xen-devel , Jacob Shin , Suravee Suthikulanit , Sherry Hurwitz List-Id: xen-devel@lists.xenproject.org On 28.06.13 16:52, Jan Beulich wrote: >>>> On 28.06.13 at 16:20, Suravee Suthikulanit > wrote: >> On 6/28/2013 2:58 AM, Jan Beulich wrote: >>>>>> On 28.06.13 at 02:44, Suravee Suthikulanit >> wrote: >>>> So, I have finally able to get the crash dump (see below). The crash is due >>>> to an assert >>>> >>>> (XEN) Assertion 'va >= XEN_VIRT_START' failed at >>>> /sandbox/xen/xen.git/xen/include/asm/x86_64/page.h:86 >>>> >>>> * Debugging show the va=ffff82c40002d000, XEN_VIRT_START=ffff82c4c0000000, >>>> DIRECTMAP_VIRT_END=ffffff8000000000. >>>> * Backtrace symbol showing the crash is in "svm_vmexit_handler()", which is >>>> inlined from "svm_vmexit_do_vmsave()" and "svm_vmsave()". >>> Which helps in no way identifying where the problem is - >>> svm_vmexit_handler() is just too large to spot this without either >>> the matching xen-syms at hand, or you adding further >>> instrumentation. >>> >>> Jan >> >> What I am trying to say is, the assertion is in the __virt_to_maddr which is >> called from >> svm_vmexit_do_vmsave(). However, this is a bit complicate due to macros and >> inlines. >> Here is the callchain supposed to look like: >> >> ASSERT(va >= XEN_VIRT_START ) >> __virt_to_maddr <---- inlined >> virt_to_mfn () <---- macro >> __pa () <---- macro >> smv_vmasave() <---- inlined >> svm_vmexit_do_vmsave() <---- inlined >> svm_vmexit_handler() <---- symbol > > So the problem is the inverse of the usual one (and that's part of > why I didn't spot it when searching the tree for code that needs > fixing; the other part is that while running into these functions I > knew that VMCBs get allocated from the Xen heap, but didn't > notice that the same functions also get used for dealing with > guest VMCBs): > > nestedsvm_vmcb_map() properly does the necessary mapping, > but svm_vmsave() (just like svm_vmload()) blindly uses __pa() on > something that's not an address in the direct mapping region. > Which means that on 4.2.0, where we still had a 32-bit hypervisor, > nested SVM was completely broken (and presumably never tested) > in that 32-bit case. Luckily we meanwhile disabled the use of nested > HVM in 4.2.x's 32-bit builds. I never tested nested svm on 32bit on the host. I did test 32bit hypervisors as guest. Christoph