Re: x86/AMD: Nested hvm crashes in 4.3

["Egger, Christoph" <chegger@amazon.de>]

On 28.06.13 16:52, Jan Beulich wrote:
>>>> On 28.06.13 at 16:20, Suravee Suthikulanit <suravee.suthikulpanit@amd.com>
> wrote:
>> On 6/28/2013 2:58 AM, Jan Beulich wrote:
>>>>>> On 28.06.13 at 02:44, Suravee Suthikulanit <suravee.suthikulpanit@amd.com> 
>> wrote:
>>>> So, I have finally able to get the crash dump (see below). The crash is due
>>>> to an assert
>>>>
>>>>       (XEN) Assertion 'va >= XEN_VIRT_START' failed at
>>>> /sandbox/xen/xen.git/xen/include/asm/x86_64/page.h:86
>>>>
>>>> * Debugging show the va=ffff82c40002d000, XEN_VIRT_START=ffff82c4c0000000,
>>>> DIRECTMAP_VIRT_END=ffffff8000000000.
>>>> * Backtrace symbol showing the crash is in "svm_vmexit_handler()", which is
>>>> inlined from "svm_vmexit_do_vmsave()" and "svm_vmsave()".
>>> Which helps in no way identifying where the problem is -
>>> svm_vmexit_handler() is just too large to spot this without either
>>> the matching xen-syms at hand, or you adding further
>>> instrumentation.
>>>
>>> Jan
>>
>> What I am trying to say is, the assertion is in the __virt_to_maddr which is 
>> called from
>> svm_vmexit_do_vmsave().  However, this is a bit complicate due to macros and 
>> inlines.
>> Here is the callchain supposed to look like:
>>
>>      ASSERT(va >= XEN_VIRT_START )
>>      __virt_to_maddr        <---- inlined
>>      virt_to_mfn ()         <---- macro
>>      __pa ()                <---- macro
>>      smv_vmasave()          <---- inlined
>>      svm_vmexit_do_vmsave() <---- inlined
>>      svm_vmexit_handler()   <---- symbol
> 
> So the problem is the inverse of the usual one (and that's part of
> why I didn't spot it when searching the tree for code that needs
> fixing; the other part is that while running into these functions I
> knew that VMCBs get allocated from the Xen heap, but didn't
> notice that the same functions also get used for dealing with
> guest VMCBs):
> 
> nestedsvm_vmcb_map() properly does the necessary mapping,
> but svm_vmsave() (just like svm_vmload()) blindly uses __pa() on
> something that's not an address in the direct mapping region.
> Which means that on 4.2.0, where we still had a 32-bit hypervisor,
> nested SVM was completely broken (and presumably never tested)
> in that 32-bit case. Luckily we meanwhile disabled the use of nested
> HVM in 4.2.x's 32-bit builds.

I never tested nested svm on 32bit on the host. I did test
32bit hypervisors as guest.

Christoph