Re: Intermittent fatal page fault with XEN 4.3.1 (Centos 6.3 DOM0 with linux kernel 3.10.16.)

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 07/11/13 15:41, Jeff_Zimmerman@McAfee.com wrote:
> On Nov 7, 2013, at 1:30 AM, Ian Campbell <Ian.Campbell@citrix.com>
>  wrote:
>
>> On Thu, 2013-11-07 at 09:10 +0000, Jan Beulich wrote:
>>>>>> On 06.11.13 at 18:07, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>>> On 06/11/13 17:06, Ian Campbell wrote:
>>>>> On Wed, 2013-11-06 at 16:54 +0000, Andrew Cooper wrote:
>>>>>> I looked over the xl code and thought that nestedhvm should default to
>>>>>> false, but I would prefer someone more familar with libxl and the idl to
>>>>>> confirm what the default should be.
>>>>> libxl thinks the default is false and will set HVM_PARAM_NESTEDHVM to 0
>>>>> in that case. Is there some way to query the hypervisor for what it
>>>>> thinks the setting is?
>>>> A get hvmparam hypercall will retrieve the value, but it is initialised
>>>> to 0 and only ever set by a set hvmparam hypercall.
>>> Which makes me start suspecting that the guest might be deriving
>>> its information on VMX being available from something other than
>>> CPUID. Of course we ought to confirm that we don't unintentionally
>>> return the VMX flag set (and that the config file doesn't override it
>>> in this way - I think we shouldn't be suppressing user overrides
>>> here, but I didn't go check whether we do).
>> I was also wondering about the behaviour of using vmx instructions in a
>> guest despite vmx not being visible in cpuid...
>>
>> Ian.
>>
>>
> We have found in our situation this is exactly the case. To verify we wrote some
> test code that makes vmx calls without checking cupid. On bare hardware the program
> executes as expected. In a VM on Xen it causes the hypervisor to panic.
>
> From a security standpoint this is very very bad. It might be a good idea to provide either
> a run-time or build-time option to disable nestedhvm. Just turning off the vmx bit is not enough
> as malicious or badly written code can cause a system crash.
>
> For us it looks like we can disable these instructions and avoid the crash.
>
> Jeff.

Hmm - that is very concerning that.

And there does look to be a bug.

Can you try the following patch and see whether it helps?

diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h
index c9afb56..7b1a349 100644
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@@ -359,7 +359,7 @@ static inline int hvm_event_pending(struct vcpu *v)
 /* These bits in CR4 cannot be set by the guest. */
 #define HVM_CR4_GUEST_RESERVED_BITS(_v)                 \
     (~((unsigned long)                                  \
-       (X86_CR4_VME | X86_CR4_PVI | X86_CR4_TSD |       \
+       (X86_CR4_PVI | X86_CR4_TSD |                     \
         X86_CR4_DE  | X86_CR4_PSE | X86_CR4_PAE |       \
         X86_CR4_MCE | X86_CR4_PGE | X86_CR4_PCE |       \
         X86_CR4_OSFXSR | X86_CR4_OSXMMEXCPT |           \

~Andrew