From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: Intermittent fatal page fault with XEN 4.3.1 (Centos 6.3 DOM0 with linux kernel 3.10.16.) Date: Thu, 7 Nov 2013 15:54:18 +0000 Message-ID: <527BB7AA.4080801@citrix.com> References: <5278D0BB02000078000FF6E4@nat28.tlf.novell.com> <527912E9.3010304@xen.org> <5E2B3362-4D93-4FEF-987A-E477B0DCEE51@mcafee.com> <527A5BB50200007800100255@nat28.tlf.novell.com> <4E8EC677-8D22-4672-9C5C-12DD094107D2@McAfee.com> <1383754737.26213.136.camel@kazak.uk.xensource.com> <527A7454.9060309@citrix.com> <1383757590.26213.139.camel@kazak.uk.xensource.com> <527A7769.8070700@citrix.com> <527B67280200007800100817@nat28.tlf.novell.com> <1383816652.26213.143.camel@kazak.uk.xensource.com> <24B7017F-1AC7-4900-93EB-F54C11002491@McAfee.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1VeRuY-0003CS-GL for xen-devel@lists.xenproject.org; Thu, 07 Nov 2013 15:54:22 +0000 In-Reply-To: <24B7017F-1AC7-4900-93EB-F54C11002491@McAfee.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jeff_Zimmerman@McAfee.com Cc: lars.kurth.xen@gmail.com, xen-devel@lists.xenproject.org, lars.kurth@xen.org, Ian.Campbell@citrix.com, JBeulich@suse.com List-Id: xen-devel@lists.xenproject.org On 07/11/13 15:41, Jeff_Zimmerman@McAfee.com wrote: > On Nov 7, 2013, at 1:30 AM, Ian Campbell > wrote: > >> On Thu, 2013-11-07 at 09:10 +0000, Jan Beulich wrote: >>>>>> On 06.11.13 at 18:07, Andrew Cooper wrote: >>>> On 06/11/13 17:06, Ian Campbell wrote: >>>>> On Wed, 2013-11-06 at 16:54 +0000, Andrew Cooper wrote: >>>>>> I looked over the xl code and thought that nestedhvm should default to >>>>>> false, but I would prefer someone more familar with libxl and the idl to >>>>>> confirm what the default should be. >>>>> libxl thinks the default is false and will set HVM_PARAM_NESTEDHVM to 0 >>>>> in that case. Is there some way to query the hypervisor for what it >>>>> thinks the setting is? >>>> A get hvmparam hypercall will retrieve the value, but it is initialised >>>> to 0 and only ever set by a set hvmparam hypercall. >>> Which makes me start suspecting that the guest might be deriving >>> its information on VMX being available from something other than >>> CPUID. Of course we ought to confirm that we don't unintentionally >>> return the VMX flag set (and that the config file doesn't override it >>> in this way - I think we shouldn't be suppressing user overrides >>> here, but I didn't go check whether we do). >> I was also wondering about the behaviour of using vmx instructions in a >> guest despite vmx not being visible in cpuid... >> >> Ian. >> >> > We have found in our situation this is exactly the case. To verify we wrote some > test code that makes vmx calls without checking cupid. On bare hardware the program > executes as expected. In a VM on Xen it causes the hypervisor to panic. > > From a security standpoint this is very very bad. It might be a good idea to provide either > a run-time or build-time option to disable nestedhvm. Just turning off the vmx bit is not enough > as malicious or badly written code can cause a system crash. > > For us it looks like we can disable these instructions and avoid the crash. > > Jeff. Hmm - that is very concerning that. And there does look to be a bug. Can you try the following patch and see whether it helps? diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h index c9afb56..7b1a349 100644 --- a/xen/include/asm-x86/hvm/hvm.h +++ b/xen/include/asm-x86/hvm/hvm.h @@ -359,7 +359,7 @@ static inline int hvm_event_pending(struct vcpu *v) /* These bits in CR4 cannot be set by the guest. */ #define HVM_CR4_GUEST_RESERVED_BITS(_v) \ (~((unsigned long) \ - (X86_CR4_VME | X86_CR4_PVI | X86_CR4_TSD | \ + (X86_CR4_PVI | X86_CR4_TSD | \ X86_CR4_DE | X86_CR4_PSE | X86_CR4_PAE | \ X86_CR4_MCE | X86_CR4_PGE | X86_CR4_PCE | \ X86_CR4_OSFXSR | X86_CR4_OSXMMEXCPT | \ ~Andrew