Re: Weekly VMX status report. Xen: #18846 & Xen0: #749

[Gianluca Guida <gianluca.guida@eu.citrix.com>]

Hello,

Keir Fraser wrote:
> On 07/12/2008 02:23, "Li, Xin" <xin.li@intel.com> wrote:
> 
>>>> There's a good chance that at least bug #1 is fixed on current tip
>>>> (c/s 18881).
>>> OK, we will check it with c/s 18881, thanks.
>> The root cause of the crash when booting a 64bit Solaris 10u5 guest is that
>> Xen hypervisor has turned off NX as guest AP has not turned on NX, but shadow
>> already has NX set...

This is what I think is going on:

BSP has finished its bootstrap phase, has enabled the EFER's NX bit and 
set the kernel mapping to pages that are going to be used as pagetable 
non-executable.

AP enables long mode, but not the EFER's NX. It accesses an address 
whose guest walk has pages still not shadowed, and the shadow code 
enters the game trying to remove writable mappings of that given guest page.

And here's -- I think -- the bug: when we update the MSR (in context 
switch) it is my understanding that we update the MSR based on the 
guest's vcpu state. So, when the shadow code will try to read the shadow 
mapping of the soon-to-be-promoted page will access a shadow mapping 
with NX bit and get a reserved-bit pagefault, because the host's EFER 
will have NX feature disabled.

I see two ways to fix this:

- Disable NX support in shadows until all vcpus have EFER's NX enabled. 
This would means that the guest thinks it has NX bit protection in at 
least one vcpus but in reality it doesn't. Also, to properly support 
execute-disable protection, we would need to blow the shadows when we 
can finally enable NX bit in shadows.

- Always enable EFER's NX in host mode. We could also avoid changing 
EFER's status between vmentry and vmexits, but this would cause some 
issue in reserved bit handling in page faults. This could be easily 
fixed in shadow code, but in HAP would make the whole thing more 
complicated.

Do the people that know better than me the actual VMX code have any 
opinion about the best way to fix this?

Thanks,
Gianluca