Should hvm_hw_cpu's sysenter_eip always be zero after xc_domain_hvm_getcontext_partial()?

Should hvm_hw_cpu's sysenter_eip always be zero after xc_domain_hvm_getcontext_partial()?

[Razvan Cojocaru]

Hello,

It has become apparent that after a succesful
xc_domain_hvm_getcontext_partial( ..., HVM_SAVE_CODE(CPU), ...) call,
the sysenter_eip member of struct hvm_hw_cpu is always zero.

Looking in the code, the only two __vmwrite(GUEST_SYSENTER_EIP, ...)
calls occur in xen/arch/x86/hvm/vmx/vmx.c. One is in
vmx_msr_write_intercept(), but adding a printk() just after produces no
output after starting and stopping a guest.

The other is in vmx_vmcs_restore(), which seems to dutifully restore the
never-set value of zero after a save.

So this doesn't seem to be actually initialized anywhere. Could somebody
please recommend the best place to initialize it, and the best value to
initialize it with? Or maybe you could point out what I'm missing, if
that's the case?


Thanks,
Razvan

Re: Should hvm_hw_cpu's sysenter_eip always be zero after xc_domain_hvm_getcontext_partial()?

[Andrew Cooper]

On 31/07/15 16:00, Razvan Cojocaru wrote:
> Hello,
>
> It has become apparent that after a succesful
> xc_domain_hvm_getcontext_partial( ..., HVM_SAVE_CODE(CPU), ...) call,
> the sysenter_eip member of struct hvm_hw_cpu is always zero.

How odd.

>
> Looking in the code, the only two __vmwrite(GUEST_SYSENTER_EIP, ...)
> calls occur in xen/arch/x86/hvm/vmx/vmx.c. One is in
> vmx_msr_write_intercept(), but adding a printk() just after produces no
> output after starting and stopping a guest.

Write interception for that MSR is disabled (if the hardware supports
disabling interception) because it is not interesting.

You can re-enable interception by commenting out the appropriate
vmx_disable_intercept_for_msr() line in construct_vmcs()

>
> The other is in vmx_vmcs_restore(), which seems to dutifully restore the
> never-set value of zero after a save.
>
> So this doesn't seem to be actually initialized anywhere. Could somebody
> please recommend the best place to initialize it, and the best value to
> initialize it with? Or maybe you could point out what I'm missing, if
> that's the case?

Are you certain that the guest is actually setting it?  If the guest
never sets it in the first place, 0 will be the expected value.

~Andrew

Re: Should hvm_hw_cpu's sysenter_eip always be zero after xc_domain_hvm_getcontext_partial()?

[Razvan Cojocaru]

Thanks for the quick reply!

On 07/31/2015 06:13 PM, Andrew Cooper wrote:
>> Looking in the code, the only two __vmwrite(GUEST_SYSENTER_EIP, ...)
>> calls occur in xen/arch/x86/hvm/vmx/vmx.c. One is in
>> vmx_msr_write_intercept(), but adding a printk() just after produces no
>> output after starting and stopping a guest.
> 
> Write interception for that MSR is disabled (if the hardware supports
> disabling interception) because it is not interesting.
> 
> You can re-enable interception by commenting out the appropriate
> vmx_disable_intercept_for_msr() line in construct_vmcs()

It shouldn't be disabled, it's in the
vmx_introspection_force_enabled_msrs[] list and I'm testing it with
memory introspection.

>> The other is in vmx_vmcs_restore(), which seems to dutifully restore the
>> never-set value of zero after a save.
>>
>> So this doesn't seem to be actually initialized anywhere. Could somebody
>> please recommend the best place to initialize it, and the best value to
>> initialize it with? Or maybe you could point out what I'm missing, if
>> that's the case?
> 
> Are you certain that the guest is actually setting it?  If the guest
> never sets it in the first place, 0 will be the expected value.

That might be it, I'll need to look closer.


Thanks,
Razvan

Re: Should hvm_hw_cpu's sysenter_eip always be zero after xc_domain_hvm_getcontext_partial()?

[Andrew Cooper]

On 31/07/15 16:30, Razvan Cojocaru wrote:
> Thanks for the quick reply!
>
> On 07/31/2015 06:13 PM, Andrew Cooper wrote:
>>> Looking in the code, the only two __vmwrite(GUEST_SYSENTER_EIP, ...)
>>> calls occur in xen/arch/x86/hvm/vmx/vmx.c. One is in
>>> vmx_msr_write_intercept(), but adding a printk() just after produces no
>>> output after starting and stopping a guest.
>> Write interception for that MSR is disabled (if the hardware supports
>> disabling interception) because it is not interesting.
>>
>> You can re-enable interception by commenting out the appropriate
>> vmx_disable_intercept_for_msr() line in construct_vmcs()
> It shouldn't be disabled, it's in the
> vmx_introspection_force_enabled_msrs[] list and I'm testing it with
> memory introspection.

So it is - I thought it might have been, but couldn't locate it in the code.

>
>>> The other is in vmx_vmcs_restore(), which seems to dutifully restore the
>>> never-set value of zero after a save.
>>>
>>> So this doesn't seem to be actually initialized anywhere. Could somebody
>>> please recommend the best place to initialize it, and the best value to
>>> initialize it with? Or maybe you could point out what I'm missing, if
>>> that's the case?
>> Are you certain that the guest is actually setting it?  If the guest
>> never sets it in the first place, 0 will be the expected value.
> That might be it, I'll need to look closer.

Any credible 64bit kernel will not rely on sysenter, as it only
available on Intel, and any kernel trying to reduce its attack surface
will likely not enable it in the first place.

~Andrew

Re: Should hvm_hw_cpu's sysenter_eip always be zero after xc_domain_hvm_getcontext_partial()?

[Jan Beulich]

>>> On 31.07.15 at 17:36, <andrew.cooper3@citrix.com> wrote:
> On 31/07/15 16:30, Razvan Cojocaru wrote:
>> On 07/31/2015 06:13 PM, Andrew Cooper wrote:
>>> Are you certain that the guest is actually setting it?  If the guest
>>> never sets it in the first place, 0 will be the expected value.
>> That might be it, I'll need to look closer.
> 
> Any credible 64bit kernel will not rely on sysenter, as it only
> available on Intel, and any kernel trying to reduce its attack surface
> will likely not enable it in the first place.

Iirc Linux still prefers to use SYSENTER for its 32-bit compat mode
syscall entry on Intel systems.

Jan