Re: HVM x86 deprivileged mode: AMD SVM TR problem

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 20/08/15 10:34, Tim Deegan wrote:
> At 17:36 +0100 on 19 Aug (1440005801), Ben Catterall wrote:
>> On 19/08/15 16:43, Tim Deegan wrote:
>>> At 16:04 +0100 on 19 Aug (1440000260), Ben Catterall wrote:
>>>> I've hit a blocker on getting this working for AMD's SVM and would
>>>> appreciate any thoughts. Hopefully I've missed a much simpler way of
>>>> doing this or I've missed something!
>>>>
>>>> So, AMD and Intel differ in how they handle the TR on a VMEXIT and
>>>> VMRUM. On a VMEXIT, Intel Save the guest's TR and then restore the
>>>> host's TR. AMD do not save the guest's TR nor do they restore the host's
>>>> TR.
>>>>
>>>> So, we need to context switch it out. The only ways that I know of to do
>>>> this are with the ltr and str instructions. Now, ltr will throw #GP if
>>>> loaded with a null selector and, when loaded, will immediately fetch
>>>> from the current GDT the descriptor's data.
>>>>
>>>> After issuing a VMEXIT and moving into deprivileged mode, I need a valid
>>>> TSS so that we can handle exceptions in ring 3, otherwise, thanks to an
>>>> invalid TSS selector in the TR causing a system shutdown (AMD manual),
>>>> the guest could crash the system.
>>>>
>>>> At the moment, I can save the guest's TR, load the host's TR and then
>>>> happily handle exceptions when we are in ring 3 now so that's fixed the
>>>> shutdown issue. But, when moving back to the guest, I have no easy way
>>>> to restore the TR.
>>> I think the CPU will load that state for you from the VMCB when
>>> entering the guest.  (At least, if it doesn't, I don't know how VCPU
>>> migration works at the moment.)  So only the VMEXIT path needs any
>>> attention.
>> This pointed me in another direction, thanks!
>>
>>   From what I've understood, the behaviour of VMEXIT and VMRUN
>> instructions don't save/load that state from the VMCB. Though, if that's
>> the case, I'd also like to know how the migration code works :).
>>
>> However, AMD provides VMSAVE and VMLOAD (section 15.4.4 AMD manual 2)
>> which DO save/load the TR (and other registers)
> Ah, quite right.  E.g., svm_ctxt_switch_to() uses it to load that state on
> context switch.
>
>> I guess if we use this then this alleviates much of the complexity as,
>> looking at what it saves, I think we would be fine to use VMSAVE and
>> VMLOAD just when we are doing a HVM depriv operation, and not need to
>> call them every time we took a VMEXIT and that then gets round this problem.
> ...this looks like a fine plan.  In fact, looking at svm.c, I think
> you can just use hvm_get_segment_register()/hvm_set_segment_register(),
> which will DTRT internally.

(Apologies for being late to this.  I have been travelling.)

This is insufficient.  Xen must reload all host state before moving into 
depriv mode, not just TR.

Consider what would happen if the guest kernel set some crafty values in 
the sysenter MSRs, then used a vulnerability to cause depriv mode to 
execute a sysenter instruction.

Every single item covered by vmsave/vmload (off the top of my head, TR, 
LDTR, ?s_base msrs, syscall msrs, sysenter msrs) are unsafe to leave 
guest controlled while in depriv mode.

~Andrew