[PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[Anthony Liguori]

According to 16.2.5 in the SDM, eflags.vm in the tss is consulted before loading
and new segments.  If eflags.vm == 1, then the segments are treated as 16-bit
segments.  The LDTR and TR are not normally available in vm86 mode so if they
happen to somehow get loaded, they need to be treated as 32-bit segments.

This fixes an invalid vmentry failure in a custom OS that was happening after
a task switch into vm8086 mode.  Since the segments were being mistakenly
treated as 32-bit, we loaded garbage state.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

---

v1 -> v2  Simplify by folding check into kvm_load_segment_descriptor()
---
 arch/x86/kvm/x86.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 6263991..f9c9f85 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4007,12 +4007,19 @@ static int kvm_load_realmode_segment(struct kvm_vcpu *vcpu, u16 selector, int se
 	return 0;
 }
 
+static int is_vm86_segment(struct kvm_vcpu *vcpu, int seg)
+{
+	return (seg != VCPU_SREG_LDTR) &&
+		(seg != VCPU_SREG_TR) &&
+		(kvm_x86_ops->get_rflags(vcpu) & X86_EFLAGS_VM);
+}
+
 int kvm_load_segment_descriptor(struct kvm_vcpu *vcpu, u16 selector,
 				int type_bits, int seg)
 {
 	struct kvm_segment kvm_seg;
 
-	if (!(vcpu->arch.cr0 & X86_CR0_PE))
+	if (is_vm86_segment(vcpu, seg) || !(vcpu->arch.cr0 & X86_CR0_PE))
 		return kvm_load_realmode_segment(vcpu, selector, seg);
 	if (load_segment_descriptor_to_kvm_desct(vcpu, selector, &kvm_seg))
 		return 1;
-- 
1.6.2.5

Re: [PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[Avi Kivity]

On 08/11/2009 11:57 PM, Anthony Liguori wrote:
> According to 16.2.5 in the SDM, eflags.vm in the tss is consulted before loading
> and new segments.  If eflags.vm == 1, then the segments are treated as 16-bit
> segments.  The LDTR and TR are not normally available in vm86 mode so if they
> happen to somehow get loaded, they need to be treated as 32-bit segments.
>
> This fixes an invalid vmentry failure in a custom OS that was happening after
> a task switch into vm8086 mode.  Since the segments were being mistakenly
> treated as 32-bit, we loaded garbage state.
>    

Applied, thanks.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[Avi Kivity]

On 08/11/2009 11:57 PM, Anthony Liguori wrote:
> According to 16.2.5 in the SDM, eflags.vm in the tss is consulted before loading
> and new segments.  If eflags.vm == 1, then the segments are treated as 16-bit
> segments.  The LDTR and TR are not normally available in vm86 mode so if they
> happen to somehow get loaded, they need to be treated as 32-bit segments.
>
> This fixes an invalid vmentry failure in a custom OS that was happening after
> a task switch into vm8086 mode.  Since the segments were being mistakenly
> treated as 32-bit, we loaded garbage state.
>    

I'm getting a Vista x64 installer failure on reboot with this.  One 
possibility is that ->get_rflags() is returning the mangled flags state 
(we add vm86 for real mode), which confuses the new code.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[Avi Kivity]

On 08/12/2009 03:11 PM, Avi Kivity wrote:
> On 08/11/2009 11:57 PM, Anthony Liguori wrote:
>> According to 16.2.5 in the SDM, eflags.vm in the tss is consulted 
>> before loading
>> and new segments.  If eflags.vm == 1, then the segments are treated 
>> as 16-bit
>> segments.  The LDTR and TR are not normally available in vm86 mode so 
>> if they
>> happen to somehow get loaded, they need to be treated as 32-bit 
>> segments.
>>
>> This fixes an invalid vmentry failure in a custom OS that was 
>> happening after
>> a task switch into vm8086 mode.  Since the segments were being 
>> mistakenly
>> treated as 32-bit, we loaded garbage state.
>
> I'm getting a Vista x64 installer failure on reboot with this.  One 
> possibility is that ->get_rflags() is returning the mangled flags 
> state (we add vm86 for real mode), which confuses the new code.
>

That's indeed the case, I'm testing a patch now.

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[Anthony Liguori]

Avi Kivity wrote:
> On 08/12/2009 03:11 PM, Avi Kivity wrote:
>> On 08/11/2009 11:57 PM, Anthony Liguori wrote:
>>> According to 16.2.5 in the SDM, eflags.vm in the tss is consulted 
>>> before loading
>>> and new segments.  If eflags.vm == 1, then the segments are treated 
>>> as 16-bit
>>> segments.  The LDTR and TR are not normally available in vm86 mode 
>>> so if they
>>> happen to somehow get loaded, they need to be treated as 32-bit 
>>> segments.
>>>
>>> This fixes an invalid vmentry failure in a custom OS that was 
>>> happening after
>>> a task switch into vm8086 mode.  Since the segments were being 
>>> mistakenly
>>> treated as 32-bit, we loaded garbage state.
>>
>> I'm getting a Vista x64 installer failure on reboot with this.  One 
>> possibility is that ->get_rflags() is returning the mangled flags 
>> state (we add vm86 for real mode), which confuses the new code.
>>
>
> That's indeed the case, I'm testing a patch now.

While the code looks nicer with the second patch, the fact that 
get_rflags() does a vmcs_read() seems 7 times more than before seems 
unfortunate.

-- 
Regards,

Anthony Liguori

Re: [PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[Avi Kivity]

On 08/12/2009 04:15 PM, Anthony Liguori wrote:
> While the code looks nicer with the second patch, the fact that 
> get_rflags() does a vmcs_read() seems 7 times more than before seems 
> unfortunate.

We can add kvm_rflags_read(), see kvm_cache_regs.h.  In any case, it's 
purely academic since task switches are rare and incredibly slow anyway 
(you have to vmcs_write() the entire register set for one).

-- 
error compiling committee.c: too many arguments to function

Re: [PATCH] When switching to a vm8086 task, load segments as 16-bit (v2)

[Anthony Liguori]

Avi Kivity wrote:
> On 08/12/2009 04:15 PM, Anthony Liguori wrote:
>> While the code looks nicer with the second patch, the fact that 
>> get_rflags() does a vmcs_read() seems 7 times more than before seems 
>> unfortunate.
>
> We can add kvm_rflags_read(), see kvm_cache_regs.h.  In any case, it's 
> purely academic since task switches are rare and incredibly slow 
> anyway (you have to vmcs_write() the entire register set for one).

Fair enough.

-- 
Regards,

Anthony Liguori