[PATCH] enable gfxboot on VMX

[PATCH] enable gfxboot on VMX

[Alexander Graf]

[-- Attachment #1: Type: text/plain, Size: 1267 bytes --]

Hi,

this issue has already been talked about previously. Gfxboot on VMX is
broken, because it reads SS after switching from real to protected mode,
where SS contains an invalid value, which VMX does not allow.
As far as I know, gfxboot is the only application that suffers from this
issue.
The current "fix" is to make gfxboot use a previously stored SS value,
which works fine for new releases. Already shipped versions of the
software can not be changed though, so there needs to be another way to
make kvm work with older versions of gfxboot.

As everything except gfxboot works, we can simply change gfxboot in
runtime to use a different value. Unfortunately the mov instruction,
used to read the SS register is only 2 bytes long, so there is no way to
binary patch the mov to something that would contain an address. So the
only way I could think of was an invalid instruction. The UD exception
is intercepted in KVM and is already emulated for VMCALLs. This can be
extended to an opcode, that is officially unused (0f 0c) and have the
emulator do a mov realmode_ss, %eax.

This patch implements exactly this idea and fixes openSUSE < 11.0 and
Ubuntu CD booting on VMX for me. Comments are, as always, welcome.

Signed-off-by: Alexander Graf <alex@csgraf.de>



[-- Attachment #2: kvm-gfxboot.patch --]
[-- Type: text/x-patch, Size: 4326 bytes --]

diff -ur kernel/include/asm-x86/kvm_host.h kernel.patched/include/asm-x86/kvm_host.h
--- kernel/include/asm-x86/kvm_host.h	2008-02-15 07:59:28.000000000 +0100
+++ kernel.patched/include/asm-x86/kvm_host.h	2008-02-15 07:42:41.000000000 +0100
@@ -195,6 +195,7 @@
 	unsigned long regs[NR_VCPU_REGS]; /* for rsp: vcpu_load_rsp_rip() */
 	unsigned long rip;      /* needs vcpu_load_rsp_rip() */
 
+	u16 backup_ss;
 	unsigned long cr0;
 	unsigned long cr2;
 	unsigned long cr3;
diff -ur kernel/vmx.c kernel.patched/vmx.c
--- kernel/vmx.c	2008-02-15 07:59:28.000000000 +0100
+++ kernel.patched/vmx.c	2008-02-15 07:51:27.000000000 +0100
@@ -1112,6 +1112,8 @@
 static void enter_pmode(struct kvm_vcpu *vcpu)
 {
 	unsigned long flags;
+	unsigned long rip;
+	u8 opcodes[2];
 
 	vcpu->arch.rmode.active = 0;
 
@@ -1134,12 +1136,39 @@
 	fix_pmode_dataseg(VCPU_SREG_GS, &vcpu->arch.rmode.gs);
 	fix_pmode_dataseg(VCPU_SREG_FS, &vcpu->arch.rmode.fs);
 
+	/* Save real mode SS */
+	vcpu->arch.backup_ss = vmcs_read16(GUEST_SS_SELECTOR);
+
 	vmcs_write16(GUEST_SS_SELECTOR, 0);
 	vmcs_write32(GUEST_SS_AR_BYTES, 0x93);
 
 	vmcs_write16(GUEST_CS_SELECTOR,
 		     vmcs_read16(GUEST_CS_SELECTOR) & ~SELECTOR_RPL_MASK);
 	vmcs_write32(GUEST_CS_AR_BYTES, 0x9b);
+
+	/* VMX checks for SS.CPL = CS.CPL on VM entry, if we are in
+	 * protected mode. This fails on the transistion from real mode
+	 * to protected mode, as just after that, SS still contains the
+	 * real mode segment, which does not know anything about CPLs.
+	 * 
+	 * As far as I know only gfxboot exploits this feature, by using
+	 * the old real mode SS value to find a new SS selector in protected
+	 * mode. This happens using a mov %ss, %eax instruction, which we
+	 * can patch to an invalid opcode and emulate later on, giving eax
+	 * the real SS value, that existed before the protected mode
+	 * switch. */
+	rip = vcpu->arch.rip + vmcs_readl(GUEST_CS_BASE) + 14;
+	emulator_read_std(rip, (void *)opcodes, 2, vcpu);
+
+	if ( opcodes[0] ==  0x8c && opcodes[1] == 0xd0 ) {
+		vcpu_printf(vcpu, "%s: patching mov SS\n", __FUNCTION__);
+		opcodes[0] = 0x0f;
+		opcodes[1] = 0x0c;
+		if (emulator_write_emulated(rip, opcodes,
+		    2, vcpu) != X86EMUL_CONTINUE)
+			vcpu_printf(vcpu, "%s: unable to patch mov SS\n",
+				__FUNCTION__);
+	}
 }
 
 static gva_t rmode_tss_base(struct kvm *kvm)
diff -ur kernel/x86.c kernel.patched/x86.c
--- kernel/x86.c	2008-02-15 07:59:28.000000000 +0100
+++ kernel.patched/x86.c	2008-02-14 16:47:27.000000000 +0100
@@ -1886,13 +1886,14 @@
 
 		r = x86_decode_insn(&vcpu->arch.emulate_ctxt, &emulate_ops);
 
-		/* Reject the instructions other than VMCALL/VMMCALL when
+		/* Reject the instructions other than VMCALL/VMMCALL/HACKS when
 		 * try to emulate invalid opcode */
 		c = &vcpu->arch.emulate_ctxt.decode;
 		if ((emulation_type & EMULTYPE_TRAP_UD) &&
-		    (!(c->twobyte && c->b == 0x01 &&
+		    ((!(c->twobyte && c->b == 0x01 &&
 		      (c->modrm_reg == 0 || c->modrm_reg == 3) &&
-		       c->modrm_mod == 3 && c->modrm_rm == 1)))
+		       c->modrm_mod == 3 && c->modrm_rm == 1)) &&
+		       c->b != 0x0c))
 			return EMULATE_FAIL;
 
 		++vcpu->stat.insn_emulation;
diff -ur kernel/x86_emulate.c kernel.patched/x86_emulate.c
--- kernel/x86_emulate.c	2008-02-15 07:59:28.000000000 +0100
+++ kernel.patched/x86_emulate.c	2008-02-15 07:53:39.000000000 +0100
@@ -174,7 +175,7 @@
 static u16 twobyte_table[256] = {
 	/* 0x00 - 0x0F */
 	0, SrcMem | ModRM | DstReg, 0, 0, 0, 0, ImplicitOps, 0,
-	ImplicitOps, ImplicitOps, 0, 0, 0, ImplicitOps | ModRM, 0, 0,
+	ImplicitOps, ImplicitOps, 0, 0, ImplicitOps, ImplicitOps | ModRM, 0, 0,
 	/* 0x10 - 0x1F */
 	0, 0, 0, 0, 0, 0, 0, 0, ImplicitOps | ModRM, 0, 0, 0, 0, 0, 0, 0,
 	/* 0x20 - 0x2F */
@@ -1744,6 +1745,16 @@
 	case 0x18:		/* Grp16 (prefetch/nop) */
 		c->dst.type = OP_NONE;
 		break;
+	case 0x0c: /* Invalid (used to patch mov %ss, %eax) */
+		/* This opcode is declared invalid, according to the Intel
+		 * specification. As it is only used on VMX, we do not have
+		 * to take AMD instructions into account. For more
+		 * information, why this is needed, please see
+		 * vmx.c:enter_pmode.
+		 */
+		c->dst.type = OP_NONE;
+		c->regs[VCPU_REGS_RAX] = ctxt->vcpu->arch.backup_ss;
+		break;
 	case 0x20: /* mov cr, reg */
 		if (c->modrm_mod != 3)
 			goto cannot_emulate;

[-- Attachment #3: Type: text/plain, Size: 228 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[-- Attachment #4: Type: text/plain, Size: 158 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [PATCH] enable gfxboot on VMX

[Avi Kivity]

Alexander Graf wrote:
> Hi,
>
> this issue has already been talked about previously. Gfxboot on VMX is
> broken, because it reads SS after switching from real to protected mode,
> where SS contains an invalid value, which VMX does not allow.
> As far as I know, gfxboot is the only application that suffers from this
> issue.
> The current "fix" is to make gfxboot use a previously stored SS value,
> which works fine for new releases. Already shipped versions of the
> software can not be changed though, so there needs to be another way to
> make kvm work with older versions of gfxboot.
>
> As everything except gfxboot works, we can simply change gfxboot in
> runtime to use a different value. Unfortunately the mov instruction,
> used to read the SS register is only 2 bytes long, so there is no way to
> binary patch the mov to something that would contain an address. So the
> only way I could think of was an invalid instruction. The UD exception
> is intercepted in KVM and is already emulated for VMCALLs. This can be
> extended to an opcode, that is officially unused (0f 0c) and have the
> emulator do a mov realmode_ss, %eax.
>
> This patch implements exactly this idea and fixes openSUSE < 11.0 and
> Ubuntu CD booting on VMX for me. Comments are, as always, welcome.
>   

While enabling gfxboot over vmx is very desirable, I'd like to avoid 
guest-specific hacks.  IMO the correct fix is to set a "non_vt_friendly" 
flag when switching from real mode to protected mode, then continue 
emulation, re-computing the flag after every instruction.  After a few 
instruction, the condition disappears and we can enter guest mode again.

The same approach works for big real mode.

The downside is that we have to implement more instructions in the 
emulator for this, but these instructions will be generally useful, not 
just for gfxboot.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Alexander Graf]

On Feb 15, 2008, at 2:47 PM, Avi Kivity wrote:

> Alexander Graf wrote:
>> Hi,
>>
>> this issue has already been talked about previously. Gfxboot on VMX  
>> is
>> broken, because it reads SS after switching from real to protected  
>> mode,
>> where SS contains an invalid value, which VMX does not allow.
>> As far as I know, gfxboot is the only application that suffers from  
>> this
>> issue.
>> The current "fix" is to make gfxboot use a previously stored SS  
>> value,
>> which works fine for new releases. Already shipped versions of the
>> software can not be changed though, so there needs to be another  
>> way to
>> make kvm work with older versions of gfxboot.
>>
>> As everything except gfxboot works, we can simply change gfxboot in
>> runtime to use a different value. Unfortunately the mov instruction,
>> used to read the SS register is only 2 bytes long, so there is no  
>> way to
>> binary patch the mov to something that would contain an address. So  
>> the
>> only way I could think of was an invalid instruction. The UD  
>> exception
>> is intercepted in KVM and is already emulated for VMCALLs. This can  
>> be
>> extended to an opcode, that is officially unused (0f 0c) and have the
>> emulator do a mov realmode_ss, %eax.
>>
>> This patch implements exactly this idea and fixes openSUSE < 11.0 and
>> Ubuntu CD booting on VMX for me. Comments are, as always, welcome.
>>
>
> While enabling gfxboot over vmx is very desirable, I'd like to avoid
> guest-specific hacks.  IMO the correct fix is to set a  
> "non_vt_friendly"
> flag when switching from real mode to protected mode, then continue
> emulation, re-computing the flag after every instruction.  After a few
> instruction, the condition disappears and we can enter guest mode  
> again.

So when would the flag disappear? Basically an SS read can happen any  
time after the switch, even after 100 instructions if it has to. While  
this should fix more problems, the one thing I am concerned about is  
that I have not encountered any other code that does have this problem.

> The same approach works for big real mode.

I agree on that.

> The downside is that we have to implement more instructions in the
> emulator for this, but these instructions will be generally useful,  
> not
> just for gfxboot.

I am not trying to talk you into anything - I would very much prefer a  
rather clean solution as well. Nevertheless I do not see full  
protected mode emulation code coming in the very near future and on a  
user perspective would prefer to have something that works, even if  
it's ugly.
So while KVM is able to run most (if not all?) current major Operating  
Systems unmodified, it fails to install them (at least on the Linux  
side).

Even though I would greatly appreciate any effort made to get things  
cleaned up, the gfxboot issue has been standing for months now without  
even a hacky workaround (except for disabling gfxboot in all) or any  
visible progress, so I believe a hack like this is at least worth  
something to distributions that want to enable their users to work  
with KVM.

Alex

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Anthony Liguori]

Alexander Graf wrote:
> Hi,
>
> this issue has already been talked about previously. Gfxboot on VMX is
> broken, because it reads SS after switching from real to protected mode,
> where SS contains an invalid value, which VMX does not allow.
> As far as I know, gfxboot is the only application that suffers from this
> issue.
> The current "fix" is to make gfxboot use a previously stored SS value,
> which works fine for new releases. Already shipped versions of the
> software can not be changed though, so there needs to be another way to
> make kvm work with older versions of gfxboot.
>
> As everything except gfxboot works, we can simply change gfxboot in
> runtime to use a different value. Unfortunately the mov instruction,
> used to read the SS register is only 2 bytes long, so there is no way to
> binary patch the mov to something that would contain an address. So the
> only way I could think of was an invalid instruction. The UD exception
> is intercepted in KVM and is already emulated for VMCALLs. This can be
> extended to an opcode, that is officially unused (0f 0c) and have the
> emulator do a mov realmode_ss, %eax.
>
> This patch implements exactly this idea and fixes openSUSE < 11.0 and
> Ubuntu CD booting on VMX for me. Comments are, as always, welcome.
>   

Have you tried SLES-9 or openSUSE variants of the same age?  The ss 
issue in gfxboot is only something recently introduced.  Prior to that, 
gfxboot used big real mode so your patch wouldn't be sufficient for 
those versions of gfxboot.

One thing I've thought about is converting gfxboot-disable[1] to 
generate a qcow2 that backs to the actual CDROM ISO.  Then in QEMU we 
could take an MD5 of an ISO if trying to boot from it, compare it to a 
white list of known bad CDs, and then generate a qcow2 automatically 
with gfxboot disabled.  When we eventually support big real mode in the 
kernel, we can disable this.

[1] http://hg.codemonkey.ws/gfxboot-disable

Regards,

Anthony Liguori


> Signed-off-by: Alexander Graf <alex@csgraf.de>
>
>
>   
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> ------------------------------------------------------------------------
>
> _______________________________________________
> kvm-devel mailing list
> kvm-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/kvm-devel


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Alexander Graf]

On Feb 15, 2008, at 3:56 PM, Anthony Liguori wrote:

> Alexander Graf wrote:
>> Hi,
>>
>> this issue has already been talked about previously. Gfxboot on VMX  
>> is
>> broken, because it reads SS after switching from real to protected  
>> mode,
>> where SS contains an invalid value, which VMX does not allow.
>> As far as I know, gfxboot is the only application that suffers from  
>> this
>> issue.
>> The current "fix" is to make gfxboot use a previously stored SS  
>> value,
>> which works fine for new releases. Already shipped versions of the
>> software can not be changed though, so there needs to be another  
>> way to
>> make kvm work with older versions of gfxboot.
>>
>> As everything except gfxboot works, we can simply change gfxboot in
>> runtime to use a different value. Unfortunately the mov instruction,
>> used to read the SS register is only 2 bytes long, so there is no  
>> way to
>> binary patch the mov to something that would contain an address. So  
>> the
>> only way I could think of was an invalid instruction. The UD  
>> exception
>> is intercepted in KVM and is already emulated for VMCALLs. This can  
>> be
>> extended to an opcode, that is officially unused (0f 0c) and have the
>> emulator do a mov realmode_ss, %eax.
>>
>> This patch implements exactly this idea and fixes openSUSE < 11.0 and
>> Ubuntu CD booting on VMX for me. Comments are, as always, welcome.
>>
>
> Have you tried SLES-9 or openSUSE variants of the same age?  The ss  
> issue in gfxboot is only something recently introduced.  Prior to  
> that, gfxboot used big real mode so your patch wouldn't be  
> sufficient for those versions of gfxboot.

SLES7 - SLES-9  and SUSE 9.1 through to openSUSE 10.1 do not need the  
patch. They work 'as is'. SLES10 starts in text mode.
Starting with 10.2 the mov ss issue came along, but maybe Steffen can  
tell us more about the history of this issue.

> One thing I've thought about is converting gfxboot-disable[1] to  
> generate a qcow2 that backs to the actual CDROM ISO.  Then in QEMU  
> we could take an MD5 of an ISO if trying to boot from it, compare it  
> to a white list of known bad CDs, and then generate a qcow2  
> automatically with gfxboot disabled.  When we eventually support big  
> real mode in the kernel, we can disable this.

This sounds rather hacky too and does not enable the use of gfxboot.

Regards,

Alex

>
>
> [1] http://hg.codemonkey.ws/gfxboot-disable
>
> Regards,
>
> Anthony Liguori
>
>
>> Signed-off-by: Alexander Graf <alex@csgraf.de>
>>
>>
>>   
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> kvm-devel mailing list
>> kvm-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Steffen Winterfeldt]

On Fri, 15 Feb 2008, Alexander Graf wrote:

> On Feb 15, 2008, at 3:56 PM, Anthony Liguori wrote:
> 
> >Have you tried SLES-9 or openSUSE variants of the same age?  The ss issue in
> >gfxboot is only something recently introduced.  Prior to that, gfxboot used
> >big real mode so your patch wouldn't be sufficient for those versions of
> >gfxboot.
> 
> SLES7 - SLES-9  and SUSE 9.1 through to openSUSE 10.1 do not need the patch.
> They work 'as is'. SLES10 starts in text mode.
> Starting with 10.2 the mov ss issue came along, but maybe Steffen can tell us
> more about the history of this issue.

The use of memory > 1MB was optional in older versions, so they might work
even if the pm switch doesn't work. sles10 has a special check so it doesn't
run in xen; maybe that gets in the way here, too. After sles10 big segments
in real mode are no longer used.


Steffen

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Avi Kivity]

Alexander Graf wrote:
>>
>> While enabling gfxboot over vmx is very desirable, I'd like to avoid
>> guest-specific hacks.  IMO the correct fix is to set a  
>> "non_vt_friendly"
>> flag when switching from real mode to protected mode, then continue
>> emulation, re-computing the flag after every instruction.  After a few
>> instruction, the condition disappears and we can enter guest mode  
>> again.
>>     
>
> So when would the flag disappear?

Whenever the register state becomes consistent with VT again.  
vmx_set_segment() looks like the right point for turning it off.

>  Basically an SS read can happen any  
> time after the switch, even after 100 instructions if it has to.

That's okay.  Most (all) code will reload all segments shortly after the 
protected mode switch.

I am concerned about switches to real-mode not reloading fs and gs, so 
we'd stay in vt-unfriendly state and emulate even though the real mode 
code doesn't use fs and gs at all.  I believe we can live with it; newer 
Xen for example emulates 100% of real mode code.

>  While  
> this should fix more problems, the one thing I am concerned about is  
> that I have not encountered any other code that does have this problem.
>
>   

I think some Ubuntus use big real mode, which can use the same fix.

>> The downside is that we have to implement more instructions in the
>> emulator for this, but these instructions will be generally useful,  
>> not
>> just for gfxboot.
>>     
>
> I am not trying to talk you into anything - I would very much prefer a  
> rather clean solution as well. Nevertheless I do not see full  
> protected mode emulation code coming in the very near future and on a  
> user perspective would prefer to have something that works, even if  
> it's ugly.
> So while KVM is able to run most (if not all?) current major Operating  
> Systems unmodified, it fails to install them (at least on the Linux  
> side).
>   

I'd like to keep ugliness out of the kernel side.

I don't think there's much work to get protected mode emulation 
working.  There aren't that many instructions before we get to a 
vt-friendly state (a couple dozen?) and some of them are already 
implemented.

An alternative is to work around it in userspace.  If we recognise the 
exit reason, we can read the instructions around rip and attempt to fix 
things up.

> Even though I would greatly appreciate any effort made to get things  
> cleaned up, the gfxboot issue has been standing for months now without  
> even a hacky workaround (except for disabling gfxboot in all) or any  
> visible progress, so I believe a hack like this is at least worth  
> something to distributions that want to enable their users to work  
> with KVM.

On the other hand, merging the hacks discourages the right fix from 
being developed.  I do agree that the current situation is disgraceful.

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Alexander Graf]

On Feb 16, 2008, at 10:06 AM, Avi Kivity wrote:

> Alexander Graf wrote:
>>>
>>> While enabling gfxboot over vmx is very desirable, I'd like to avoid
>>> guest-specific hacks.  IMO the correct fix is to set a   
>>> "non_vt_friendly"
>>> flag when switching from real mode to protected mode, then continue
>>> emulation, re-computing the flag after every instruction.  After a  
>>> few
>>> instruction, the condition disappears and we can enter guest mode   
>>> again.
>>>
>>
>> So when would the flag disappear?
>
> Whenever the register state becomes consistent with VT again.   
> vmx_set_segment() looks like the right point for turning it off.

Sounds good. As basically the only problem we have are the sanity  
checks done on VMENTER, this should work.

>> Basically an SS read can happen any  time after the switch, even  
>> after 100 instructions if it has to.
>
> That's okay.  Most (all) code will reload all segments shortly after  
> the protected mode switch.
>
> I am concerned about switches to real-mode not reloading fs and gs,  
> so we'd stay in vt-unfriendly state and emulate even though the real  
> mode code doesn't use fs and gs at all.  I believe we can live with  
> it; newer Xen for example emulates 100% of real mode code.

Emulating all of the real mode shouldn't be too much of a problem on  
the performance side. I wouldn't be surprised if the vmenter/exits  
take about as much time as the emulation overhead.

>> While  this should fix more problems, the one thing I am concerned  
>> about is  that I have not encountered any other code that does have  
>> this problem.
>
> I think some Ubuntus use big real mode, which can use the same fix.

Do you have any file / pointer to where I could get one? I did try the  
feisty server iso which worked just fine.

>>> The downside is that we have to implement more instructions in the
>>> emulator for this, but these instructions will be generally  
>>> useful,  not
>>> just for gfxboot.
>>
>> I am not trying to talk you into anything - I would very much  
>> prefer a  rather clean solution as well. Nevertheless I do not see  
>> full  protected mode emulation code coming in the very near future  
>> and on a  user perspective would prefer to have something that  
>> works, even if  it's ugly.
>> So while KVM is able to run most (if not all?) current major  
>> Operating  Systems unmodified, it fails to install them (at least  
>> on the Linux  side).
>
> I'd like to keep ugliness out of the kernel side.
>
> I don't think there's much work to get protected mode emulation  
> working.  There aren't that many instructions before we get to a vt- 
> friendly state (a couple dozen?) and some of them are already  
> implemented.

The hardest one being ljmp. You need to do the whole pm transition in  
the emulator then. I believe there is a reason this hasn't been done  
yet?

> An alternative is to work around it in userspace.  If we recognise  
> the exit reason, we can read the instructions around rip and attempt  
> to fix things up.

So just get the CR0 write and UD exception as event to the userspace?  
I'd really love that approach. The "invalid opcode" hack, as I  
implemented it, is actually quite extensible. You could simply put the  
rip and an operation that is supposed to occur in a list and emulate  
whatever comes when the UD occurs. This might be the easiest way to  
fix things.

We could also have something more extensible, say a "generic binary  
patching" framework, so we know that if memory page 0x1234000 contains  
specific content, just patch it and apply a "what happens in case of  
invalid opcodes" script. This could all be in userspace and should  
enable us to circumvent most problems in a generic way.

>> Even though I would greatly appreciate any effort made to get  
>> things  cleaned up, the gfxboot issue has been standing for months  
>> now without  even a hacky workaround (except for disabling gfxboot  
>> in all) or any  visible progress, so I believe a hack like this is  
>> at least worth  something to distributions that want to enable  
>> their users to work  with KVM.
>
> On the other hand, merging the hacks discourages the right fix from  
> being developed.  I do agree that the current situation is  
> disgraceful.

Don't get me wrong on this - I really want to see something "right". I  
just don't see anyone working on it, as there are a lot of places KVM  
improves right now, which are a lot more important than real mode  
fixes. Usually real mode is completely unused as soon as you're done  
with bootstrapping, so why care about it that much?

I'm also perfectly fine with this not being merged. I built this hack  
for me, because I was rather unhappy with the situation as is and  
wanted to see gfxboot working, as I couldn't just "plug in" a current  
iso and install from that. If anyone benefits from it, I'm fine with  
it. If not, that's ok with me too. I just couldn't stand the situation  
that no fix was available at all (disabling gfxboot was no fix for  me).

Alex

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Avi Kivity]

Alexander Graf wrote:
>
> Emulating all of the real mode shouldn't be too much of a problem on 
> the performance side. I wouldn't be surprised if the vmenter/exits 
> take about as much time as the emulation overhead.
>

For the bootstrap process emulation is good enough, since the process is 
short enough. Good real mode performance may be necessary for 
applications like ghost which are still used for deployment.

In addition, if a real mode app uses x87/mmx/sse, then you are forced to 
add emulation for these instructions if you don't use v8086. Of course, 
if that app also uses big real mode, you are forced to do so anyway,

>>> While this should fix more problems, the one thing I am concerned 
>>> about is that I have not encountered any other code that does have 
>>> this problem.
>>
>> I think some Ubuntus use big real mode, which can use the same fix.
>
> Do you have any file / pointer to where I could get one? I did try the 
> feisty server iso which worked just fine.

No, sorry. This is just from memory.


>> I don't think there's much work to get protected mode emulation 
>> working. There aren't that many instructions before we get to a 
>> vt-friendly state (a couple dozen?) and some of them are already 
>> implemented.
>
> The hardest one being ljmp. You need to do the whole pm transition in 
> the emulator then. I believe there is a reason this hasn't been done yet?
>

No reason other than it's extremely icky. Looks like Xen has implemented 
it though.

>> An alternative is to work around it in userspace. If we recognise the 
>> exit reason, we can read the instructions around rip and attempt to 
>> fix things up.
>
> So just get the CR0 write and UD exception as event to the userspace? 
> I'd really love that approach. The "invalid opcode" hack, as I 
> implemented it, is actually quite extensible. You could simply put the 
> rip and an operation that is supposed to occur in a list and emulate 
> whatever comes when the UD occurs. This might be the easiest way to 
> fix things.

I meant trapping the vmentry failure (this is propagated to userspace 
anyway), recognizing gfxboot, fixing up the state, and continuing.

>
> We could also have something more extensible, say a "generic binary 
> patching" framework, so we know that if memory page 0x1234000 contains 
> specific content, just patch it and apply a "what happens in case of 
> invalid opcodes" script. This could all be in userspace and should 
> enable us to circumvent most problems in a generic way.
>

I can't say I'm thrilled about it when a real fix is possible.

>
> Don't get me wrong on this - I really want to see something "right". I 
> just don't see anyone working on it, as there are a lot of places KVM 
> improves right now, which are a lot more important than real mode 
> fixes. Usually real mode is completely unused as soon as you're done 
> with bootstrapping, so why care about it that much?

As I mentioned earlier, deployment apps like ghost.

>
> I'm also perfectly fine with this not being merged. I built this hack 
> for me, because I was rather unhappy with the situation as is and 
> wanted to see gfxboot working, as I couldn't just "plug in" a current 
> iso and install from that. If anyone benefits from it, I'm fine with 
> it. If not, that's ok with me too. I just couldn't stand the situation 
> that no fix was available at all (disabling gfxboot was no fix for me).

Well, I too would really like to see a fix for gfxboot merged, but I 
want to avoid patching whenever possible.

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Guillaume Thouvenin]

On Sat, 16 Feb 2008 14:34:09 +0100
Alexander Graf <alex@csgraf.de> wrote:

> > Whenever the register state becomes consistent with VT again.   
> > vmx_set_segment() looks like the right point for turning it off.
> 
> Sounds good. As basically the only problem we have are the sanity  
> checks done on VMENTER, this should work.

Hello,

 I tried to detect VMentry failure in order to do real mode emulation.
I tested the patch (pasted at the end of this message) with the
installation of OpenSuse 10.3 but it failed to detect the VMentry
failure. I suspect that on my computer the failure is not due to a
VMentry failure. Can you test the patch and tell me if it detects a
VMentry failure?

Thanks for your help,
Regards,

Guillaume

---
Index: kvm/arch/x86/kvm/vmx.c
===================================================================
--- kvm.orig/arch/x86/kvm/vmx.c	2008-02-18 09:22:53.000000000 +0100
+++ kvm/arch/x86/kvm/vmx.c	2008-02-18 09:43:13.000000000 +0100
@@ -2255,6 +2255,15 @@
 static const int kvm_vmx_max_exit_handlers =
 	ARRAY_SIZE(kvm_vmx_exit_handlers);
 
+static int kvm_handle_vmentry_failure(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
+{
+	printk(KERN_WARNING "VMENTRY failure detected \n");
+	if (vcpu->arch.rmode.active)
+		printk(KERN_WARNING "Big Real Mode emulation needed \n");
+
+	return 0;
+}
+
 /*
  * The guest has exited.  See if we can fix it or if we need userspace
  * assistance.
@@ -2265,6 +2274,9 @@
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 	u32 vectoring_info = vmx->idt_vectoring_info;
 
+	if (unlikely(exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY))
+		return kvm_handle_vmentry_failure(kvm_run, vcpu);
+
 	if (unlikely(vmx->fail)) {
 		kvm_run->exit_reason = KVM_EXIT_FAIL_ENTRY;
 		kvm_run->fail_entry.hardware_entry_failure_reason

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Alexander Graf]

On Feb 18, 2008, at 10:17 AM, Guillaume Thouvenin wrote:

> On Sat, 16 Feb 2008 14:34:09 +0100
> Alexander Graf <alex@csgraf.de> wrote:
>
>>> Whenever the register state becomes consistent with VT again.
>>> vmx_set_segment() looks like the right point for turning it off.
>>
>> Sounds good. As basically the only problem we have are the sanity
>> checks done on VMENTER, this should work.
>
> Hello,
>
> I tried to detect VMentry failure in order to do real mode emulation.
> I tested the patch (pasted at the end of this message) with the
> installation of OpenSuse 10.3 but it failed to detect the VMentry
> failure. I suspect that on my computer the failure is not due to a
> VMentry failure. Can you test the patch and tell me if it detects a
> VMentry failure?

You won't receive any VMentry failure as is. I'll try to explain the  
real problem to you.

This is the real-mode to protected-mode switch. I will comment the  
assembler commands if important for the issue.

Real mode segments are mere offsets. They get lshifted by 4 (as far as  
I remember) and just added to the offset. SS is on some value here  
(can be 0x0, can be 0x5380, can be whatever)

[...]
0x0000000000046e26:  cli
0x0000000000046e27:  movw   %ss,%cs:2256
0x0000000000046e2c:  movw   %ds,%cs:2260
0x0000000000046e31:  movw   %es,%cs:2262
0x0000000000046e36:  movw   %fs,%cs:2264
0x0000000000046e3b:  movw   %gs,%cs:2266
0x0000000000046e40:  or     $0x1,%al

enable protected mode flag in eax

0x0000000000046e42:  lgdtl  %cs:2250
0x0000000000046e49:  lidtl  %cs:2238
0x0000000000046e50:  mov    %eax,%cr0

enable protected mode in cr0 by writing the pm flag
-- we have a vmexit here --

0x0000000000046e53:  ljmp   $0x18,$0x6e18

ljmp to the protected mode segment 0x18 (CS), so we really are in  
protected mode

0x0000000000046e58:  mov    $0x20,%ax
0x0000000000046e5c:  mov    %eax,%ds
0x0000000000046e5e:  mov    %ss,%eax

read SS
and calculate the segment based on the address

0x0000000000046e60:  and    $0xffff,%esp
0x0000000000046e66:  shl    $0x4,%eax
0x0000000000046e69:  add    %eax,%esp
0x0000000000046e6b:  mov    $0x8,%ax
0x0000000000046e6f:  mov    %eax,%ss
0x0000000000046e71:  mov    %eax,%es
0x0000000000046e73:  mov    %eax,%fs
0x0000000000046e75:  mov    %eax,%gs

set the segments to the selector we just calculated

[...]

So what could be the problem with this code? It looks slightly hacky,  
but works on most x86 computers (some VIAs choke here too).

After the vmexit on writing cr0, we are in a somewhat undefined state.  
For most of it, we are in protected mode by now. But the segments have  
not been set to something meaningful in protected mode, so they still  
are on their old values.

Unfortunately VMX does a lot of sanity checks, if the virtual machine  
is actually valid and can be entered into, when trying to do so. One  
of these checks is 'CS.RPL == SS.RPL'. The RPL are lowest 2 bits (I am  
not too sure about this, read in the manuals if you really need to  
know) of the selector. We don't have a real selector in this  
transitioning phase though, but instead have real mode offsets in  
them. This means, if CS was on 0x0 and SS on 0x1235, there is no way  
the RPL equals condition can be met.

This is why in the vmx.c you can find in enter_pmode() code like this:

         vmcs_write16(GUEST_SS_SELECTOR, 0);
         vmcs_write32(GUEST_SS_AR_BYTES, 0x93);

This sets SS to 0 whenever the above mov to cr0 occurs. As most code  
does not rely on SS being available after a PM transition, but instead  
simply writes a sane value to it, that's fine for 99.9% of all code  
out there. Unfortunately it is not for gfxboot.

So if you want to see a VMentry failure, just remove the SS patching  
and you'll see one. My guess would be that you see a lot of problems  
with otherwise working code too then, though, as SS can be anything in  
that state.

I hope this helped,

Alex

>
>
> Thanks for your help,
> Regards,
>
> Guillaume
>
> ---
> Index: kvm/arch/x86/kvm/vmx.c
> ===================================================================
> --- kvm.orig/arch/x86/kvm/vmx.c	2008-02-18 09:22:53.000000000 +0100
> +++ kvm/arch/x86/kvm/vmx.c	2008-02-18 09:43:13.000000000 +0100
> @@ -2255,6 +2255,15 @@
> static const int kvm_vmx_max_exit_handlers =
> 	ARRAY_SIZE(kvm_vmx_exit_handlers);
>
> +static int kvm_handle_vmentry_failure(struct kvm_run *kvm_run,  
> struct kvm_vcpu *vcpu)
> +{
> +	printk(KERN_WARNING "VMENTRY failure detected \n");
> +	if (vcpu->arch.rmode.active)
> +		printk(KERN_WARNING "Big Real Mode emulation needed \n");
> +
> +	return 0;
> +}
> +
> /*
>  * The guest has exited.  See if we can fix it or if we need userspace
>  * assistance.
> @@ -2265,6 +2274,9 @@
> 	struct vcpu_vmx *vmx = to_vmx(vcpu);
> 	u32 vectoring_info = vmx->idt_vectoring_info;
>
> +	if (unlikely(exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY))
> +		return kvm_handle_vmentry_failure(kvm_run, vcpu);
> +
> 	if (unlikely(vmx->fail)) {
> 		kvm_run->exit_reason = KVM_EXIT_FAIL_ENTRY;
> 		kvm_run->fail_entry.hardware_entry_failure_reason
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> kvm-devel mailing list
> kvm-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/kvm-devel


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Alexander Graf]

On Feb 18, 2008, at 10:34 AM, Alexander Graf wrote:

>
> On Feb 18, 2008, at 10:17 AM, Guillaume Thouvenin wrote:
>
>> On Sat, 16 Feb 2008 14:34:09 +0100
>> Alexander Graf <alex@csgraf.de> wrote:
>>
>>>> Whenever the register state becomes consistent with VT again.
>>>> vmx_set_segment() looks like the right point for turning it off.
>>>
>>> Sounds good. As basically the only problem we have are the sanity
>>> checks done on VMENTER, this should work.
>>
>> Hello,
>>
>> I tried to detect VMentry failure in order to do real mode emulation.
>> I tested the patch (pasted at the end of this message) with the
>> installation of OpenSuse 10.3 but it failed to detect the VMentry
>> failure. I suspect that on my computer the failure is not due to a
>> VMentry failure. Can you test the patch and tell me if it detects a
>> VMentry failure?
>
> You won't receive any VMentry failure as is. I'll try to explain the
> real problem to you.
>
> This is the real-mode to protected-mode switch. I will comment the
> assembler commands if important for the issue.
>
> Real mode segments are mere offsets. They get lshifted by 4 (as far as
> I remember) and just added to the offset. SS is on some value here
> (can be 0x0, can be 0x5380, can be whatever)
>
> [...]
> 0x0000000000046e26:  cli
> 0x0000000000046e27:  movw   %ss,%cs:2256
> 0x0000000000046e2c:  movw   %ds,%cs:2260
> 0x0000000000046e31:  movw   %es,%cs:2262
> 0x0000000000046e36:  movw   %fs,%cs:2264
> 0x0000000000046e3b:  movw   %gs,%cs:2266
> 0x0000000000046e40:  or     $0x1,%al
>
> enable protected mode flag in eax
>
> 0x0000000000046e42:  lgdtl  %cs:2250
> 0x0000000000046e49:  lidtl  %cs:2238
> 0x0000000000046e50:  mov    %eax,%cr0
>
> enable protected mode in cr0 by writing the pm flag
> -- we have a vmexit here --
>
> 0x0000000000046e53:  ljmp   $0x18,$0x6e18
>
> ljmp to the protected mode segment 0x18 (CS), so we really are in
> protected mode
>
> 0x0000000000046e58:  mov    $0x20,%ax
> 0x0000000000046e5c:  mov    %eax,%ds
> 0x0000000000046e5e:  mov    %ss,%eax
>
> read SS
> and calculate the segment based on the address

Actually esp gets calculated, based on the old SS address + SP

>
>
> 0x0000000000046e60:  and    $0xffff,%esp
> 0x0000000000046e66:  shl    $0x4,%eax
> 0x0000000000046e69:  add    %eax,%esp
> 0x0000000000046e6b:  mov    $0x8,%ax
> 0x0000000000046e6f:  mov    %eax,%ss
> 0x0000000000046e71:  mov    %eax,%es
> 0x0000000000046e73:  mov    %eax,%fs
> 0x0000000000046e75:  mov    %eax,%gs
>
> set the segments to the selector we just calculated

and the other segments get set to 0x8

>
>
> [...]
>
> So what could be the problem with this code? It looks slightly hacky,
> but works on most x86 computers (some VIAs choke here too).
>
> After the vmexit on writing cr0, we are in a somewhat undefined state.
> For most of it, we are in protected mode by now. But the segments have
> not been set to something meaningful in protected mode, so they still
> are on their old values.
>
> Unfortunately VMX does a lot of sanity checks, if the virtual machine
> is actually valid and can be entered into, when trying to do so. One
> of these checks is 'CS.RPL == SS.RPL'. The RPL are lowest 2 bits (I am
> not too sure about this, read in the manuals if you really need to
> know) of the selector. We don't have a real selector in this
> transitioning phase though, but instead have real mode offsets in
> them. This means, if CS was on 0x0 and SS on 0x1235, there is no way
> the RPL equals condition can be met.
>
> This is why in the vmx.c you can find in enter_pmode() code like this:
>
>         vmcs_write16(GUEST_SS_SELECTOR, 0);
>         vmcs_write32(GUEST_SS_AR_BYTES, 0x93);
>
> This sets SS to 0 whenever the above mov to cr0 occurs. As most code
> does not rely on SS being available after a PM transition, but instead
> simply writes a sane value to it, that's fine for 99.9% of all code
> out there. Unfortunately it is not for gfxboot.
>
> So if you want to see a VMentry failure, just remove the SS patching
> and you'll see one. My guess would be that you see a lot of problems
> with otherwise working code too then, though, as SS can be anything in
> that state.
>
> I hope this helped,
>
> Alex
>
>
>>
>>
>> Thanks for your help,
>> Regards,
>>
>> Guillaume
>>
>> ---
>> Index: kvm/arch/x86/kvm/vmx.c
>> ===================================================================
>> --- kvm.orig/arch/x86/kvm/vmx.c	2008-02-18 09:22:53.000000000 +0100
>> +++ kvm/arch/x86/kvm/vmx.c	2008-02-18 09:43:13.000000000 +0100
>> @@ -2255,6 +2255,15 @@
>> static const int kvm_vmx_max_exit_handlers =
>> 	ARRAY_SIZE(kvm_vmx_exit_handlers);
>>
>> +static int kvm_handle_vmentry_failure(struct kvm_run *kvm_run,
>> struct kvm_vcpu *vcpu)
>> +{
>> +	printk(KERN_WARNING "VMENTRY failure detected \n");
>> +	if (vcpu->arch.rmode.active)
>> +		printk(KERN_WARNING "Big Real Mode emulation needed \n");
>> +
>> +	return 0;
>> +}
>> +
>> /*
>> * The guest has exited.  See if we can fix it or if we need userspace
>> * assistance.
>> @@ -2265,6 +2274,9 @@
>> 	struct vcpu_vmx *vmx = to_vmx(vcpu);
>> 	u32 vectoring_info = vmx->idt_vectoring_info;
>>
>> +	if (unlikely(exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY))
>> +		return kvm_handle_vmentry_failure(kvm_run, vcpu);
>> +
>> 	if (unlikely(vmx->fail)) {
>> 		kvm_run->exit_reason = KVM_EXIT_FAIL_ENTRY;
>> 		kvm_run->fail_entry.hardware_entry_failure_reason
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> kvm-devel mailing list
>> kvm-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> kvm-devel mailing list
> kvm-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/kvm-devel


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Andi Kleen]

Alexander Graf <alex@csgraf.de> writes:
>
> As everything except gfxboot works, we can simply change gfxboot in
> runtime to use a different value. Unfortunately the mov instruction,
> used to read the SS register is only 2 bytes long, so there is no way to
> binary patch the mov to something that would contain an address. So the
> only way I could think of was an invalid instruction. The UD exception
> is intercepted in KVM and is already emulated for VMCALLs. This can be
> extended to an opcode, that is officially unused (0f 0c) and have the

It quite is dangerous to use unused opcodes like this. Intel/AMD tend
to regularly add new instructions with new CPUs, so currently unused
opcodes can quick turn into used ones which don't trap anymore or trap
differently or do something unexpected. And yes this has broken
software in the past.

If you want to continue to use this method (it is unclear given the
rest of the thread), I would recommend you keep track of the patched
locations instead and use some documented to be trapping 1 or 2 byte opcode
and then check in the #UD handler if you hit one of the previously
patched locations.

Given that solution would have some corner cases (e.g. the table
would likely need to have a fixed upper limit), but at least for the
gfxboot case just using a very small limited table would be probably
enough.

-Andi


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Alexander Graf]

On Feb 18, 2008, at 12:46 PM, Andi Kleen wrote:

> Alexander Graf <alex@csgraf.de> writes:
>>
>> As everything except gfxboot works, we can simply change gfxboot in
>> runtime to use a different value. Unfortunately the mov instruction,
>> used to read the SS register is only 2 bytes long, so there is no  
>> way to
>> binary patch the mov to something that would contain an address. So  
>> the
>> only way I could think of was an invalid instruction. The UD  
>> exception
>> is intercepted in KVM and is already emulated for VMCALLs. This can  
>> be
>> extended to an opcode, that is officially unused (0f 0c) and have the
>
> It quite is dangerous to use unused opcodes like this. Intel/AMD tend
> to regularly add new instructions with new CPUs, so currently unused
> opcodes can quick turn into used ones which don't trap anymore or trap
> differently or do something unexpected. And yes this has broken
> software in the past.
>
> If you want to continue to use this method (it is unclear given the
> rest of the thread), I would recommend you keep track of the patched

I don't think Avi wants anything as hacky in the kernel.

>
> locations instead and use some documented to be trapping 1 or 2 byte  
> opcode
> and then check in the #UD handler if you hit one of the previously
> patched locations.

This is quite a nice idea I thought of as well (that's what the  
'generic' binary patching approach would look like). Nevertheless it'd  
be a lot better to have something that works for all cases without  
patching binaries, as we might get into TPM problems with that either  
way.

>
>
> Given that solution would have some corner cases (e.g. the table
> would likely need to have a fixed upper limit), but at least for the
> gfxboot case just using a very small limited table would be probably
> enough.

For the gfxboot case a simple variable would be enough. There is only  
one position where this UD would occur, so it's only one IP to save.
Nevertheless please think of this as a hack, not a solution. I only  
wanted to have something for the meantime while no real fix is  
available. As I don't have the time right now to actually write a  
proper one, this was the only way to have something that 'works' at  
all right now. I strongly hope we will have a working solution until  
the point where CPUs show up, which will have this opcode used.

Alex

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Anthony Liguori]

Alexander Graf wrote:
> Real mode segments are mere offsets. They get lshifted by 4 (as far as  
> I remember) and just added to the offset. SS is on some value here  
>   

Yes, this is correct.

> 0x0000000000046e60:  and    $0xffff,%esp
> 0x0000000000046e66:  shl    $0x4,%eax
> 0x0000000000046e69:  add    %eax,%esp
> 0x0000000000046e6b:  mov    $0x8,%ax
> 0x0000000000046e6f:  mov    %eax,%ss
> 0x0000000000046e71:  mov    %eax,%es
> 0x0000000000046e73:  mov    %eax,%fs
> 0x0000000000046e75:  mov    %eax,%gs
>
> set the segments to the selector we just calculated
>
> [...]
>
> So what could be the problem with this code? It looks slightly hacky,  
> but works on most x86 computers (some VIAs choke here too).
>
> After the vmexit on writing cr0, we are in a somewhat undefined state.  
> For most of it, we are in protected mode by now. But the segments have  
> not been set to something meaningful in protected mode, so they still  
> are on their old values.
>
> Unfortunately VMX does a lot of sanity checks, if the virtual machine  
> is actually valid and can be entered into, when trying to do so. One  
> of these checks is 'CS.RPL == SS.RPL'. The RPL are lowest 2 bits (I am  
> not too sure about this, read in the manuals if you really need to  
> know) of the selector. We don't have a real selector in this  
> transitioning phase though, but instead have real mode offsets in  
> them. This means, if CS was on 0x0 and SS on 0x1235, there is no way  
> the RPL equals condition can be met.
>   

Keep in mind, this is not big real mode.  The gfxboot code here was 
actually rewritten to not use big real mode so that it worked with Xen.  
It works with Xen because Xen uses vm86 mode within the host whereas KVM 
uses vm86 mode in the guest's VT context.  An alternative approach to 
fixing this problem (but in a less hackish way) would be to simply use 
the host's vm86 mode instead of vm86 mode within the guest's VT context.

However, this does not fix big real mode.  For that, you need to do 
proper 16-bit instruction emulation.

Regards,

Anthony Liguori

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH] enable gfxboot on VMX

[Avi Kivity]

Anthony Liguori wrote:
>
> Keep in mind, this is not big real mode.  The gfxboot code here was 
> actually rewritten to not use big real mode so that it worked with 
> Xen.  It works with Xen because Xen uses vm86 mode within the host 
> whereas KVM uses vm86 mode in the guest's VT context.

I don't think that's right.  vm86 mode doesn't work under long mode, so 
I don't see how Xen can use it in host context.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

catch vmentry failure (was enable gfxboot on VMX)

[Guillaume Thouvenin]

On Mon, 18 Feb 2008 10:39:31 +0100
Alexander Graf <alex@csgraf.de> wrote:


> > So if you want to see a VMentry failure, just remove the SS patching
> > and you'll see one. My guess would be that you see a lot of problems
> > with otherwise working code too then, though, as SS can be anything in
> > that state.

So I made some tests and you were right, removing the SS patching
showed VM entry failure but it also generated lots of problems. Thus I
tried to modify a little bit the code and with the following patch (see
the end of the email) I can detect VM Entry failures without generating
other problems. It works when you use a distribution that is
"big-real-mode free". I pasted the patch just to show the idea. 

It's interesting because we can continue to use the virtual mode for the
majority of distribution and we can detect when a VM entry failure is
detected it means that we need to switch from virtual mode to full real
mode emulation. Such failure is caught in handle_vmentry_failure() when
patch applied. If it's doable, the next step is the modification of the
SS segment selector to succeed the vm-entry and the switch from virtual
mode to a real mode emulation that could be done in
handle_vmentry_failure(). Does it make sense?

Regards,
Guillaume

---

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 46e0e58..c2c3897 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -1166,15 +1166,13 @@ static void enter_pmode(struct kvm_vcpu *vcpu)
 			(vmcs_readl(CR4_READ_SHADOW) & X86_CR4_VME));
 
 	update_exception_bitmap(vcpu);
-
+ 
+	fix_pmode_dataseg(VCPU_SREG_SS, &vcpu->arch.rmode.ss);
 	fix_pmode_dataseg(VCPU_SREG_ES, &vcpu->arch.rmode.es);
 	fix_pmode_dataseg(VCPU_SREG_DS, &vcpu->arch.rmode.ds);
 	fix_pmode_dataseg(VCPU_SREG_GS, &vcpu->arch.rmode.gs);
 	fix_pmode_dataseg(VCPU_SREG_FS, &vcpu->arch.rmode.fs);
 
-	vmcs_write16(GUEST_SS_SELECTOR, 0);
-	vmcs_write32(GUEST_SS_AR_BYTES, 0x93);
-
 	vmcs_write16(GUEST_CS_SELECTOR,
 		     vmcs_read16(GUEST_CS_SELECTOR) & ~SELECTOR_RPL_MASK);
 	vmcs_write32(GUEST_CS_AR_BYTES, 0x9b);
@@ -1228,20 +1226,12 @@ static void enter_rmode(struct kvm_vcpu *vcpu)
 	vmcs_writel(GUEST_CR4, vmcs_readl(GUEST_CR4) | X86_CR4_VME);
 	update_exception_bitmap(vcpu);
 
-	vmcs_write16(GUEST_SS_SELECTOR, vmcs_readl(GUEST_SS_BASE) >> 4);
-	vmcs_write32(GUEST_SS_LIMIT, 0xffff);
-	vmcs_write32(GUEST_SS_AR_BYTES, 0xf3);
-
-	vmcs_write32(GUEST_CS_AR_BYTES, 0xf3);
-	vmcs_write32(GUEST_CS_LIMIT, 0xffff);
-	if (vmcs_readl(GUEST_CS_BASE) == 0xffff0000)
-		vmcs_writel(GUEST_CS_BASE, 0xf0000);
-	vmcs_write16(GUEST_CS_SELECTOR, vmcs_readl(GUEST_CS_BASE) >> 4);
-
+	fix_rmode_seg(VCPU_SREG_CS, &vcpu->arch.rmode.cs);
 	fix_rmode_seg(VCPU_SREG_ES, &vcpu->arch.rmode.es);
 	fix_rmode_seg(VCPU_SREG_DS, &vcpu->arch.rmode.ds);
 	fix_rmode_seg(VCPU_SREG_GS, &vcpu->arch.rmode.gs);
 	fix_rmode_seg(VCPU_SREG_FS, &vcpu->arch.rmode.fs);
+	fix_rmode_seg(VCPU_SREG_SS, &vcpu->arch.rmode.ss);
 
 	kvm_mmu_reset_context(vcpu);
 	init_rmode_tss(vcpu->kvm);
@@ -2257,6 +2247,39 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu,
 static const int kvm_vmx_max_exit_handlers =
 	ARRAY_SIZE(kvm_vmx_exit_handlers);
 
+static int handle_vmentry_failure(u32 exit_reason, struct kvm_vcpu *vcpu)
+{
+	unsigned long exit_qualification = vmcs_read64(EXIT_QUALIFICATION);
+	u32 info_field = vmcs_read32(VMX_INSTRUCTION_INFO);
+	unsigned int basic_exit_reason = (uint16_t) exit_reason;
+
+	printk("%s: exit reason 0x%x  \n", __FUNCTION__, exit_reason);
+	printk("%s: vmentry failure reason %u  \n", __FUNCTION__, basic_exit_reason);
+	printk("%s: VMX-instruction Information field 0x%x  \n", __FUNCTION__, info_field);
+
+	switch (basic_exit_reason) {
+		case EXIT_REASON_INVALID_GUEST_STATE:
+			printk("caused by invalid guest state (%ld).\n", exit_qualification);
+			/* At this point we need to modify SS selector to pass vmentry test.
+			 * This modification prevent the usage of virtual mode to emulate real 
+			 * mode so we need to pass in big real mode emulation
+			 * with somehting like:
+			 * vcpu->arch.rmode.emulate = 1
+			 */
+			break;
+		case EXIT_REASON_MSR_LOADING:
+			printk("caused by MSR entry %ld loading.\n", exit_qualification);
+			break;
+		case EXIT_REASON_MACHINE_CHECK:
+			printk("caused by machine check.\n");
+			break;
+		default:
+			printk("reason not known yet!\n");
+			break;
+	}
+	return 0;
+}
+
 /*
  * The guest has exited.  See if we can fix it or if we need userspace
  * assistance.
@@ -2274,6 +2297,9 @@ static int kvm_handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
 		return 0;
 	}
 
+	if (unlikely(exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY)) 
+		return handle_vmentry_failure(exit_reason, vcpu);
+
 	if ((vectoring_info & VECTORING_INFO_VALID_MASK) &&
 				exit_reason != EXIT_REASON_EXCEPTION_NMI)
 		printk(KERN_WARNING "%s: unexpected, valid vectoring info and "
diff --git a/arch/x86/kvm/vmx.h b/arch/x86/kvm/vmx.h
index 5dff460..200c0f8 100644
--- a/arch/x86/kvm/vmx.h
+++ b/arch/x86/kvm/vmx.h
@@ -223,7 +223,10 @@ enum vmcs_field {
 #define EXIT_REASON_IO_INSTRUCTION      30
 #define EXIT_REASON_MSR_READ            31
 #define EXIT_REASON_MSR_WRITE           32
+#define EXIT_REASON_INVALID_GUEST_STATE 33
+#define EXIT_REASON_MSR_LOADING         34
 #define EXIT_REASON_MWAIT_INSTRUCTION   36
+#define EXIT_REASON_MACHINE_CHECK       41
 #define EXIT_REASON_TPR_BELOW_THRESHOLD 43
 #define EXIT_REASON_APIC_ACCESS         44
 #define EXIT_REASON_WBINVD		54
diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h
index 024b57c..b6e4006 100644
--- a/include/asm-x86/kvm_host.h
+++ b/include/asm-x86/kvm_host.h
@@ -261,7 +261,7 @@ struct kvm_vcpu_arch {
 			unsigned long base;
 			u32 limit;
 			u32 ar;
-		} tr, es, ds, fs, gs;
+		} tr, cs, es, ds, fs, gs, ss;
 	} rmode;
 	int halt_request; /* real mode on Intel only */
 
 

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: catch vmentry failure (was enable gfxboot on VMX)

[Avi Kivity]

Guillaume Thouvenin wrote:
> On Mon, 18 Feb 2008 10:39:31 +0100
> Alexander Graf <alex@csgraf.de> wrote:
>
>
>   
>>> So if you want to see a VMentry failure, just remove the SS patching
>>> and you'll see one. My guess would be that you see a lot of problems
>>> with otherwise working code too then, though, as SS can be anything in
>>> that state.
>>>       
>
> So I made some tests and you were right, removing the SS patching
> showed VM entry failure but it also generated lots of problems. Thus I
> tried to modify a little bit the code and with the following patch (see
> the end of the email) I can detect VM Entry failures without generating
> other problems. It works when you use a distribution that is
> "big-real-mode free". I pasted the patch just to show the idea. 
>
> It's interesting because we can continue to use the virtual mode for the
> majority of distribution and we can detect when a VM entry failure is
> detected it means that we need to switch from virtual mode to full real
> mode emulation. Such failure is caught in handle_vmentry_failure() when
> patch applied. If it's doable, the next step is the modification of the
> SS segment selector to succeed the vm-entry and the switch from virtual
> mode to a real mode emulation that could be done in
> handle_vmentry_failure(). Does it make sense?
>
>   

Yes.  An alternative (useful if a failed vmentry corrupts the guest 
state) is to check all register state when switching modes.

> -
> +	fix_rmode_seg(VCPU_SREG_CS, &vcpu->arch.rmode.cs);
>  	fix_rmode_seg(VCPU_SREG_ES, &vcpu->arch.rmode.es);
>  	fix_rmode_seg(VCPU_SREG_DS, &vcpu->arch.rmode.ds);
>  	fix_rmode_seg(VCPU_SREG_GS, &vcpu->arch.rmode.gs);
>  	fix_rmode_seg(VCPU_SREG_FS, &vcpu->arch.rmode.fs);
> +	fix_rmode_seg(VCPU_SREG_SS, &vcpu->arch.rmode.ss);
>   

Ideally you wouldn't call fix_rmode_seg() at all.  The guest will 
emulate until such time as the segments are valid for v8086, for example 
when the guest reloads them itself.

> +	switch (basic_exit_reason) {
> +		case EXIT_REASON_INVALID_GUEST_STATE:
> +			printk("caused by invalid guest state (%ld).\n", exit_qualification);
> +			/* At this point we need to modify SS selector to pass vmentry test.
> +			 * This modification prevent the usage of virtual mode to emulate real 
> +			 * mode so we need to pass in big real mode emulation
> +			 * with somehting like:
> +			 * vcpu->arch.rmode.emulate = 1
>   

Note you might need to emulate in protected mode as well, for a small 
part of the switch, for similar reasons.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/