From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: Doamin crash when trying to install disk encryption (PointSec) on Windows HVM Date: Wed, 22 Apr 2009 15:25:57 +0100 Message-ID: References: <8686c3cd0904220720t5d529509x85150382da85f670@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <8686c3cd0904220720t5d529509x85150382da85f670@mail.gmail.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Tom Rotenberg Cc: Tim Deegan , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Then it probably is a mis-emulation somewhere down the line. Unfortunately that could be hard to track down, even if we had PointSec software to hand, which we do not. -- Keir On 22/04/2009 15:20, "Tom Rotenberg" wrote: > Well, just tried this patch, and it doesn't seem to work. > I get the following error: >=20 > (XEN) HVM1: Booting from 0000:7c00 > (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest st= ate > (0). > (XEN) ************* VMCS Area ************** > (XEN) *** Guest State *** > (XEN) CR0: actual=3D0x0000000080010039, shadow=3D0x0000000080000019, > gh_mask=3Dffffffffffffffff > (XEN) CR4: actual=3D0x0000000000002060, shadow=3D0x0000000000000000, > gh_mask=3Dffffffffffffffff > (XEN) CR3: actual=3D0x000000000a22fa20, target_count=3D0 > (XEN)=A0=A0=A0=A0=A0 target0=3D0000000000000000, target1=3D0000000000000000 > (XEN)=A0=A0=A0=A0=A0 target2=3D0000000000000000, target3=3D0000000000000000 > (XEN) RSP =3D 0x0000000000000080 (0x0000000000000080)=A0 RIP =3D 0x000000000000= 002a > (0x000000000000002a) > (XEN) RFLAGS=3D0x0000000000023202 (0x0000000000023202)=A0 DR7 =3D 0x00000000000= 00400 > (XEN) Sysenter RSP=3D0000000000000000 CS:RIP=3D0000:0000000000000000 > (XEN) CS: sel=3D0x0060, attr=3D0x0c09b, limit=3D0xffffffff, base=3D0x000000000020= 0000 > (XEN) DS: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, base=3D0x000000000020= 0000 > (XEN) SS: sel=3D0x0070, attr=3D0x0c093, limit=3D0xfc000fff, base=3D0x000000000020= ba62 > (XEN) ES: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, base=3D0x000000000020= 0000 > (XEN) FS: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, base=3D0x000000000020= 0000 > (XEN) GS: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, base=3D0x000000000020= 0000 > (XEN) GDTR:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 limit=3D0x00001dd8, > base=3D0x0000000000200000 > (XEN) LDTR: sel=3D0x0000, attr=3D0x1c000, limit=3D0xffffffff, > base=3D0x0000000000000000 > (XEN) IDTR:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 limit=3D0x00000188, > base=3D0x0000000000201df0 > (XEN) TR: sel=3D0x0058, attr=3D0x0008b, limit=3D0x0000ffff, base=3D0x000000000020= 1ff2 > (XEN) Guest PAT =3D 0x0000000000000000 > (XEN) TSC Offset =3D ffffffe2242f086e > (XEN) DebugCtl=3D0000000000000000 DebugExceptions=3D0000000000000000 > (XEN) Interruptibility=3D0001 ActivityState=3D0000 > (XEN) *** Host State *** > (XEN) RSP =3D 0xffff83007e4f7fa0=A0 RIP =3D 0xffff828c8019aa20 > (XEN) CS=3De008 DS=3D0000 ES=3D0000 FS=3D0000 GS=3D0000 SS=3D0000 TR=3De040 > (XEN) FSBase=3D0000000000000000 GSBase=3D0000000000000000 TRBase=3Dffff828c802a= 8b00 > (XEN) GDTBase=3Dffff83007e9a3000 IDTBase=3Dffff83007e62e010 > (XEN) CR0=3D0000000080050033 CR3=3D000000007cfd8000 CR4=3D00000000000026f0 > (XEN) Sysenter RSP=3Dffff83007e4f7fd0 CS:RIP=3De008:ffff828c801c7290 > (XEN) Host PAT =3D 0x0000000000000000 > (XEN) *** Control State *** > (XEN) PinBased=3D0000003f CPUBased=3Db6a1e7fe SecondaryExec=3D00000041 > (XEN) EntryControls=3D000011ff ExitControls=3D0003efff > (XEN) ExceptionBitmap=3D00044080 > (XEN) VMEntry: intr_info=3D80000b0b errcode=3D00001eac ilen=3D00000000 > (XEN) VMExit: intr_info=3D00000000 errcode=3D00008000 ilen=3D00000000 > (XEN)=A0=A0=A0=A0=A0=A0=A0=A0 reason=3D80000021 qualification=3D00000000 > (XEN) IDTVectoring: info=3D00000000 errcode=3D00000000 > (XEN) TPR Threshold =3D 0x00 > (XEN) EPT pointer =3D 0x0000000000000000 > (XEN) Virtual processor ID =3D 0x0000 > (XEN) ************************************** > (XEN) domain_crash called from vmx.c:2218 > (XEN) Domain 1 (vcpu#0) crashed on cpu#1: > (XEN) ----[ Xen-3.4.0-rc3-pre=A0 x86_64=A0 debug=3Dn=A0 Not tainted ]---- > (XEN) CPU:=A0=A0=A0 1 > (XEN) RIP:=A0=A0=A0 0060:[<000000000000002a>] > (XEN) RFLAGS: 0000000000023202=A0=A0 CONTEXT: hvm guest > (XEN) rax: 0000000000000007=A0=A0 rbx: 0000000000001490=A0=A0 rcx: 00000000000000= 00 > (XEN) rdx: 0000000000001da8=A0=A0 rsi: 0000000000000000=A0=A0 rdi: 00000000000000= 00 > (XEN) rbp: 0000000000008ebf=A0=A0 rsp: 0000000000000080=A0=A0 r8:=A0 00000000000000= 00 > (XEN) r9:=A0 0000000000000000=A0=A0 r10: 0000000000000000=A0=A0 r11: 00000000000000= 00 > (XEN) r12: 0000000000000000=A0=A0 r13: 0000000000000000=A0=A0 r14: 00000000000000= 00 > (XEN) r15: 0000000000000000=A0=A0 cr0: 0000000080000019=A0=A0 cr4: 00000000000000= 00 > (XEN) cr3: 0000000001443000=A0=A0 cr2: 0000000000000000 > (XEN) ds: 0068=A0=A0 es: 0068=A0=A0 fs: 0068=A0=A0 gs: 0068=A0=A0 ss: 0070=A0=A0 cs: 0060 >=20 >=20 > Any ideas? >=20 > Tom >=20 >=20 > 2009/4/22 Keir Fraser >> That should do it. >>=20 >> =A0K. >>=20 >>=20 >> On 22/04/2009 15:04, "Tom Rotenberg" wrote: >>=20 >>> Keir, >>> Just to make sure, i am using the following patch, in order to disable = the >>> vm86 acceleration: >>>=20 >>> diff -r cdc044f665dc xen/arch/x86/hvm/vmx/vmx.c >>> --- a/xen/arch/x86/hvm/vmx/vmx.c=A0=A0=A0 Wed Apr 22 11:26:37 2009 +0100 >>> +++ b/xen/arch/x86/hvm/vmx/vmx.c=A0=A0=A0 Wed Apr 22 17:03:20 2009 +0300 >>> @@ -829,7 +829,7 @@ >>> =A0=A0=A0=A0=A0=A0=A0=A0 >>> =A0=A0=A0=A0=A0=A0=A0=A0 if ( seg =3D=3D x86_seg_tr ) >>> =A0=A0=A0=A0=A0=A0=A0=A0 { >>> -=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if ( v->domain->arch.hvm_domain.params[HVM_PARAM_VM86_TSS]= ) >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (0) >>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 { >>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sel =3D 0; >>> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 attr =3D vm86_tr_attr; >>>=20 >>> Is this OK? >>>=20 >>> Tom >>>=20 >>> 2009/4/22 Keir Fraser >>>> Yes, the safest way to be sure is probably to replace the if() stateme= nt in >>>> vmx_set_segment_register() that tests HVM_PARAM_VM86_TSS with if(0). T= hat >>>> is >>>> the only place in Xen that checks HVM_PARAM_VM86_TSS. Then you can >>>> re-build/install Xen and be sure that vm86 accel must be disabled. >>>>=20 >>>> =A0-- Keir >>>>=20 >>>> On 22/04/2009 14:52, "Tom Rotenberg" wrote: >>>>=20 >>>>> So, do u suggest, that i will set HVM_PARAM_VM86_TSS to 0, and re-che= ck >>>>> it? >>>>>=20 >>>>> 2009/4/22 Tim Deegan >>>>>> At 14:34 +0100 on 22 Apr (1240410866), Keir Fraser wrote: >>>>>>> It could be an issue with the vm86 acceleration, possibly. I'm pret= ty >>>>>>> sure >>>>>>> the guest would have to IRET from protected mode to enter vm86 mode >>>>>>> itself, >>>>>>> and we don't emulate that. Tim: what would we need to do to disable= the >>>>>>> vm86 >>>>>>> acceleration for testing purposes? You suggested not setting VM86_T= SS >>>>>>> param >>>>>>> from hvmloader, but I couldn't convince myself what effect that wou= ld >>>>>>> actually have as the logic in Xen is non-trivial. >>>>>>=20 >>>>>> Yes; if HVM_PARAM_VM86_TSS is zero, vmx_set_segment_register() will >>>>>> always set the tss bit in the bitmap of segments that aren't safe to >>>>>> enter VM86 with. >>>>>>=20 >>>>>> Tim. >>>>>>=20 >>>>>>>=20 >>>>>>> =A0-- Keir >>>>>>>=20 >>>>>>> On 22/04/2009 14:23, "Tom Rotenberg" wrot= e: >>>>>>>=20 >>>>>>>> Tim, >>>>>>>>=20 >>>>>>>> so what does it mean? could it be that we have a bug in the real m= ode >>>>>>>> emulation, which causes the segment state to be invalid (maybe it'= s >>>>>>>> because >>>>>>>> of >>>>>>>> a bug in the patch that Keir made for me, which emulated the LLDT,= and >>>>>>>> the >>>>>>>> LTR >>>>>>>> instructions)? >>>>>>>>=20 >>>>>>>> Keir suggested to trace back where the problem (segment state) occ= ured, >>>>>>>> and >>>>>>>> from there to try and find the bug which caused it. Do u have any >>>>>>>> better >>>>>>>> suggestion for solving this? >>>>>>>>=20 >>>>>>>> Tom >>>>>>>>=20 >>>>>>>> 2009/4/22 Tim Deegan >>>>>>>>> At 13:39 +0100 on 22 Apr (1240407546), Tom Rotenberg wrote: >>>>>>>>>> Keir, >>>>>>>>>>=20 >>>>>>>>>> I have tried your latest patch, and it looks like now it passes = the >>>>>>>>>> emulation problem. However, =A0now the domain crashes with the >>>>>>>>>> following >>>>>>>>>> error: >>>>>>>>>>=20 >>>>>>>>>> (XEN) HVM1: Booting from 0000:7c00 >>>>>>>>>> (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid >>>>>>>>>> guest >>>>>>>>>> state >>>>>>>>>> (0). >>>>>>>>>> (XEN) ************* VMCS Area ************** >>>>>>>>>> (XEN) *** Guest State *** >>>>>>>>>> (XEN) CR0: actual=3D0x0000000080010039, shadow=3D0x0000000080000019, >>>>>>>>>> gh_mask=3Dffffffffffffffff >>>>>>>>>> (XEN) CR4: actual=3D0x0000000000002060, shadow=3D0x0000000000000000, >>>>>>>>>> gh_mask=3Dffffffffffffffff >>>>>>>>>> (XEN) CR3: actual=3D0x000000000a213a20, target_count=3D0 >>>>>>>>>> (XEN) =A0 =A0 =A0target0=3D0000000000000000, target1=3D0000000000000000 >>>>>>>>>> (XEN) =A0 =A0 =A0target2=3D0000000000000000, target3=3D0000000000000000 >>>>>>>>>> (XEN) RSP =3D 0x0000000000000080 (0x0000000000000080) =A0RIP =3D >>>>>>>>>> 0x000000000000002a (0x000000000000002a) >>>>>>>>>> (XEN) RFLAGS=3D0x0000000000023202 (0x0000000000023202) =A0DR7 =3D >>>>>>>>>> 0x0000000000000400 >>>>>>>>>=20 >>>>>>>>> Looks like we're trying to VMENTER in virtual 8086 mode but witho= ut >>>>>>>>> tidying up the segment state. =A0That could either be the guest ent= ering >>>>>>>>> virtual 8086 mode itself or Xen entering vitrual 8086 mode to emu= late >>>>>>>>> real mode, but Xen is always careful to make the segment state ag= ree >>>>>>>>> with Intel's rather strict requrements when it does that. >>>>>>>>>=20 >>>>>>>>> Tim. >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>> (XEN) Sysenter RSP=3D0000000000000000 CS:RIP=3D0000:0000000000000000 >>>>>>>>>> (XEN) CS: sel=3D0x0060, attr=3D0x0c09b, limit=3D0xffffffff, >>>>>>>>>> base=3D0x0000000000200000 >>>>>>>>>> (XEN) DS: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, >>>>>>>>>> base=3D0x0000000000200000 >>>>>>>>>> (XEN) SS: sel=3D0x0070, attr=3D0x0c093, limit=3D0xfc000fff, >>>>>>>>>> base=3D0x000000000020ba62 >>>>>>>>>> (XEN) ES: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, >>>>>>>>>> base=3D0x0000000000200000 >>>>>>>>>> (XEN) FS: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, >>>>>>>>>> base=3D0x0000000000200000 >>>>>>>>>> (XEN) GS: sel=3D0x0068, attr=3D0x0c093, limit=3D0xffffffff, >>>>>>>>>> base=3D0x0000000000200000 >>>>>>>>>> (XEN) GDTR: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 limit=3D0x00001dd8, >>>>>>>>>> base=3D0x0000000000200000 >>>>>>>>>> (XEN) LDTR: sel=3D0x0000, attr=3D0x1c000, limit=3D0xffffffff, >>>>>>>>>> base=3D0x0000000000000000 >>>>>>>>>> (XEN) IDTR: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 limit=3D0x00000188, >>>>>>>>>> base=3D0x0000000000201df0 >>>>>>>>>> (XEN) TR: sel=3D0x0058, attr=3D0x0008b, limit=3D0x0000ffff, >>>>>>>>>> base=3D0x0000000000201ff2 >>>>>>>>>> (XEN) Guest PAT =3D 0x0000000000000000 >>>>>>>>>> (XEN) TSC Offset =3D ffffffe4920110b7 >>>>>>>>>> (XEN) DebugCtl=3D0000000000000000 DebugExceptions=3D0000000000000000 >>>>>>>>>> (XEN) Interruptibility=3D0001 ActivityState=3D0000 >>>>>>>>>> (XEN) *** Host State *** >>>>>>>>>> (XEN) RSP =3D 0xffff83007e4f7fa0 =A0RIP =3D 0xffff828c8019aa20 >>>>>>>>>> (XEN) CS=3De008 DS=3D0000 ES=3D0000 FS=3D0000 GS=3D0000 SS=3D0000 TR=3De040 >>>>>>>>>> (XEN) FSBase=3D0000000000000000 GSBase=3D0000000000000000 >>>>>>>>>> TRBase=3Dffff828c802a8b00 >>>>>>>>>> (XEN) GDTBase=3Dffff83007e9a3000 IDTBase=3Dffff83007e62e010 >>>>>>>>>> (XEN) CR0=3D0000000080050033 CR3=3D000000007cfdc000 CR4=3D000000000000= 26f0 >>>>>>>>>> (XEN) Sysenter RSP=3Dffff83007e4f7fd0 CS:RIP=3De008:ffff828c801c7290 >>>>>>>>>> (XEN) Host PAT =3D 0x0000000000000000 >>>>>>>>>> (XEN) *** Control State *** >>>>>>>>>> (XEN) PinBased=3D0000003f CPUBased=3Db6a1e7fe SecondaryExec=3D00000041 >>>>>>>>>> (XEN) EntryControls=3D000011ff ExitControls=3D0003efff >>>>>>>>>> (XEN) ExceptionBitmap=3D00044080 >>>>>>>>>> (XEN) VMEntry: intr_info=3D80000b0b errcode=3D00001eac ilen=3D00000000 >>>>>>>>>> (XEN) VMExit: intr_info=3D00000000 errcode=3D00008000 ilen=3D00000000 >>>>>>>>>> (XEN) =A0 =A0 =A0 =A0 reason=3D80000021 qualification=3D00000000 >>>>>>>>>> (XEN) IDTVectoring: info=3D00000000 errcode=3D00000000 >>>>>>>>>> (XEN) TPR Threshold =3D 0x00 >>>>>>>>>> (XEN) EPT pointer =3D 0x0000000000000000 >>>>>>>>>> (XEN) Virtual processor ID =3D 0x0000 >>>>>>>>>> (XEN) ************************************** >>>>>>>>>> (XEN) domain_crash called from vmx.c:2218 >>>>>>>>>> (XEN) Domain 1 (vcpu#0) crashed on cpu#1: >>>>>>>>>> (XEN) ----[ Xen-3.4.0-rc3-pre =A0x86_64 =A0debug=3Dn =A0Not tainted ]---= - >>>>>>>>>> (XEN) CPU: =A0 =A01 >>>>>>>>>> (XEN) RIP: =A0 =A00060:[<000000000000002a>] >>>>>>>>>> (XEN) RFLAGS: 0000000000023202 =A0 CONTEXT: hvm guest >>>>>>>>>> (XEN) rax: 0000000000000007 =A0 rbx: 0000000000001490 =A0 rcx: >>>>>>>>>> 0000000000000000 >>>>>>>>>> (XEN) rdx: 0000000000001da8 =A0 rsi: 0000000000000000 =A0 rdi: >>>>>>>>>> 0000000000000000 >>>>>>>>>> (XEN) rbp: 0000000000008ebf =A0 rsp: 0000000000000080 =A0 r8: >>>>>>>>>> =A00000000000000000 >>>>>>>>>> (XEN) r9: =A00000000000000000 =A0 r10: 0000000000000000 =A0 r11: >>>>>>>>>> 0000000000000000 >>>>>>>>>> (XEN) r12: 0000000000000000 =A0 r13: 0000000000000000 =A0 r14: >>>>>>>>>> 0000000000000000 >>>>>>>>>> (XEN) r15: 0000000000000000 =A0 cr0: 0000000080000019 =A0 cr4: >>>>>>>>>> 0000000000000000 >>>>>>>>>> (XEN) cr3: 0000000001443000 =A0 cr2: 0000000000000000 >>>>>>>>>> (XEN) ds: 0068 =A0 es: 0068 =A0 fs: 0068 =A0 gs: 0068 =A0 ss: 0070 =A0 cs:= 0060 >>>>>>>>>>=20 >>>>>>>>>> Could it be, that the real mode emulation code has a bug? What d= oes >>>>>>>>>> this >>>>>>>>>> error means? >>>>>>>>>>=20 >>>>>>>>>> Tom >>>>>>>>>>=20 >>>>>>>>>> 2009/4/22 Keir Fraser >>>>>>>>>> > >>>>>>>>>> On 22/04/2009 12:18, "Tom Rotenberg" >>>>>>>>>> > wrote: >>>>>>>>>>=20 >>>>>>>>>>> Keir, >>>>>>>>>>>=20 >>>>>>>>>>> I have applied your patch, and it seemed to work. However, the >>>>>>>>>>> domain >>>>>>>>>>> still >>>>>>>>>>> crashes, and now it looks like it's because of the 'LTR' >>>>>>>>>>> instruction. >>>>>>>>>>=20 >>>>>>>>>> Try the attached patch. It replaces the one I sent last time, an= d >>>>>>>>>> emulates >>>>>>>>>> both LLDT and LTR. >>>>>>>>>>=20 >>>>>>>>>> =A0-- Keir >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Content-Description: ATT00001.txt >>>>>>>>>> _______________________________________________ >>>>>>>>>> Xen-devel mailing list >>>>>>>>>> Xen-devel@lists.xensource.com >>>>>>>>>> http://lists.xensource.com/xen-devel >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> -- >>>>>>>>> Tim Deegan >>>>>>>>> Principal Software Engineer, Citrix Systems (R&D) Ltd. >>>>>>>>> [Company #02300071, SL9 0DZ, UK.] >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> Tim Deegan >>>>>> Principal Software Engineer, Citrix Systems (R&D) Ltd. >>>>>> [Company #02300071, SL9 0DZ, UK.] >>>>>=20 >>>>=20 >>>>=20 >>>=20 >>=20 >>=20 >=20