[hybrid]: unable to boot hvm due to eflags.ID

[hybrid]: unable to boot hvm due to eflags.ID

[Mukesh Rathor]

Hi guys,

At a loss trying to figure why 
  if (has_eflag(X86_EFLAGS_ID))

returns false in my HVM domU. Standard function has_eflag() in
cpucheck.c running in real mode. Works fine on PV dom0, but fails when
guest is booting on my hybrid dom0.

LMK if any ideas. I'll keep digging in the manuals, but nothing so far.

thanks,
Mukesh

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Jan Beulich]

>>> On 04.05.12 at 04:10, Mukesh Rathor <mukesh.rathor@oracle.com> wrote:
> At a loss trying to figure why 
>   if (has_eflag(X86_EFLAGS_ID))
> 
> returns false in my HVM domU. Standard function has_eflag() in
> cpucheck.c running in real mode. Works fine on PV dom0,

How that? This is 16-bit code. If called with a 32-bit code segment (I'm
not aware of PV setting up any 16-bit segments), it'll be touching only
the low 16 bits of all operands (as the operand size prefixes all these
instructions have will have inverted meaning).

> but fails when guest is booting on my hybrid dom0.

Unless PV works because you made this 32-bit code (and you call this
in a 16-bit code segment in your hybrid), I can't make other suggestions.

Jan

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Ian Campbell]

On Fri, 2012-05-04 at 03:10 +0100, Mukesh Rathor wrote:
> Hi guys,
> 
> At a loss trying to figure why 
>   if (has_eflag(X86_EFLAGS_ID))
> 
> returns false in my HVM domU. Standard function has_eflag() in
> cpucheck.c running in real mode. Works fine on PV dom0,

Are you sure code in cpucheck.c is even being called for the PV/dom0
case? I'm reasonably sure that PV Xen boot doesn't go anywhere near
arch/x86/boot -- we have a totally different entry point.

> but fails when guest is booting on my hybrid dom0.

What does the guest EFLAGS actually contain at this point? Is it
different to what it contains on non-hybrid dom0?

How do we execute 16 bit code in a VMX guest these days, using vm86 or
by emulation? How does that interact with EFLAGS_ID I wonder (although
why hybrid dom0 would have any bearing on this I've no idea).

> LMK if any ideas. I'll keep digging in the manuals, but nothing so far.
> 
> thanks,
> Mukesh
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Tim Deegan]

At 10:08 +0100 on 04 May (1336126102), Ian Campbell wrote:
> What does the guest EFLAGS actually contain at this point? Is it
> different to what it contains on non-hybrid dom0?
> 
> How do we execute 16 bit code in a VMX guest these days, using vm86 or
> by emulation?

Both, and also sometimes neither. :) 

If the chip supports it, we just run in real mode inside the VM (this is
called "unrestricted guest" in the VMX code and will be reported in the
Xen boot messages).

Otherwise we run in vm86 when we can, and emulate things that vm86
doesn't support, and all I/O.  We also have to emulate when the segment
state is wierd (e.g., big-real-mode) because the VMENTER into vm86 has
stricter checks on segment state than actual real mode.

> How does that interact with EFLAGS_ID I wonder (although
> why hybrid dom0 would have any bearing on this I've no idea).

I'm not aware of anything deliberately tinkering with EFLAGS_ID, but
it's possible that vm86 interacts with it (of that we accidentally lose
some parts of EFLAGS in emulation).

Tim.

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Ian Campbell]

On Fri, 2012-05-04 at 11:49 +0100, Tim Deegan wrote:
> At 10:08 +0100 on 04 May (1336126102), Ian Campbell wrote:
> > What does the guest EFLAGS actually contain at this point? Is it
> > different to what it contains on non-hybrid dom0?
> > 
> > How do we execute 16 bit code in a VMX guest these days, using vm86 or
> > by emulation?
> 
> Both, and also sometimes neither. :) 

Of course ;-)

> If the chip supports it, we just run in real mode inside the VM (this is
> called "unrestricted guest" in the VMX code and will be reported in the
> Xen boot messages).

Thanks, I thought this had been added but didn't know the name of the
feature.

> Otherwise we run in vm86 when we can, and emulate things that vm86
> doesn't support, and all I/O.  We also have to emulate when the segment
> state is wierd (e.g., big-real-mode) because the VMENTER into vm86 has
> stricter checks on segment state than actual real mode.
> 
> > How does that interact with EFLAGS_ID I wonder (although
> > why hybrid dom0 would have any bearing on this I've no idea).
> 
> I'm not aware of anything deliberately tinkering with EFLAGS_ID,

I couldn't find any use other than the definition, although it maybe
open coded somewhere.

> but
> it's possible that vm86 interacts with it (of that we accidentally lose
> some parts of EFLAGS in emulation).

Yeah, but I can't figure out why hybrid dom0 would change that...

Ian,

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Tim Deegan]

[-- Attachment #1: Type: text/plain, Size: 671 bytes --]

At 11:53 +0100 on 04 May (1336132407), Ian Campbell wrote:
> > I'm not aware of anything deliberately tinkering with EFLAGS_ID,
> 
> I couldn't find any use other than the definition, although it maybe
> open coded somewhere.

Looks like maybe it's the absence of it that we should worry about:
Mukesh, can you try the attached patch?

> > but
> > it's possible that vm86 interacts with it (of that we accidentally lose
> > some parts of EFLAGS in emulation).
> 
> Yeah, but I can't figure out why hybrid dom0 would change that...

Indeed.  I'd expect this to fail for the same kernel in normal HVM.
Maybe there's some difference in how the segment state is set up

Tim.

[-- Attachment #2: x --]
[-- Type: text/plain, Size: 889 bytes --]

diff -r 8f556a70ae0b xen/arch/x86/x86_emulate/x86_emulate.c
--- a/xen/arch/x86/x86_emulate/x86_emulate.c	Thu May 03 17:21:09 2012 +0100
+++ b/xen/arch/x86/x86_emulate/x86_emulate.c	Fri May 04 12:13:55 2012 +0100
@@ -372,6 +372,7 @@ typedef union {
 #define CR4_TSD   (1<<2)
 
 /* EFLAGS bit definitions. */
+#define EFLG_ID   (1<<21)
 #define EFLG_VIP  (1<<20)
 #define EFLG_VIF  (1<<19)
 #define EFLG_AC   (1<<18)
@@ -424,7 +425,7 @@ typedef union {
  * These EFLAGS bits are restored from saved value during emulation, and
  * any changes are written back to the saved value after emulation.
  */
-#define EFLAGS_MASK (EFLG_OF|EFLG_SF|EFLG_ZF|EFLG_AF|EFLG_PF|EFLG_CF)
+#define EFLAGS_MASK (EFLG_OF|EFLG_SF|EFLG_ZF|EFLG_AF|EFLG_PF|EFLG_CF|EFLG_ID)
 
 /* Before executing instruction: restore necessary bits in EFLAGS. */
 #define _PRE_EFLAGS(_sav, _msk, _tmp)                           \

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Tim Deegan]

(oops, seem to have dropped Mukesh from CC)

At 12:14 +0100 on 04 May (1336133690), Tim Deegan wrote:
> At 11:53 +0100 on 04 May (1336132407), Ian Campbell wrote:
> > > I'm not aware of anything deliberately tinkering with EFLAGS_ID,
> > 
> > I couldn't find any use other than the definition, although it maybe
> > open coded somewhere.
> 
> Looks like maybe it's the absence of it that we should worry about:
> Mukesh, can you try the attached patch?
> 
> > > but
> > > it's possible that vm86 interacts with it (of that we accidentally lose
> > > some parts of EFLAGS in emulation).
> > 
> > Yeah, but I can't figure out why hybrid dom0 would change that...
> 
> Indeed.  I'd expect this to fail for the same kernel in normal HVM.
> Maybe there's some difference in how the segment state is set up
> 
> Tim.

> diff -r 8f556a70ae0b xen/arch/x86/x86_emulate/x86_emulate.c
> --- a/xen/arch/x86/x86_emulate/x86_emulate.c	Thu May 03 17:21:09 2012 +0100
> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c	Fri May 04 12:13:55 2012 +0100
> @@ -372,6 +372,7 @@ typedef union {
>  #define CR4_TSD   (1<<2)
>  
>  /* EFLAGS bit definitions. */
> +#define EFLG_ID   (1<<21)
>  #define EFLG_VIP  (1<<20)
>  #define EFLG_VIF  (1<<19)
>  #define EFLG_AC   (1<<18)
> @@ -424,7 +425,7 @@ typedef union {
>   * These EFLAGS bits are restored from saved value during emulation, and
>   * any changes are written back to the saved value after emulation.
>   */
> -#define EFLAGS_MASK (EFLG_OF|EFLG_SF|EFLG_ZF|EFLG_AF|EFLG_PF|EFLG_CF)
> +#define EFLAGS_MASK (EFLG_OF|EFLG_SF|EFLG_ZF|EFLG_AF|EFLG_PF|EFLG_CF|EFLG_ID)
>  
>  /* Before executing instruction: restore necessary bits in EFLAGS. */
>  #define _PRE_EFLAGS(_sav, _msk, _tmp)                           \

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Jan Beulich]

>>> On 04.05.12 at 13:14, Tim Deegan <tim@xen.org> wrote:
> At 11:53 +0100 on 04 May (1336132407), Ian Campbell wrote:
>> > I'm not aware of anything deliberately tinkering with EFLAGS_ID,
>> 
>> I couldn't find any use other than the definition, although it maybe
>> open coded somewhere.
> 
> Looks like maybe it's the absence of it that we should worry about:
> Mukesh, can you try the attached patch?

But EFLAGS_MASK doesn't affect PUSHF/POPF emulation, and that's
the only ones that would matter here (short of some of the arithmetic
emulation being broken, which wouldn't be addressed by this mask
change either). POPF emulation itself only disallows modifying VIP,
VIF, and VM.

Jan

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Mukesh Rathor]

On Fri, 4 May 2012 10:08:22 +0100
Ian Campbell <Ian.Campbell@citrix.com> wrote:

> Are you sure code in cpucheck.c is even being called for the PV/dom0
> case? I'm reasonably sure that PV Xen boot doesn't go anywhere near
> arch/x86/boot -- we have a totally different entry point.

I didn't make myself clear (rushing to type email and leave with a 
brain fried from debugging all day :).) 

I'm talking about HVM domU on PV/dom0.

Basically, I've the PV linux kernel modified for hybrid. I can boot it
as both hybrid domU (say Lu) and hybrid dom0 (say L0).

  Orig/normal PV DOM0: Lu boots in hybrid mode, PV mode, HVM mode.
  L0 as Dom0: Lu boots in hybrid mode, PV mode but in  HVM mode fails.

  HVM mode is failing because very early during boot its failing the 
  eflags.ID test, which should be simple. It should be running
  in real mode in the VMX. I just got outb working, so I can debug
  further now. It's not doing vmexit's so should be running in the
  'unrestricted mode' in the container, which is why i'm baffled.
   Anyways, i'll dig further.

thanks a lot,
Mukesh

Re: [hybrid]: unable to boot hvm due to eflags.ID

[Mukesh Rathor]

On Fri, 4 May 2012 11:19:35 -0700
Mukesh Rathor <mukesh.rathor@oracle.com> wrote:

>   HVM mode is failing because very early during boot its failing the 
>   eflags.ID test, which should be simple. It should be running
>   in real mode in the VMX. I just got outb working, so I can debug
>   further now. It's not doing vmexit's so should be running in the
>   'unrestricted mode' in the container, which is why i'm baffled.
>    Anyways, i'll dig further.

Now that I'm able to print stuff, because of the outb working, I see
the problem is executing cpuid and not eflags.ID. I should be able
to nail this soon.

Thanks a lot,
Mukesh