Commit f06e4ec breaks vmware

Commit f06e4ec breaks vmware

[Jan Engelhardt]

Hi,


sad to say, but f06e4ec... breaks booting the kernel in vmware 
(bisected). Booting just stops after

	Checking for 'hlt' instruction...

commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
Author: Ingo Molnar <mingo@elte.hu>
Date:   Wed Jan 30 13:32:39 2008 +0100

    x86: read_tsc sync
    
    make native_read_tsc() always non-speculative.


Any debugging hintss?


thanks,
Jan

Re: Commit f06e4ec breaks vmware

[Petr Vandrovec]

Jan Engelhardt wrote:
> Hi,
> 
> 
> sad to say, but f06e4ec... breaks booting the kernel in vmware 
> (bisected). Booting just stops after
> 
> 	Checking for 'hlt' instruction...
> 
> commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
> Author: Ingo Molnar <mingo@elte.hu>
> Date:   Wed Jan 30 13:32:39 2008 +0100
> 
>     x86: read_tsc sync
>     
>     make native_read_tsc() always non-speculative.
> 
> 
> Any debugging hintss?

It does not make lot of sense...  What is processor (AMD/Intel, VT/SVM, 
...), kernel (32/64), and product?

Can you try adding

monitor_control.disable_rdtscopt_bt="TRUE"

or

monitor_control.virtual_rdtsc="FALSE"

to your configuration file?
					Thanks,
						Petr

Re: Commit f06e4ec breaks vmware

[Ingo Molnar]

* Jan Engelhardt <jengelh@computergmbh.de> wrote:

> sad to say, but f06e4ec... breaks booting the kernel in vmware 
> (bisected). Booting just stops after
> 
> 	Checking for 'hlt' instruction...
> 
> commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
> Author: Ingo Molnar <mingo@elte.hu>
> Date:   Wed Jan 30 13:32:39 2008 +0100
> 
>     x86: read_tsc sync
>     
>     make native_read_tsc() always non-speculative.

could you send us the /proc/cpuinfo data of the guest CPU?

if latest -git still fails, does the patch below solve the problem?

	Ingo

---
 include/asm-x86/system.h |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

Index: linux-x86.q/include/asm-x86/system.h
===================================================================
--- linux-x86.q.orig/include/asm-x86/system.h
+++ linux-x86.q/include/asm-x86/system.h
@@ -407,8 +407,7 @@ void default_idle(void);
  */
 static inline void rdtsc_barrier(void)
 {
-	alternative(ASM_NOP3, "mfence", X86_FEATURE_MFENCE_RDTSC);
-	alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
+	smp_mb();
 }
 
 #endif

Re: Commit f06e4ec breaks vmware

[Jeff Chua]

On Feb 4, 2008 7:51 PM, Ingo Molnar <mingo@elte.hu> wrote:

> > sad to say, but f06e4ec... breaks booting the kernel in vmware
> > commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71

I had the same problem. But I bisect down to a earlier commit.
Reverting this patch, and I can boot up using vmware.

commit 8d947344c47a40626730bb80d136d8daac9f2060
Author: Glauber de Oliveira Costa <gcosta@redhat.com>
Date:   Wed Jan 30 13:31:12 2008 +0100

    x86: change write_idt_entry signature


> could you send us the /proc/cpuinfo data of the guest CPU?

Here's my guest's /proc/cpuinfo

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 14
model name      : Intel(R) Core(TM) Duo CPU      L2500  @ 1.83GHz
stepping        : 8
cpu MHz         : 1828.645
cache size      : 2048 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss constant_tsc up
arch_perfmon pebs bts pni
bogomips        : 3546.54
clflush size    : 64


> if latest -git still fails, does the patch below solve the problem?

Doesn't seem to work for me.


Thanks,
Jeff.

Re: Commit f06e4ec breaks vmware

[Ingo Molnar]

* Jeff Chua <jeff.chua.linux@gmail.com> wrote:

> On Feb 4, 2008 7:51 PM, Ingo Molnar <mingo@elte.hu> wrote:
> 
> > > sad to say, but f06e4ec... breaks booting the kernel in vmware
> > > commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
> 
> I had the same problem. But I bisect down to a earlier commit.
> Reverting this patch, and I can boot up using vmware.
> 
> commit 8d947344c47a40626730bb80d136d8daac9f2060
> Author: Glauber de Oliveira Costa <gcosta@redhat.com>
> Date:   Wed Jan 30 13:31:12 2008 +0100
> 
>     x86: change write_idt_entry signature

32-bit or 64-bit guest kernel?

	Ingo

Re: Commit f06e4ec breaks vmware

[Ingo Molnar]

* Jeff Chua <jeff.chua.linux@gmail.com> wrote:

> On Feb 4, 2008 7:51 PM, Ingo Molnar <mingo@elte.hu> wrote:
> 
> > > sad to say, but f06e4ec... breaks booting the kernel in vmware
> > > commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
> 
> I had the same problem. But I bisect down to a earlier commit.
> Reverting this patch, and I can boot up using vmware.
> 
> commit 8d947344c47a40626730bb80d136d8daac9f2060
> Author: Glauber de Oliveira Costa <gcosta@redhat.com>
> Date:   Wed Jan 30 13:31:12 2008 +0100
> 
>     x86: change write_idt_entry signature

does the patch below ontop of x86.git#mm fix this?

	Ingo

---
 arch/x86/kernel/vmi_32.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Index: linux-x86.q/arch/x86/kernel/vmi_32.c
===================================================================
--- linux-x86.q.orig/arch/x86/kernel/vmi_32.c
+++ linux-x86.q/arch/x86/kernel/vmi_32.c
@@ -220,21 +220,21 @@ static void vmi_set_tr(void)
 static void vmi_write_idt_entry(gate_desc *dt, int entry, const gate_desc *g)
 {
 	u32 *idt_entry = (u32 *)g;
-	vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[2]);
+	vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[1]);
 }
 
 static void vmi_write_gdt_entry(struct desc_struct *dt, int entry,
 				const void *desc, int type)
 {
 	u32 *gdt_entry = (u32 *)desc;
-	vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[2]);
+	vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[1]);
 }
 
 static void vmi_write_ldt_entry(struct desc_struct *dt, int entry,
 				const void *desc)
 {
 	u32 *ldt_entry = (u32 *)desc;
-	vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[2]);
+	vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[1]);
 }
 
 static void vmi_load_sp0(struct tss_struct *tss,

Re: Commit f06e4ec breaks vmware

[Jeff Chua]

On Feb 4, 2008 10:53 PM, Ingo Molnar <mingo@elte.hu> wrote:
> > commit 8d947344c47a40626730bb80d136d8daac9f2060
> > Author: Glauber de Oliveira Costa <gcosta@redhat.com>
> > Date:   Wed Jan 30 13:31:12 2008 +0100
> >
> >     x86: change write_idt_entry signature
>
> does the patch below ontop of x86.git#mm fix this?


> 32-bit or 64-bit guest kernel?

32-bit.

Yep, this fixed the problem.

You're amazingly fast!

Thank you,
Jeff.

Re: Commit f06e4ec breaks vmware

[Ingo Molnar]

* Jeff Chua <jeff.chua.linux@gmail.com> wrote:

> On Feb 4, 2008 10:53 PM, Ingo Molnar <mingo@elte.hu> wrote:
> > > commit 8d947344c47a40626730bb80d136d8daac9f2060
> > > Author: Glauber de Oliveira Costa <gcosta@redhat.com>
> > > Date:   Wed Jan 30 13:31:12 2008 +0100
> > >
> > >     x86: change write_idt_entry signature
> >
> > does the patch below ontop of x86.git#mm fix this?
> 
> 
> > 32-bit or 64-bit guest kernel?
> 
> 32-bit.
> 
> Yep, this fixed the problem.

great! I've added:

   Tested-by: Jeff Chua <jeff.chua.linux@gmail.com>

to the commit message as well, if you dont mind. Full patch is below.

> You're amazingly fast!

you did all the hard work by bisecting it down so fast - fixing it was 
easy :)

	Ingo

----------------------->
Subject: x86: VMI fix
From: Ingo Molnar <mingo@elte.hu>

Jeff Chua bisected down a vmware guest boot breakage (hang) to
this paravirt change:

  commit 8d947344c47a40626730bb80d136d8daac9f2060
  Author: Glauber de Oliveira Costa <gcosta@redhat.com>
  Date:   Wed Jan 30 13:31:12 2008 +0100

    x86: change write_idt_entry signature

fix the off-by-one indexing bug ...

Bisected-by: Jeff Chua <jeff.chua.linux@gmail.com>
Tested-by: Jeff Chua <jeff.chua.linux@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 arch/x86/kernel/vmi_32.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Index: linux-x86.q/arch/x86/kernel/vmi_32.c
===================================================================
--- linux-x86.q.orig/arch/x86/kernel/vmi_32.c
+++ linux-x86.q/arch/x86/kernel/vmi_32.c
@@ -220,21 +220,21 @@ static void vmi_set_tr(void)
 static void vmi_write_idt_entry(gate_desc *dt, int entry, const gate_desc *g)
 {
 	u32 *idt_entry = (u32 *)g;
-	vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[2]);
+	vmi_ops.write_idt_entry(dt, entry, idt_entry[0], idt_entry[1]);
 }
 
 static void vmi_write_gdt_entry(struct desc_struct *dt, int entry,
 				const void *desc, int type)
 {
 	u32 *gdt_entry = (u32 *)desc;
-	vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[2]);
+	vmi_ops.write_gdt_entry(dt, entry, gdt_entry[0], gdt_entry[1]);
 }
 
 static void vmi_write_ldt_entry(struct desc_struct *dt, int entry,
 				const void *desc)
 {
 	u32 *ldt_entry = (u32 *)desc;
-	vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[2]);
+	vmi_ops.write_idt_entry(dt, entry, ldt_entry[0], ldt_entry[1]);
 }
 
 static void vmi_load_sp0(struct tss_struct *tss,

Re: Commit f06e4ec breaks vmware

[Jeff Chua]

On Feb 4, 2008 11:36 PM, Ingo Molnar <mingo@elte.hu> wrote:
> great! I've added:
> you did all the hard work by bisecting it down so fast - fixing it was
> easy :)

Ingo,

Took me the whole of Friday night. I thought it was just me and my
vmware, so I didn't bother reporting until Jan reported it. Worried
that someone might just shoot me down for reporting a vmware problem
as it booted fine with kvm, and on X60s.

Thanks again for fixing it. Now I can test latest linux git version on
vmware again (host is running linux as well).

Jeff.

Re: Commit f06e4ec breaks vmware

[Zachary Amsden]

On Mon, 2008-02-04 at 16:36 +0100, Ingo Molnar wrote:
> * Jeff Chua <jeff.chua.linux@gmail.com> wrote:
> 
> > On Feb 4, 2008 10:53 PM, Ingo Molnar <mingo@elte.hu> wrote:
> > > > commit 8d947344c47a40626730bb80d136d8daac9f2060
> > > > Author: Glauber de Oliveira Costa <gcosta@redhat.com>
> > > > Date:   Wed Jan 30 13:31:12 2008 +0100
> > > >
> > > >     x86: change write_idt_entry signature
> > >
> > > does the patch below ontop of x86.git#mm fix this?
> > 
> > 
> > > 32-bit or 64-bit guest kernel?
> > 
> > 32-bit.
> > 
> > Yep, this fixed the problem.
> 
> great! I've added:
> 
>    Tested-by: Jeff Chua <jeff.chua.linux@gmail.com>
> 
> to the commit message as well, if you dont mind. Full patch is below.

Acked-by: Zachary Amsden <zach@vmware.com>

Thanks, Ingo!

Re: Commit f06e4ec breaks vmware

[Jan Engelhardt]

On Feb 3 2008 20:19, Petr Vandrovec wrote:
>> 
>> sad to say, but f06e4ec... breaks booting the kernel in vmware (bisected).
>> Booting just stops after
>> 
>>  Checking for 'hlt' instruction...
>> 
>> commit f06e4ec1c15691b0cfd2397ae32214fa36c90d71
>> Author: Ingo Molnar <mingo@elte.hu>
>> Date:   Wed Jan 30 13:32:39 2008 +0100
>> 
>>     x86: read_tsc sync
>>     
>>     make native_read_tsc() always non-speculative.
>> 
>> Any debugging hints?
>
> It does not make lot of sense...  What is processor (AMD/Intel, VT/SVM, ...),
> kernel (32/64), and product?

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 6
model           : 8
model name      : AMD Athlon(tm) XP 2000+
stepping        : 0
cpu MHz         : 1666.779
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow up ts
bogomips        : 3334.92
clflush size    : 32

No hardware virtualization.
Host: 32-bit 2.6.23.14, VMware Workstation 6.0.2-b59824.
Guest: 32-bit f06e4ec...


> Can you try adding
>
> monitor_control.disable_rdtscopt_bt="TRUE"

Nope.

> monitor_control.virtual_rdtsc="FALSE"

No change either.


But booting the guest kernel with 'notsc' works around the issue.


On Feb 4 2008 12:51, Ingo Molnar wrote:
>
>if latest -git still fails, does the patch below solve the problem?

Latest git (9135f1901ee6449dfe338adf6e40e9c2025b8150) works, as does the 
patch on top of f06e4ec1c15691b0cfd2397ae32214fa36c90d71.


>@@ -407,8 +407,7 @@ void default_idle(void);
>  */
> static inline void rdtsc_barrier(void)
> {
>-	alternative(ASM_NOP3, "mfence", X86_FEATURE_MFENCE_RDTSC);
>-	alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
>+	smp_mb();
> }
> 
> #endif
>