Re: pvops DomU kernels [was Re: "Hotplug Scripts not working ..." error on jaunty]

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 10/21/09 22:56, Dulloor wrote:
> console/network/everything is fine with the 2.6.18 DomU (on pvops DomU
> and on Ubuntu 9.04 server). I assume it would be so with the any other
> forward port too (2.6.27, for instance).
>
> The problem is with the pvops DomU kernel.
> 1. With Linus' git tree (CONFIG_XEN and other DomU options compiled
> in), the kernel crashes with the following (in xm dmesg) :
> (XEN) mm.c:840:d33 Error getting mfn 18c3 (pfn 21603) from L1 entry
> 80000000018c3061 for l1e_owner=33, pg_owner=33
> (XEN) traps.c:465:d33 Unhandled invalid opcode fault/trap [#6] on VCPU
> 0 [ec=0000]
> (XEN) domain_crash_sync called from entry.S
> (XEN) Domain 33 (vcpu#0) crashed on cpu#7:
> (XEN) ----[ Xen-3.5-unstable  x86_64  debug=y  Not tainted ]----
> (XEN) CPU:    7

Where in the boot does this happen?  Does anything appear on the Linux
console?  Is there a rip/eip you can map to a kernel symbol?

> 2. With Jeremy's git, i.e the same kernel as Dom0 (but with CONFIG_XEN
> and other DomU options compiled in), the kernel just freezes in
> init_intel function
> (start_kernel->...->identify_boot_cpu->...->c_init). To be more
> precise, in detect_extended_topology function. Also, for the DomU, the
> init_intel function in "arch/x86/kernel/cpu/intel.c" gets executed
> rather than the function in "setup-xen.c". I am not well-versed the
> setup code in pvops. Does that look fine ?

Which setup-xen.c would that be?  The normal CPU setup is supposed to be
run, but the extended topology info is rather meaningless in a Xen
guest.  I think the right answer is to knobble it in xen_cpuid(),
assuming it isn't stuff that Xen should be suppressing.  Does this work?

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index fb7ebc6..ecb9b0d 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -186,6 +186,7 @@ static __read_mostly unsigned int cpuid_leaf81_edx_mask = ~0;
 static void xen_cpuid(unsigned int *ax, unsigned int *bx,
 		      unsigned int *cx, unsigned int *dx)
 {
+	unsigned maskebx = ~0;
 	unsigned maskecx = ~0;
 	unsigned maskedx = ~0;
 
@@ -199,6 +200,11 @@ static void xen_cpuid(unsigned int *ax, unsigned int *bx,
 		maskedx = cpuid_leaf1_edx_mask;
 		break;
 
+	case 0xb:
+		/* Suppress extended topology stuff */
+		maskebx = 0;
+		break;
+
 	case 0x80000001:
 		maskedx = cpuid_leaf81_edx_mask;
 		break;
@@ -211,6 +217,7 @@ static void xen_cpuid(unsigned int *ax, unsigned int *bx,
 		  "=d" (*dx)
 		: "0" (*ax), "2" (*cx));
 
+	*bx &= maskebx;
 	*cx &= maskecx;
 	*dx &= maskedx;
 }



>
> I have attached the configs for both 1 and 2.
>
> thanks
> dulloor
>
> PS : Also, DomU compilation (with just DomU options compiled in) fails
> with the following error, if CONFIG_SMP is not defined -
> arch/x86/kernel/pvclock.c: In function ‘pvclock_clocksource_vread’:
> arch/x86/kernel/pvclock.c:172: error: implicit declaration of function
> ‘fix_to_virt’
> arch/x86/kernel/pvclock.c:172: error: ‘FIX_PVCLOCK_TIME_INFO’
> undeclared (first use in this function)
> arch/x86/kernel/pvclock.c:172: error: (Each undeclared identifier is
> reported only once

Does this fix it?

diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c
index 0bed867..f47f91e 100644
--- a/arch/x86/kernel/pvclock.c
+++ b/arch/x86/kernel/pvclock.c
@@ -21,6 +21,7 @@
 
 #include <asm/pvclock.h>
 #include <asm/vsyscall.h>
+#include <asm/fixmap.h>
 #include <asm/vgtod.h>
 
 /*


	J