2.6.27-rc4 pvops domU on Xen 3.3 Kernel panic

2.6.27-rc4 pvops domU on Xen 3.3 Kernel panic

[Todd Deshane]

I'm trying to build a kernel.org domU to run on Xen 3.3

My first attempt produced a kernel panic.

Below is my setup and the panic that occurred during boot right after:

[    0.004000] Initializing cgroup subsys cpuacct
[    0.008000]
[    0.008000] CPU 0/0 -> Node 0
[    0.008000] CPU: Physical Processor ID: 0
[    0.008000] CPU: Processor Core ID: 0
[    0.008000] SMP alternatives: switching to UP code
[    0.020001] Freeing SMP alternatives: 22k freed

The same panic happens for both default vcpus, vcpus=1, and vcpus=2

Any ideas or hints?

Thanks in advance,
Todd


dom0:
Xen 3.3, 2.6.18.8 from 3.3 release.

domU:
2.6.27-rc4 (from kernel.org)

CONFIG_XEN=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=32
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_NETXEN_NIC=m
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_XEN_FBDEV_FRONTEND=y
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y

CONFIG_PARAVIRT_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_PARAVIRT_CLOCK=y
CONFIG_PARAVIRT_DEBUG=y

CONFIG_HVC_DRIVER=y
CONFIG_HVC_IRQ=y
CONFIG_HVC_XEN=y

full .config here:
http://pastebin.com/m5a665cc0


[    0.020001] ------------[ cut here ]------------
[    0.020001] kernel BUG at include/asm/paravirt.h:1092!
[    0.020001] invalid opcode: 0000 [1] SMP
[    0.020001] CPU 0
[    0.020001] Modules linked in:
[    0.020001] Pid: 0, comm: swapper Not tainted 2.6.27-rc4 #4
[    0.020001] RIP: e030:[<ffffffff80230718>]  [<ffffffff80230718>]
__change_page_attr+0x268/0x280
[    0.020001] RSP: e02b:ffffffff80645ed8  EFLAGS: 00010206
[    0.020001] RAX: 00003ffffffff000 RBX: 001000011938a027 RCX: 0000000000000000
[    0.020001] RDX: ffff800000000000 RSI: ffffffff80645efc RDI: 001000011938a027
[    0.020001] RBP: ffffffff80645efc R08: 0000000000000001 R09: 0000000000000002
[    0.020001] R10: 000000000000066c R11: 0000000000000000 R12: ffff88000ae06000
[    0.020001] R13: ffff880000000000 R14: ffffffff80645f68 R15: 0000000000000001
[    0.020001] FS:  0000000000000000(0000) GS:ffffffff8063c380(0000)
knlGS:0000000000000000
[    0.020001] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.020001] CR2: 0000000000000000 CR3: 0000000000201000 CR4: 0000000000002620
[    0.020001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.020001] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.020001] Process swapper (pid: 0, threadinfo ffffffff80644000,
task ffffffff805e3340)
[    0.020001] Stack:  0000000000000001 ffffffff80577fdf
001000011938a027 ffffe200000167a0
[    0.020001]  00000001804933bf ffffffff80645f68 0000000000000001
0000000000000000
[    0.020001]  0000000000000001 0000000000000001 0000000000000001
ffffffff80230785
[    0.020001] Call Trace:
[    0.020001]  [<ffffffff80230785>] ? __change_page_attr_set_clr+0x55/0x80
[    0.020001]  [<ffffffff80230853>] ? change_page_attr_set_clr+0xa3/0x160
[    0.020001]  [<ffffffff8022d897>] ? free_init_pages+0xa7/0xc0
[    0.020001]  [<ffffffff8066de9f>] ? start_kernel+0x20f/0x300
[    0.020001]
[    0.020001]
[    0.020001] Code: e9 48 c7 c2 80 f2 57 80 be 5b 02 00 00 48 c7 c7
75 ec 57 80 e8 ea 48 01 00 b8 ea ff ff ff e9 26 fe ff ff 0f 0b eb fe
0f 0b eb fe <0f> 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb fe 0f 0b eb
fe 0f
[    0.020001] RIP  [<ffffffff80230718>] __change_page_attr+0x268/0x280
[    0.020001]  RSP <ffffffff80645ed8>
[    0.024001] ---[ end trace 4eaa2a86a8e2da22 ]---

guest config:

kernel = "/root/kernels/vmlinuz-2.6.27-rc4"
ramdisk = "/root/kernels/initrd.img-2.6.27-rc4"
root="/dev/hda2 ro"
memory = 512
disk = [ 'tap:aio:/root/benchvm/bin/img-xen-guest/image-3.img,hda,w' ]
vif = [ 'bridge=br0','bridge=br1']
extra='console=hvc0'

Re: 2.6.27-rc4 pvops domU on Xen 3.3 Kernel panic

[Jeremy Fitzhardinge]

Todd Deshane wrote:
> I'm trying to build a kernel.org domU to run on Xen 3.3
>
> My first attempt produced a kernel panic.
>
> Below is my setup and the panic that occurred during boot right after:
>
> [    0.004000] Initializing cgroup subsys cpuacct
> [    0.008000]
> [    0.008000] CPU 0/0 -> Node 0
> [    0.008000] CPU: Physical Processor ID: 0
> [    0.008000] CPU: Processor Core ID: 0
> [    0.008000] SMP alternatives: switching to UP code
> [    0.020001] Freeing SMP alternatives: 22k freed
>
> The same panic happens for both default vcpus, vcpus=1, and vcpus=2
>
> Any ideas or hints?
>   

Ah, it's a bug which has been since fixed, but it looks like the fix
hasn't made it into mainline.  Patch below.

(Ingo, can you push this to Linus?  It needs to be in mainline.)

    J

commit b56afe1d41653fb07ab1b5af5ccc12001c4dd5a0
Author: Eduardo Habkost <ehabkost@redhat.com>
Date:   Thu Jul 24 12:15:45 2008 -0300

    x86, xen: Use native_pte_flags instead of native_pte_val for .pte_flags
    
    Using native_pte_val triggers the BUG_ON() in the paravirt_ops
    version of pte_flags().
    
    Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
    Acked-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
    Signed-off-by: Ingo Molnar <mingo@elte.hu>

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 06219e6..e2767c2 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1347,7 +1347,7 @@ static const struct pv_mmu_ops xen_mmu_ops __initdata = {
 	.ptep_modify_prot_commit = __ptep_modify_prot_commit,
 
 	.pte_val = xen_pte_val,
-	.pte_flags = native_pte_val,
+	.pte_flags = native_pte_flags,
 	.pgd_val = xen_pgd_val,
 
 	.make_pte = xen_make_pte,

Re: 2.6.27-rc4 pvops domU on Xen 3.3 Kernel panic

[Todd Deshane]

>
> Ah, it's a bug which has been since fixed, but it looks like the fix
> hasn't made it into mainline.  Patch below.

That was the problem, it boots now. Thanks for the quick fix.

Also thanks for theXenParavirtOps wiki page
(http://wiki.xensource.com/xenwiki/XenParavirtOps).
It really helped to get this up and going.

The one issue that I am currently having though is, I am not getting a
login prompt.

This has been a long standing issue with the Ubuntu userspace.
Generally fixed with
extra=xencons=tty or the like.

I have also tried extra=console=hvc0

Both of these get me to just before there would be a "login:" prompt.

I can successful SSH to the box, so it is fully up and not crashed,
but no login prompt.

Any ideas or hints?

This may be some Ubuntu-specific hack to getty or the like, but all
the old tricks that
I can think of don't seem to help.

Thanks in advance,
Todd

-- 
Todd Deshane
http://todddeshane.net
check out our book: http://runningxen.com

Re: 2.6.27-rc4 pvops domU on Xen 3.3 Kernel panic

[Todd Deshane]

>
> Both of these get me to just before there would be a "login:" prompt.

Found the solution:

in:
/etc/event.d/tty1

change:
exec /sbin/getty 38400 tty1
to:
exec /sbin/getty 38400 hvc0

Now I have a login prompt too :)

Cheers,
Todd

-- 
Todd Deshane
http://todddeshane.net
check out our book: http://runningxen.com

Re: 2.6.27-rc4 pvops domU on Xen 3.3 Kernel panic

[Ingo Molnar]

* Jeremy Fitzhardinge <jeremy@goop.org> wrote:

> Todd Deshane wrote:
> > I'm trying to build a kernel.org domU to run on Xen 3.3
> >
> > My first attempt produced a kernel panic.
> >
> > Below is my setup and the panic that occurred during boot right after:
> >
> > [    0.004000] Initializing cgroup subsys cpuacct
> > [    0.008000]
> > [    0.008000] CPU 0/0 -> Node 0
> > [    0.008000] CPU: Physical Processor ID: 0
> > [    0.008000] CPU: Processor Core ID: 0
> > [    0.008000] SMP alternatives: switching to UP code
> > [    0.020001] Freeing SMP alternatives: 22k freed
> >
> > The same panic happens for both default vcpus, vcpus=1, and vcpus=2
> >
> > Any ideas or hints?
> >   
> 
> Ah, it's a bug which has been since fixed, but it looks like the fix
> hasn't made it into mainline.  Patch below.
> 
> (Ingo, can you push this to Linus?  It needs to be in mainline.)

ok - cherry-picked it into tip/x86/urgent.

	Ingo