Re: PV-GRUB crashes with FC11 PV guest.

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 09/16/09 08:01, Konrad Rzeszutek Wilk wrote:
> This is with cs 20189. I hadn't tried to bisect builds
> to narrow this down.
>
>
> tst070 login: 
> Fedora release 11 (Leonidas)
> Kernel 2.6.31-xen-01396-g12e8537 on an x86_64 (/dev/hvc0)
>
> tst070 login: [  407.456209] device vif2.0 entered promiscuous mode
> [  407.470978] eth1: port 2(vif2.0) entering forwarding state
> Bootstrapping...
> Xen Minimal OS!
>   start_info: 0xaa0000(VA)
>     nr_pages: 0x20000
>   shared_inf: 0x7f810000(MA)
>      pt_base: 0xaa3000(VA)
> nr_pt_frames: 0x9
>     mfn_list: 0x9a0000(VA)
>    mod_start: 0x0(VA)
>      mod_len: 0
>        flags: 0x0
>     cmd_line: (hd0,0)/grub/menu.lst
>   stack:      0x95f9a0-0x97f9a0
> MM: Init
>       _text: 0x0(VA)
>      _etext: 0x64c8c(VA)
>    _erodata: 0x89000(VA)
>      _edata: 0x91b20(VA)
> stack start: 0x95f9a0(VA)
>        _end: 0x99ffa8(VA)
>   start_pfn: aaf
>     max_pfn: 20000
> Mapping memory range 0xc00000 - 0x20000000
> setting 0x0-0x89000 readonly
> skipped 0x1000
> MM: Initialise page allocator for ba9000(ba9000)-20000000(20000000)
> MM: done
> Demand map pfns at 20001000-2020001000.
> Heap resides at 2020002000-4020002000.
> Initialising timer interface
> Initialising console ... done.
> gnttab_table mapped at 0x20001000.
> Initialising scheduler
> Thread "Idle": pointer: 0x2020002050, stack: 0xcc0000
> Initialising xenbus
> Thread "xenstore": pointer: 0x2020002800, stack: 0xcd0000
> Dummy main: start_info=0x97faa0
> Thread "main": pointer: 0x2020002fb0, stack: 0xce0000
> vbd 768 is hd0
> ******************* BLKFRONT for device/vbd/768 **********
>
>
> backend at /local/domain/0/backend/vbd/2/768
> [  407.988989] blkback: ring-ref 2047, event-channel 4, protocol 1 (x86_64-abi)
> Failed to read /local/domain/0/backend/vbd/2/768/feature-flush-cache.
> 76389075 sectors of 512 bytes
> **************************
> Thread "kbdfront": pointer: 0x2020130010, stack: 0xcf0000
> ******************* FBFRONT for device/vfb/0 **********
>
>
> ******************* KBDFRONT for device/vkbd/0 **********
>
>
> Failed to read device/vfb/0/backend-id.
> Failed to read device/vkbd/0/backend-id.
> error writing initialized: ENOENT
> GPF rip: 0x54d6b, error_code=0
> Thread: main
> RIP: e030:[<0000000000054d6b>] 
> RSP: e02b:0000000000cefcc8  EFLAGS: 00010212
> RAX: 0000000000000000 RBX: 0000002020002ff0 RCX: 0000000000091200
> RDX: 0000000000001000 RSI: 2f302f6e69616d6f RDI: 00000000000919e0
> RBP: 0000000000cefcd8 R08: 0000000000000001 R09: 0000000000000000
> R10: 0000000000000008 R11: 000000000006cd80 R12: 2f302f6e69616d6f
> R13: 0000000000cefd8c R14: 0000000000007118 R15: 0000000000cefd30
> base is 0xcefcd8 caller is 0x55e2c
> base is 0xcefcf8 caller is 0x417bb
> base is 0xcefd18 caller is 0x4240c
> base is 0xcefdc8 caller is 0x384d
> base is 0xcefe78 caller is 0xdb4b
> base is 0xcefe88 caller is 0x10eb1
> base is 0xceff48 caller is 0x41c3
> base is 0xceff58 caller is 0x47928
> base is 0xceffe8 caller is 0x33da
>
> cefcb0: c8 fc ce 00 00 00 00 00 2b e0 00 00 00 00 00 00
> cefcc0: 6b 4d 05 00 00 00 00 00 6f 6d 61 69 6e 2f 30 2f
> cefcd0: 90 fd ce 00 00 00 00 00 f8 fc ce 00 00 00 00 00
> cefce0: 2c 5e 05 00 00 00 00 00 cb e6 06 00 00 00 00 00
>
> cefcc0: 6b 4d 05 00 00 00 00 00 6f 6d 61 69 6e 2f 30 2f
> cefcd0: 90 fd ce 00 00 00 00 00 f8 fc ce 00 00 00 00 00
> cefce0: 2c 5e 05 00 00 00 00 00 cb e6 06 00 00 00 00 00
> cefcf0: 90 3d 00 20 20 00 00 00 18 fd ce 00 00 00 00 00
>
> 54d50: 55 48 85 f6 48 89 e5 41 54 49 89 f4 53 48 89 fb
> 54d60: 0f 84 bc 02 00 00 e8 85 1c 00 00 4d 8b 44 24 f8
> 54d70: 49 8d 54 24 f0 be c0 11 09 00 4c 89 c7 48 83 e7
> 54d80: fe 48 8d 04 3a 48 8b 48 08 48 83 e1 fc 48 39 05
> [  409.769670] eth1: port 2(vif2.0) entering disabled state
> [  409.777601] eth1: port 2(vif2.0) entering disabled state
>
> The .xm file is rather simple:
>
> disk = [ 'phy:/dev/sda4,hda,w' ]
> extra = "(hd0,0)/grub/menu.lst"
> kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
> memory = 512
> name = "FC11"
> vif = [ 'mac=00:16:3e:00:00:11, bridge=eth1' ]
>
> If try to use kernel and ramdisk extracted from the disk, the
> guest boots up fine:
>
> [  420.424372] device vif3.0 entered promiscuous mode
> [  420.439713] eth1: port 2(vif3.0) entering forwarding state
> mapping kernel into physical memory
> about to get started...
> (XEN) domain.c:592:d3 Attempt to change CR4 flags 00000660 -> 00000620
> [  421.498616] blkback: ring-ref 8, event-channel 9, protocol 1 (x86_64-abi)
> [  426.849150]   alloc irq_desc for 119 on node 0
> [  426.849623]   alloc kstat_irqs on node 0
> [  431.220055] vif3.0: no IPv6 routers present
> (XEN) traps.c:2264:d3 Domain attempted WRMSR 000000000000008b from 00000017:00000000 to 00000000:00000000.
>
>   

I can start a 64-bit PV guest OK, but I get a very similar-looking crash
with 32-bit guests:

Started domain f10pv32 (id=17)
                              Xen Minimal OS!
  start_info: 0x9d6000(VA)
    nr_pages: 0x18000
  shared_inf: 0x3aa43000(MA)
     pt_base: 0x9d9000(VA)
nr_pt_frames: 0x9
    mfn_list: 0x976000(VA)
   mod_start: 0x0(VA)
     mod_len: 0
       flags: 0x0
    cmd_line: (hd0,0)/grub/menu.lst
  stack:      0x954e00-0x974e00
MM: Init
      _text: 0x0(VA)
     _etext: 0x65ab8(VA)
   _erodata: 0x85000(VA)
     _edata: 0x8a6d4(VA)
stack start: 0x954e00(VA)
       _end: 0x9753b4(VA)
  start_pfn: 9e5
    max_pfn: 18000
Mapping memory range 0xc00000 - 0x18000000
  Booting command-list

root (hd0,0)
Error ENOENT when reading the backend path device/vkbd/0/backend
Page fault at linear address 0x1c, eip 0x4cdae, regs 0xb4ff7c, sp 0xb4fff0, our_sp 0xb4ff50, code 2
Thread: kbdfront
EIP: 4cdae, EFLAGS 10002.
EBX: 746f6f72 ECX: 00000013 EDX: 00000000
ESI: 00b3fe98 EDI: 00000001 EBP: 00b4ffb0 EAX: 00000000
DS: c2c2e021 ES: e021 orig_eax: ffffffff, eip: 0004cdae
CS: 1e019 EFLAGS: 00010002 esp: 00b4fff0 ss: 37eb
base is 0xb4ffb0 caller is 0x37eb
base is 0xb4fff0 caller is 0x31ad

b4ffe0: c2 c2 c2 c2 8f 37 00 00 9a 14 00 00 40 20 00 98
b4fff0: 00 00 00 00 ad 31 00 00 94 fe b3 00 00 00 00 00
b50000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b50010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

b4ffa0: ff ff ff ff ae cd 04 00 19 e0 01 00 02 00 01 00
b4ffb0: f0 ff b4 00 eb 37 00 00 00 00 00 00 01 00 00 00
b4ffc0: c2 c2 c2 c2 c2 c2 c2 c2 c2 c2 c2 c2 c2 c2 c2 c2
b4ffd0: c2 c2 c2 c2 c2 c2 c2 00 c2 c2 c2 c2 c2 c2 c2 c2

4cd90: e5 8b 45 08 c7 40 1c 00 00 00 00 c7 40 20 00 00
4cda0: 00 00 83 60 18 fe 5d c3 55 89 e5 8b 45 08 c7 40
4cdb0: 1c 00 00 00 00 c7 40 20 00 00 00 00 83 48 18 01
4cdc0: 5d c3 55 89 e5 53 83 ec 14 a1 e0 01 95 00 8b 58
Pagetable walk from virt 1c, base 9d9000:
  L3 = 000000003a05b027 (0x9da000)  [offset = 0]
   L2 = 000000003a059067 (0x9dc000)  [offset = 0]
    L1 = 0000000000000000 [offset = 0]


I have vfb disabled.  If I have it enabled then I can (apparently) start the kernel
but I'm not seeing any output for some (likely unrelated) reason.

I have a patch Samuel gave me when I first reported this, but it doesn't seem to have
helped.  I wonder if there's something else being left uninitialized?

	J