Re: pvops dom0 work roadmap

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Wed, Aug 19, 2009 at 10:17:00AM -0700, Jeremy Fitzhardinge wrote:
> On 08/19/09 06:26, Konrad Rzeszutek Wilk wrote:
> >  - if you don't have CONFIG_HIBERNATION set it hangs at startup on the VGA init code (right after "about to get started").  I played
> >    with turning CONFIG_SWAP off, which in turn unselects _HIBERNATION off.
> >   
> 
> It looks like I've been testing without CONFIG_HIBERNATION enabled, so I
> can't repro this.  Are there any messages, either from the kernel log or
> in the Xen log?  What does pressing '0' on the Xen console show?  Are
They are from Linux:

<BEGIN>
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 140kB init memory.
mapping kernel into physical memory
Xen: setup ISA identity maps
<END>

Instrumenting the code lightly (xen_raw_console_write sprinkled) about shows that:
1132                 xen_init_vga(info, xen_start_info->console.dom0.info_size);

is the last time I see anything.

These are the boot options:

 APPEND xen.gz  com1=115200,8n1,0xcf00,0  console=com1,vga guest_loglvl=all  dom0_mem=max:512MB--- vmlinuz debug selinux=0 initrd_ignore_loglevel sysrq_always_enable ip=:::::eth1:dhcp earlyprintk=xen,keep irqpoll console=tty console=hvc0 loglevel=10 bb_inittab=/etc/inittab-debug  --- initramfs.cpio.gz

And my serial console, keyboard, nor the VGA screen show any activity.
I didn't really dig any deeper than that.


I hadn't tried to press '0'.
> you building Xen with DEBUG=y?

I belive so.


I will retry this, but in the meantime I am tracking balloon accessing non-existent pages:

<BEGIN>
[    0.239963] xen_balloon: Initialising balloon driver.
[    0.240057] xen_ballon: 131072 -> 1245184		<-- added a printk. 
[    0.257938] BUG: unable to handle kernel [    0.257938] BUG: unable to handle kernel paging requestpaging request at ffffea0002d80028
 at ffffea0002d80028
[    0.257960] IP:[    0.257960] IP: [<ffffffff81215f7b>] balloon_append+0x13/0x36
 [<ffffffff81215f7b>] balloon_append+0x13/0x36
[    0.257960] PGD 10402067 [    0.257960] PGD 10402067 PUD 10403067 PUD 10403067 PMD 1301a067 PMD 1301a067 PTE 0PTE 0

[    0.257960] Oops: 0002 [#1] [    0.257960] Oops: 0002 [#1] SMP SMP 

[    0.257960] last sysfs file: 
[    0.257960] last sysfs file: 
[    0.257960] CPU 0 [    0.257960] CPU 0 

[    0.257960] Modules linked in:[    0.257960] Modules linked in:

[    0.257960] Pid: 1, comm: swapper Not tainted 2.6.31-rc5-xen0-23866-g8584ce8-dirty #6 .  
[    0.257960] Pid: 1, comm: swapper Not tainted 2.6.31-rc5-xen0-23866-g8584ce8-dirty #6 .  
[    0.257960] RIP: e030:[<ffffffff81215f7b>] [    0.257960] RIP: e030:[<ffffffff81215f7b>]  [<ffffffff81215f7b>] balloon_append+0x13/0x36
 [<ffffffff81215f7b>] balloon_append+0x13/0x36
[    0.257960] RSP: e02b:ffff88001fc81e70  EFLAGS: 00010282
[    0.257960] RSP: e02b:ffff88001fc81e70  EFLAGS: 00010282
[    0.257960] RAX: ffffea0002d80028 RBX: 00000000000d0001 RCX: 0000000000000064
[    0.257960] RAX: ffffea0002d80028 RBX: 00000000000d0001 RCX: 0000000000000064
[    0.257960] RDX: ffffea0002d7fff0 RSI: ffffffff810000df RDI: ffffea0002d80000
[    0.257960] RDX: ffffea0002d7fff0 RSI: ffffffff810000df RDI: ffffea0002d80000
[    0.257960] RBP: ffff88001fc81e70 R08: 0000000000000000 R09: ffff88001fc38000
[    0.257960] RBP: ffff88001fc81e70 R08: 0000000000000000 R09: ffff88001fc38000
[    0.257960] R10: ffff88001fc81c20 R11: ffff88001fc81da0 R12: ffffea0002d80038
[    0.257960] R10: ffff88001fc81c20 R11: ffff88001fc81da0 R12: ffffea0002d80038
[    0.257960] R13: ffffffff816e1907 R14: 0000000000000000 R15: 0000000000000000
[    0.257960] R13: ffffffff816e1907 R14: 0000000000000000 R15: 0000000000000000
[    0.257960] FS:  0000000000000000(0000) GS:ffffc90000000000(0000) knlGS:0000000000000000
[    0.257960] FS:  0000000000000000(0000) GS:ffffc90000000000(0000) knlGS:0000000000000000
[    0.257960] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.257960] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.257960] CR2: ffffea0002d80028 CR3: 0000000001001000 CR4: 0000000000002660
[    0.257960] CR2: ffffea0002d80028 CR3: 0000000001001000 CR4: 0000000000002660
[    0.257960] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.257960] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.257960] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.257960] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.257960] Process swapper (pid: 1, threadinfo ffff88001fc80000, task ffff88001fc78000)
[    0.257960] Process swapper (pid: 1, threadinfo ffff88001fc80000, task ffff88001fc78000)
[    0.257960] Stack:
[    0.257960] Stack:
[    0.257960]  ffff88001fc81ea0[    0.257960]  ffff88001fc81ea0 ffffffff816e1ad0 ffffffff816e1ad0 ffff88001fc81ea0 ffff88001fc81ea0 ffffffff816e18ad ffffffff816e18ad

[    0.257960] <0>[    0.257960] <0> ffffffff817167a8 ffffffff817167a8 0000000000000040 0000000000000040 ffff88001fc81f10 ffff88001fc81f10 ffffffff8100a05c ffffffff8100a05c

[    0.257960] <0>[    0.257960] <0> ffff88001fc81e00 ffff88001fc81e00 ffffffff8108e691 ffffffff8108e691 0000000000003333 0000000000003333 ffffffff81110000 ffffffff81110000

[    0.257960] Call Trace:
[    0.257960] Call Trace:
[    0.257960]  [<ffffffff816e1ad0>] balloon_init+0x1c9/0x1fd
[    0.257960]  [<ffffffff816e1ad0>] balloon_init+0x1c9/0x1fd
[    0.257960]  [<ffffffff816e18ad>] ? xenbus_probe_frontend_init+0x53/0x59
[    0.257960]  [<ffffffff816e18ad>] ? xenbus_probe_frontend_init+0x53/0x59
[    0.257960]  [<ffffffff8100a05c>] do_one_initcall+0x56/0x136
[    0.257960]  [<ffffffff8100a05c>] do_one_initcall+0x56/0x136
[    0.257960]  [<ffffffff8108e691>] ? register_irq_proc+0xae/0xca
[    0.257960]  [<ffffffff8108e691>] ? register_irq_proc+0xae/0xca
[    0.257960]  [<ffffffff81110000>] ? elf_core_dump+0x81e/0xc29
[    0.257960]  [<ffffffff81110000>] ? elf_core_dump+0x81e/0xc29
[    0.257960]  [<ffffffff816bc9aa>] kernel_init+0x15e/0x1b6
[    0.257960]  [<ffffffff816bc9aa>] kernel_init+0x15e/0x1b6
[    0.257960]  [<ffffffff81012cea>] child_rip+0xa/0x20
[    0.257960]  [<ffffffff81012cea>] child_rip+0xa/0x20
[    0.257960]  [<ffffffff81011ea7>] ? int_ret_from_sys_call+0x7/0x1b
[    0.257960]  [<ffffffff81011ea7>] ? int_ret_from_sys_call+0x7/0x1b
[    0.257960]  [<ffffffff8101265d>] ? retint_restore_args+0x5/0x6
[    0.257960]  [<ffffffff8101265d>] ? retint_restore_args+0x5/0x6
[    0.257960]  [<ffffffff81012ce0>] ? child_rip+0x0/0x20
[    0.257960]  [<ffffffff81012ce0>] ? child_rip+0x0/0x20
[    0.257960] Code: [    0.257960] Code: 55 55 89 89 ff ff 48 48 89 89 e5 e5 e8 e8 25 25 32 32 df df ff ff c9 c9 c3 c3 55 55 48 48 89 89 e5 e5 e8 e8 5a 5a 32 32 df df ff ff c9 c9 c3 c3 48 48 8b 8b 15 15 d1 d1 0c 0c 43 43 00 00 48 48 8d 8d 47 47 28 28 55 55 48 48 89 89 e5 e5 48 48 89 89 42 42 08 08 <48> <48> 89 89 57 57 28 28 48 48 c7 c7 40 40 08 08 40 40 6c 6c 64 64 81 81 48 48 ff ff 05 05 6a 6a ac ac 60 60 00 00 48 48 ff ff 

[    0.257960] RIP [    0.257960] RIP  [<ffffffff81215f7b>] balloon_append+0x13/0x36
 [<ffffffff81215f7b>] balloon_append+0x13/0x36
[    0.257960]  RSP <ffff88001fc81e70>
[    0.257960]  RSP <ffff88001fc81e70>
[    0.257960] CR2: ffffea0002d80028
[    0.257960] CR2: ffffea0002d80028
[    0.257960] ---[ end trace 4eaa2a86a8e2da22 ]---
[    0.257960] ---[ end trace 4eaa2a86a8e2da22 ]---
<END>

Sorry for the duplicate lines. I think the combination of earlycons and console both pointing
to the Xen console driver is screwing this up.

This could be a simple bug with the max_pfn being reset to phantom value:

[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 000000000009f400 (usable)
[    0.000000]  Xen: 000000000009f400 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 00000000cfee0000 (usable)
[    0.000000]  Xen: 00000000cfee0000 - 00000000cfee3000 (ACPI NVS)
[    0.000000]  Xen: 00000000cfee3000 - 00000000cfef0000 (ACPI data)
[    0.000000]  Xen: 00000000cfef0000 - 00000000cff00000 (reserved)
[    0.000000]  Xen: 00000000e0000000 - 00000000f0000000 (reserved)
[    0.000000]  Xen: 00000000fec00000 - 0000000100000000 (reserved)
[    0.000000]  Xen: 0000000100000000 - 0000000130000000 (usable)
[    0.000000] console [xenboot0] enabled
[    0.000000] DMI 2.5 present.
[    0.000000] Phoenix BIOS detected: BIOS may corrupt low RAM, working around it.
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
[    0.000000] last_pfn = 0x130000 max_arch_pfn = 0x400000000
[    0.000000] max_pfn = 1245184			<-- Added a printk
[    0.000000] num_physpages: 1245184			<-- Added a printk
[    0.000000] Searching for max PFN under 4GB		<-- Added a printk
[    0.000000] last_pfn = 0xcfee0 max_arch_pfn = 0x400000000
[    0.000000] max_low_pfn: 851680m			<-- Added a printk
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] modified physical RAM map:
[    0.000000]  modified: 0000000000000000 - 0000000000010000 (reserved)
[    0.000000]  modified: 0000000000010000 - 000000000009f400 (usable)
[    0.000000]  modified: 000000000009f400 - 0000000000100000 (reserved)
[    0.000000]  modified: 0000000000100000 - 00000000cfee0000 (usable)
[    0.000000]  modified: 00000000cfee0000 - 00000000cfee3000 (ACPI NVS)
[    0.000000]  modified: 00000000cfee3000 - 00000000cfef0000 (ACPI data)
[    0.000000]  modified: 00000000cfef0000 - 00000000cff00000 (reserved)
[    0.000000]  modified: 00000000e0000000 - 00000000f0000000 (reserved)
[    0.000000]  modified: 00000000fec00000 - 0000000100000000 (reserved)
[    0.000000]  modified: 0000000100000000 - 0000000130000000 (usable)
[    0.000000] initial memory mapped : 0 - 20000000

The machine has 4GB, but somehow the max_pfn is set to point to 5.1GB which is
surely not right. Oh, I am compiling a 64-bit Linux and Xen.