* Cannot start up HVM guest when maxmem is not equal to memory and HAP is enabled.
@ 2012-01-11 8:03 Yufang Zhang
2012-01-11 11:27 ` George Dunlap
0 siblings, 1 reply; 4+ messages in thread
From: Yufang Zhang @ 2012-01-11 8:03 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 3320 bytes --]
Hi all,
On my production deployment with xen-3.4.4-rc1, HVM guest would not start
up when maxmem is set not equal to memory, unless HAP is disable(hap=0).
The guest just hangs at 'Booting from Hard Disk...' as shown in the
attachment. I find that both Oracle VM and Fedora have the same problem per
the following links:
http://docs.oracle.com/cd/E15458_01/doc.22/e15443/toc.htm#BABBIBFD
http://docs.oracle.com/cd/E26996_01/e18546/CJAGDAGB.html#id419003
https://bugzilla.redhat.com/show_bug.cgi?id=645319
Could I ask if there is any work in upstream/xen-unstable to fix this
problem?
Thanks.
Below is the xm dmesg output related with the hvm guest:
(XEN) HVM3: HVM Loader
(XEN) HVM3: Detected Xen v3.4.4-rc1-pre
(XEN) HVM3: CPU speed is 2267 MHz
(XEN) irq.c:243: Dom3 PCI link 0 changed 0 -> 5
(XEN) HVM3: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom3 PCI link 1 changed 0 -> 10
(XEN) HVM3: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom3 PCI link 2 changed 0 -> 11
(XEN) HVM3: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom3 PCI link 3 changed 0 -> 5
(XEN) HVM3: PCI-ISA link 3 routed to IRQ5
(XEN) HVM3: pci dev 01:2 INTD->IRQ5
(XEN) HVM3: pci dev 01:3 INTA->IRQ10
(XEN) HVM3: pci dev 03:0 INTA->IRQ5
(XEN) HVM3: pci dev 04:0 INTA->IRQ5
(XEN) HVM3: pci dev 02:0 bar 10 size 02000000: f0000008
(XEN) HVM3: pci dev 03:0 bar 14 size 01000000: f2000008
(XEN) HVM3: pci dev 02:0 bar 14 size 00001000: f3000000
(XEN) HVM3: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) HVM3: pci dev 04:0 bar 10 size 00000100: 0000c101
(XEN) HVM3: pci dev 04:0 bar 14 size 00000100: f3001000
(XEN) HVM3: pci dev 01:2 bar 20 size 00000020: 0000c201
(XEN) HVM3: pci dev 01:1 bar 20 size 00000010: 0000c221
(XEN) HVM3: Multiprocessor initialisation:
(XEN) HVM3: - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ...
done.
(XEN) HVM3: Writing SMBIOS tables ...
(XEN) HVM3: Loading ROMBIOS ...
(XEN) HVM3: 10332 bytes of ROMBIOS high-memory extensions:
(XEN) HVM3: Relocating to 0xfc000000-0xfc00285c ... done
(XEN) HVM3: Creating MP tables ...
(XEN) HVM3: Loading Cirrus VGABIOS ...
(XEN) HVM3: Loading PCI Option ROM ...
(XEN) HVM3: - Manufacturer: http://etherboot.org
(XEN) HVM3: - Product name: gPXE
(XEN) HVM3: Loading ACPI ...
(XEN) HVM3: - Lo data: 000ea020-000ea04f
(XEN) HVM3: - Hi data: fc002c00-fc0060bf
(XEN) HVM3: vm86 TSS at fc006400
(XEN) HVM3: BIOS map:
(XEN) HVM3: c0000-c8fff: VGA BIOS
(XEN) HVM3: c9000-d57ff: Etherboot ROM
(XEN) HVM3: eb000-eb150: SMBIOS tables
(XEN) HVM3: f0000-fffff: Main BIOS
(XEN) HVM3: Invoking ROMBIOS ...
(XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d3 entering stdvga and caching modes
(XEN) HVM3: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM3: Bochs BIOS - build: 06/23/99
(XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM3: Options: apmbios pcibios eltorito PMM
(XEN) HVM3:
(XEN) HVM3: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM3: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (8192 MBytes)
(XEN) HVM3: IDE time out
(XEN) HVM3:
(XEN) HVM3:
(XEN) HVM3:
(XEN) HVM3: Press F12 for boot menu.
(XEN) HVM3:
(XEN) HVM3: Booting from Hard Disk...
(XEN) HVM3: Booting from 0000:7c00
(XEN) io.c:199:d3 MMIO emulation failed @ 0008:4013b1: 90 86 95 21 08 83
[-- Attachment #1.2: Type: text/html, Size: 4426 bytes --]
[-- Attachment #2: guest_hang_screen.png --]
[-- Type: image/png, Size: 20690 bytes --]
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Cannot start up HVM guest when maxmem is not equal to memory and HAP is enabled.
2012-01-11 8:03 Cannot start up HVM guest when maxmem is not equal to memory and HAP is enabled Yufang Zhang
@ 2012-01-11 11:27 ` George Dunlap
2012-01-12 9:15 ` Yufang Zhang
0 siblings, 1 reply; 4+ messages in thread
From: George Dunlap @ 2012-01-11 11:27 UTC (permalink / raw)
To: Yufang Zhang; +Cc: xen-devel
On Wed, Jan 11, 2012 at 8:03 AM, Yufang Zhang <yufang521247@gmail.com> wrote:
> Hi all,
>
> On my production deployment with xen-3.4.4-rc1, HVM guest would not start up
> when maxmem is set not equal to memory, unless HAP is disable(hap=0). The
> guest just hangs at 'Booting from Hard Disk...' as shown in the attachment.
Is there a reason you want to set maxmem != memory? Starting in 4.0,
this will cause the VM to boot in "populate-on-demand" mode; but I
think in 3.4, this may have undefined behavior.
> I find that both Oracle VM and Fedora have the same problem per the
> following links:
>
> http://docs.oracle.com/cd/E15458_01/doc.22/e15443/toc.htm#BABBIBFD
> http://docs.oracle.com/cd/E26996_01/e18546/CJAGDAGB.html#id419003
> https://bugzilla.redhat.com/show_bug.cgi?id=645319
>
> Could I ask if there is any work in upstream/xen-unstable to fix this
> problem?
Setting maxmem != memory in 4.1 will definitely have a different
effect -- if your guest has a balloon driver, you will start your VM
"pre-ballooned"; if your guest does not, it will crash as soon as it
dirties more than "memory" amount of memory.
-George
>
> Thanks.
>
> Below is the xm dmesg output related with the hvm guest:
> (XEN) HVM3: HVM Loader
> (XEN) HVM3: Detected Xen v3.4.4-rc1-pre
> (XEN) HVM3: CPU speed is 2267 MHz
> (XEN) irq.c:243: Dom3 PCI link 0 changed 0 -> 5
> (XEN) HVM3: PCI-ISA link 0 routed to IRQ5
> (XEN) irq.c:243: Dom3 PCI link 1 changed 0 -> 10
> (XEN) HVM3: PCI-ISA link 1 routed to IRQ10
> (XEN) irq.c:243: Dom3 PCI link 2 changed 0 -> 11
> (XEN) HVM3: PCI-ISA link 2 routed to IRQ11
> (XEN) irq.c:243: Dom3 PCI link 3 changed 0 -> 5
> (XEN) HVM3: PCI-ISA link 3 routed to IRQ5
> (XEN) HVM3: pci dev 01:2 INTD->IRQ5
> (XEN) HVM3: pci dev 01:3 INTA->IRQ10
> (XEN) HVM3: pci dev 03:0 INTA->IRQ5
> (XEN) HVM3: pci dev 04:0 INTA->IRQ5
> (XEN) HVM3: pci dev 02:0 bar 10 size 02000000: f0000008
> (XEN) HVM3: pci dev 03:0 bar 14 size 01000000: f2000008
> (XEN) HVM3: pci dev 02:0 bar 14 size 00001000: f3000000
> (XEN) HVM3: pci dev 03:0 bar 10 size 00000100: 0000c001
> (XEN) HVM3: pci dev 04:0 bar 10 size 00000100: 0000c101
> (XEN) HVM3: pci dev 04:0 bar 14 size 00000100: f3001000
> (XEN) HVM3: pci dev 01:2 bar 20 size 00000020: 0000c201
> (XEN) HVM3: pci dev 01:1 bar 20 size 00000010: 0000c221
> (XEN) HVM3: Multiprocessor initialisation:
> (XEN) HVM3: - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8] ...
> done.
> (XEN) HVM3: Writing SMBIOS tables ...
> (XEN) HVM3: Loading ROMBIOS ...
> (XEN) HVM3: 10332 bytes of ROMBIOS high-memory extensions:
> (XEN) HVM3: Relocating to 0xfc000000-0xfc00285c ... done
> (XEN) HVM3: Creating MP tables ...
> (XEN) HVM3: Loading Cirrus VGABIOS ...
> (XEN) HVM3: Loading PCI Option ROM ...
> (XEN) HVM3: - Manufacturer: http://etherboot.org
> (XEN) HVM3: - Product name: gPXE
> (XEN) HVM3: Loading ACPI ...
> (XEN) HVM3: - Lo data: 000ea020-000ea04f
> (XEN) HVM3: - Hi data: fc002c00-fc0060bf
> (XEN) HVM3: vm86 TSS at fc006400
> (XEN) HVM3: BIOS map:
> (XEN) HVM3: c0000-c8fff: VGA BIOS
> (XEN) HVM3: c9000-d57ff: Etherboot ROM
> (XEN) HVM3: eb000-eb150: SMBIOS tables
> (XEN) HVM3: f0000-fffff: Main BIOS
> (XEN) HVM3: Invoking ROMBIOS ...
> (XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) stdvga.c:147:d3 entering stdvga and caching modes
> (XEN) HVM3: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
> (XEN) HVM3: Bochs BIOS - build: 06/23/99
> (XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> (XEN) HVM3: Options: apmbios pcibios eltorito PMM
> (XEN) HVM3:
> (XEN) HVM3: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> (XEN) HVM3: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (8192 MBytes)
> (XEN) HVM3: IDE time out
> (XEN) HVM3:
> (XEN) HVM3:
> (XEN) HVM3:
> (XEN) HVM3: Press F12 for boot menu.
> (XEN) HVM3:
> (XEN) HVM3: Booting from Hard Disk...
> (XEN) HVM3: Booting from 0000:7c00
> (XEN) io.c:199:d3 MMIO emulation failed @ 0008:4013b1: 90 86 95 21 08 83
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Cannot start up HVM guest when maxmem is not equal to memory and HAP is enabled.
2012-01-11 11:27 ` George Dunlap
@ 2012-01-12 9:15 ` Yufang Zhang
2012-01-13 17:19 ` George Dunlap
0 siblings, 1 reply; 4+ messages in thread
From: Yufang Zhang @ 2012-01-12 9:15 UTC (permalink / raw)
To: George Dunlap; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 5195 bytes --]
2012/1/11 George Dunlap <George.Dunlap@eu.citrix.com>
> On Wed, Jan 11, 2012 at 8:03 AM, Yufang Zhang <yufang521247@gmail.com>
> wrote:
> > Hi all,
> >
> > On my production deployment with xen-3.4.4-rc1, HVM guest would not
> start up
> > when maxmem is set not equal to memory, unless HAP is disable(hap=0). The
> > guest just hangs at 'Booting from Hard Disk...' as shown in the
> attachment.
>
> Is there a reason you want to set maxmem != memory? Starting in 4.0,
> this will cause the VM to boot in "populate-on-demand" mode; but I
> think in 3.4, this may have undefined behavior.
>
>
Actually, you would be exposed to this problem whenever you restart xend
and then reboot the hvm guest, even you set maxmem == memory in the config
file of that guest. When xend is restarted, it tries to recreate guest
information. memory_dynamic_max and memory_static_max, which are used
to calculate memory and maxmem, are re-calculated from mem_kb and maxmem_kb
respectively. mem_kb is not equal to maxmem_kb. Thus memory != maxmem
after xend restarts, and rebooting this hvm guest later would leads guest
crash/hang.
> I find that both Oracle VM and Fedora have the same problem per the
> > following links:
> >
> > http://docs.oracle.com/cd/E15458_01/doc.22/e15443/toc.htm#BABBIBFD
> > http://docs.oracle.com/cd/E26996_01/e18546/CJAGDAGB.html#id419003
> > https://bugzilla.redhat.com/show_bug.cgi?id=645319
> >
> > Could I ask if there is any work in upstream/xen-unstable to fix this
> > problem?
>
> Setting maxmem != memory in 4.1 will definitely have a different
> effect -- if your guest has a balloon driver, you will start your VM
> "pre-ballooned"; if your guest does not, it will crash as soon as it
> dirties more than "memory" amount of memory.
>
> -George
I check the xend code of xen-unstable, the related logic didn't change
since xen-3.4. So I *guess* a hvm guest without balloon driver would crash
after some time(when it dirties enough memory) after you restarting xend
and rebooting the guest on xen-4.1.
>
> > Thanks.
> >
> > Below is the xm dmesg output related with the hvm guest:
> > (XEN) HVM3: HVM Loader
> > (XEN) HVM3: Detected Xen v3.4.4-rc1-pre
> > (XEN) HVM3: CPU speed is 2267 MHz
> > (XEN) irq.c:243: Dom3 PCI link 0 changed 0 -> 5
> > (XEN) HVM3: PCI-ISA link 0 routed to IRQ5
> > (XEN) irq.c:243: Dom3 PCI link 1 changed 0 -> 10
> > (XEN) HVM3: PCI-ISA link 1 routed to IRQ10
> > (XEN) irq.c:243: Dom3 PCI link 2 changed 0 -> 11
> > (XEN) HVM3: PCI-ISA link 2 routed to IRQ11
> > (XEN) irq.c:243: Dom3 PCI link 3 changed 0 -> 5
> > (XEN) HVM3: PCI-ISA link 3 routed to IRQ5
> > (XEN) HVM3: pci dev 01:2 INTD->IRQ5
> > (XEN) HVM3: pci dev 01:3 INTA->IRQ10
> > (XEN) HVM3: pci dev 03:0 INTA->IRQ5
> > (XEN) HVM3: pci dev 04:0 INTA->IRQ5
> > (XEN) HVM3: pci dev 02:0 bar 10 size 02000000: f0000008
> > (XEN) HVM3: pci dev 03:0 bar 14 size 01000000: f2000008
> > (XEN) HVM3: pci dev 02:0 bar 14 size 00001000: f3000000
> > (XEN) HVM3: pci dev 03:0 bar 10 size 00000100: 0000c001
> > (XEN) HVM3: pci dev 04:0 bar 10 size 00000100: 0000c101
> > (XEN) HVM3: pci dev 04:0 bar 14 size 00000100: f3001000
> > (XEN) HVM3: pci dev 01:2 bar 20 size 00000020: 0000c201
> > (XEN) HVM3: pci dev 01:1 bar 20 size 00000010: 0000c221
> > (XEN) HVM3: Multiprocessor initialisation:
> > (XEN) HVM3: - CPU0 ... 40-bit phys ... fixed MTRRs ... var MTRRs [2/8]
> ...
> > done.
> > (XEN) HVM3: Writing SMBIOS tables ...
> > (XEN) HVM3: Loading ROMBIOS ...
> > (XEN) HVM3: 10332 bytes of ROMBIOS high-memory extensions:
> > (XEN) HVM3: Relocating to 0xfc000000-0xfc00285c ... done
> > (XEN) HVM3: Creating MP tables ...
> > (XEN) HVM3: Loading Cirrus VGABIOS ...
> > (XEN) HVM3: Loading PCI Option ROM ...
> > (XEN) HVM3: - Manufacturer: http://etherboot.org
> > (XEN) HVM3: - Product name: gPXE
> > (XEN) HVM3: Loading ACPI ...
> > (XEN) HVM3: - Lo data: 000ea020-000ea04f
> > (XEN) HVM3: - Hi data: fc002c00-fc0060bf
> > (XEN) HVM3: vm86 TSS at fc006400
> > (XEN) HVM3: BIOS map:
> > (XEN) HVM3: c0000-c8fff: VGA BIOS
> > (XEN) HVM3: c9000-d57ff: Etherboot ROM
> > (XEN) HVM3: eb000-eb150: SMBIOS tables
> > (XEN) HVM3: f0000-fffff: Main BIOS
> > (XEN) HVM3: Invoking ROMBIOS ...
> > (XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> > (XEN) stdvga.c:147:d3 entering stdvga and caching modes
> > (XEN) HVM3: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert
> Exp $
> > (XEN) HVM3: Bochs BIOS - build: 06/23/99
> > (XEN) HVM3: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
> > (XEN) HVM3: Options: apmbios pcibios eltorito PMM
> > (XEN) HVM3:
> > (XEN) HVM3: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
> > (XEN) HVM3: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (8192 MBytes)
> > (XEN) HVM3: IDE time out
> > (XEN) HVM3:
> > (XEN) HVM3:
> > (XEN) HVM3:
> > (XEN) HVM3: Press F12 for boot menu.
> > (XEN) HVM3:
> > (XEN) HVM3: Booting from Hard Disk...
> > (XEN) HVM3: Booting from 0000:7c00
> > (XEN) io.c:199:d3 MMIO emulation failed @ 0008:4013b1: 90 86 95 21 08 83
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
>
[-- Attachment #1.2: Type: text/html, Size: 6967 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Cannot start up HVM guest when maxmem is not equal to memory and HAP is enabled.
2012-01-12 9:15 ` Yufang Zhang
@ 2012-01-13 17:19 ` George Dunlap
0 siblings, 0 replies; 4+ messages in thread
From: George Dunlap @ 2012-01-13 17:19 UTC (permalink / raw)
To: Yufang Zhang; +Cc: xen-devel, Keith Coleman
On Thu, Jan 12, 2012 at 9:15 AM, Yufang Zhang <yufang521247@gmail.com> wrote:
> Actually, you would be exposed to this problem whenever you restart xend and
> then reboot the hvm guest, even you set maxmem == memory in the config file
> of that guest. When xend is restarted, it tries to recreate guest
> information. memory_dynamic_max and memory_static_max, which are used
> to calculate memory and maxmem, are re-calculated from mem_kb and maxmem_kb
> respectively. mem_kb is not equal to maxmem_kb. Thus memory != maxmem after
> xend restarts, and rebooting this hvm guest later would leads guest
> crash/hang.
Ah, interesting. Yeah, that is certainly a bug -- not sure whom to
contact about fixing that. Keith?
> I check the xend code of xen-unstable, the related logic didn't change
> since xen-3.4. So I *guess* a hvm guest without balloon driver would crash
> after some time(when it dirties enough memory) after you restarting xend and
> rebooting the guest on xen-4.1.
Should a user be changing the "memory" setting without having a
balloon driver active? If the balloon driver *is* active, we'd want
that setting to be preserved across reboot. Does xend know if there
is an active balloon driver? If not, I think we just have to leave it
as it is.
-George
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2012-01-13 17:19 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-01-11 8:03 Cannot start up HVM guest when maxmem is not equal to memory and HAP is enabled Yufang Zhang
2012-01-11 11:27 ` George Dunlap
2012-01-12 9:15 ` Yufang Zhang
2012-01-13 17:19 ` George Dunlap
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).