bring up Hypervisor on large (512GB) memory

[Mukesh Rathor <mukesh.rathor@oracle.com>]

Hi all,

Trying to bring up the hypervisor on 512GB :

1. Started with xen 3.1.4 (last oracle release), and 512GB, the system
panic'd right away:
    (XEN) Early fatal page fault at e008:ffff828c801ce0ad
    (cr2=ffff8300cfc00000, ec=0002)

I noticed an anomaly here in the RAM map:
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009f400 (usable)
(XEN)  000000000009f400 - 00000000000a0000 (reserved)
(XEN)  00000000000f0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000cfd4c000 (usable)
(XEN)  00000000cfd4c000 - 00000000cfd56000 (ACPI data)
(XEN)  00000000cfd56000 - 00000000cfd57000 (usable)
(XEN)  00000000cfd57000 - 00000000e0000000 (reserved)
(XEN)  00000000fec00000 - 00000000fed00000 (reserved)
(XEN)  00000000fee00000 - 00000000fee10000 (reserved)
(XEN)  00000000ffc00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000802ffff000 (usable)  <-------

Seems that should be 0000008000000000 ???


2. Reduce some RAM, and booting with 440GB, map makes more sense:

(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009f400 (usable)
(XEN)  000000000009f400 - 00000000000a0000 (reserved)
(XEN)  00000000000f0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000cfd4c000 (usable)
(XEN)  00000000cfd4c000 - 00000000cfd56000 (ACPI data)
(XEN)  00000000cfd56000 - 00000000cfd57000 (usable)
(XEN)  00000000cfd57000 - 00000000e0000000 (reserved)
(XEN)  00000000fec00000 - 00000000fed00000 (reserved)
(XEN)  00000000fee00000 - 00000000fee10000 (reserved)
(XEN)  00000000ffc00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000006e00000000 (usable)

Panic again:
(XEN) Early fatal page fault at e008:ffff828c80210460
(cr2=ffff8300cfc00000, ec=0002)


The panic is trying to allocate bitmap in init_boot_allocator(). The bitmap
starts at cfac0000 and given the size dc1000, won't fit.


3. Moving to unstable 19164, looks like things are even more tighter!! I
    couldn't even boot with 64GB. bitmap_start:cfac0000, bitmap_size:201000,
    alloc_bitmap:ffff8300cfac0000


The only solution I can think of is moving the bitmap elsewhere, above 4GB in
this case:
    figure the size of bitmap, DIRECT map space, allocate the map,
    mark it reserved in the RAM map, and should work!

    I'd have add a loop around init_boot_allocator() in __start_xen()
    iterating thru the RAM map again, and finding space above 16M.


Am I on the right track?


Thanks,
Mukesh