From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mukesh Rathor Subject: bring up Hypervisor on large (512GB) memory Date: Mon, 09 Feb 2009 19:08:55 -0800 Message-ID: <4990EFC7.9000907@oracle.com> Reply-To: mukesh.rathor@oracle.com Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Hi all, Trying to bring up the hypervisor on 512GB : 1. Started with xen 3.1.4 (last oracle release), and 512GB, the system panic'd right away: (XEN) Early fatal page fault at e008:ffff828c801ce0ad (cr2=ffff8300cfc00000, ec=0002) I noticed an anomaly here in the RAM map: (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f400 (usable) (XEN) 000000000009f400 - 00000000000a0000 (reserved) (XEN) 00000000000f0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000cfd4c000 (usable) (XEN) 00000000cfd4c000 - 00000000cfd56000 (ACPI data) (XEN) 00000000cfd56000 - 00000000cfd57000 (usable) (XEN) 00000000cfd57000 - 00000000e0000000 (reserved) (XEN) 00000000fec00000 - 00000000fed00000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffc00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 000000802ffff000 (usable) <------- Seems that should be 0000008000000000 ??? 2. Reduce some RAM, and booting with 440GB, map makes more sense: (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009f400 (usable) (XEN) 000000000009f400 - 00000000000a0000 (reserved) (XEN) 00000000000f0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000cfd4c000 (usable) (XEN) 00000000cfd4c000 - 00000000cfd56000 (ACPI data) (XEN) 00000000cfd56000 - 00000000cfd57000 (usable) (XEN) 00000000cfd57000 - 00000000e0000000 (reserved) (XEN) 00000000fec00000 - 00000000fed00000 (reserved) (XEN) 00000000fee00000 - 00000000fee10000 (reserved) (XEN) 00000000ffc00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000006e00000000 (usable) Panic again: (XEN) Early fatal page fault at e008:ffff828c80210460 (cr2=ffff8300cfc00000, ec=0002) The panic is trying to allocate bitmap in init_boot_allocator(). The bitmap starts at cfac0000 and given the size dc1000, won't fit. 3. Moving to unstable 19164, looks like things are even more tighter!! I couldn't even boot with 64GB. bitmap_start:cfac0000, bitmap_size:201000, alloc_bitmap:ffff8300cfac0000 The only solution I can think of is moving the bitmap elsewhere, above 4GB in this case: figure the size of bitmap, DIRECT map space, allocate the map, mark it reserved in the RAM map, and should work! I'd have add a loop around init_boot_allocator() in __start_xen() iterating thru the RAM map again, and finding space above 16M. Am I on the right track? Thanks, Mukesh