Re: [PATCH] x86/NUMA: make init_node_heap() respect Xen heap limit

[Julien Grall <julien.grall@citrix.com>]

Hi Jan,

On 04/09/15 10:09, Jan Beulich wrote:
>> "create_xen_entries: L2 failed" tells me, through code inspection rather
>> than usefulness of the logging, that alloc_xenheap_page has returned NULL.
>>
>> I think this is simply because all RAM on Mustang is at physical address
>> 128GB onwards or so, IOW the off by one error has resulted in the xenheap
>> appearing to be empty since all RAM above the limit.

Correct, the RAM bank is:

RAM: 0000004000000000 - 00000043ffffffff

As we expose only 38 bits now, it will hide all the RAM.

> Oh, I see - I made the (x86-biased, and hence wrong) assumption
> that with Julien mentioning only the ending MFN, the start of RAM
> would be at 0.

Sorry, I forgot to mention it in my mail. On ARM, RAM
are divided in memory banks which can start anywhere in the memory. When early printk
is enabled the RAM banks are one of the first things printed in the serial console.

Anyway, I tried your suggestion to dropped the call to xenheap_max_mfn when Xen
setups the xenheap for arm64 and it allows me to boot correctly Xen on X-gene.
See patch below, I can send the patch in a separate thread if necessary.

commit b11ab8e4982228d7944e11010f5b8eec890caf30
Author: Julien Grall <julien.grall@citrix.com>
Date:   Thu Sep 3 21:49:31 2015 +0100

    xen: pagealloc: Correctly calculate the number of xenheap bits
    
    The commit 88e3ed61642bb393458acc7a9bd2f96edc337190 "x86/NUMA: make
    init_node_heap() respect Xen heap limit" breaks boot on the arm64 board
    X-Gene.
    
    The xenheap bits variable is used to know the last RAM MFN always mapped
    in Xen virtual memory. If the value is 0, it means that all the memory is
    always mapped in Xen virtual memory.
    
    On X-gene the RAM bank resides above 128GB and last xenheap MFN is
    0x4400000. With the new way to calculate the number of bits, xenheap_bits
    will be equal to 38 bits. This will result to hide all the RAM and the
    impossibility to allocate xenheap memory.
    
    Given that aarch64 have always all the memory mapped in Xen virtual
    memory, it's not necessary to call xenheap_max_mfn which set the number
    of bits.
    
    Suggested-by: Jan Beulich <jbeulich@suse.com>
    Signed-off-by: Julien Grall <julien.grall@citrix.com>

diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 6626eba..48f734f 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -665,7 +665,6 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size)
     xenheap_virt_end = XENHEAP_VIRT_START + ram_end - ram_start;
     xenheap_mfn_start = ram_start >> PAGE_SHIFT;
     xenheap_mfn_end = ram_end >> PAGE_SHIFT;
-    xenheap_max_mfn(xenheap_mfn_end);
 
     /*
      * Need enough mapped pages for copying the DTB.
-- 
Julien Grall