dom0 boot failure with 256G

[Mukesh Rathor <mukesh.rathor@oracle.com>]

Hi,

    I've been intensely debugging a dom0 crash right at boot in
    pagetable_init() with 256G. We are running 64bit hyp with 32bit dom0.
    This on xen 3.1.3, which seems to have fixes from changeset 16548.

    It boils down to this:

   crash occurs in dom0: mfn_to_pfn() during machine_to_phys_mapping[mfn]
   lookup  where mfn == 0x3f2ec09 is too big. The table starts at: 0xf5800000
   which I see comes from MACH2PHYS_VIRT_START.

   Trying to figure the hyp out, I notice phys addr is set as:

   d->arch.physaddr_bitsize =
           fls((1UL << 32) - HYPERVISOR_COMPAT_VIRT_START(d)) - 1
             + (PAGE_SIZE - 2);
   which is set to 0x1019, totally baffling me. I'd expect it to be 32,
   36, or 64????????

  moving along, in alloc_domheap_pages() :
    bits = domain_clamp_alloc_bitsize(d, bits ? : (BITS_PER_LONG+PAGE_SHIFT));

   domain_clamp_alloc_bitsize(struct domain *d, unsigned int bits)
   {
       if ( (d == NULL) || !is_pv_32on64_domain(d) )
               return bits;
       return min(d->arch.physaddr_bitsize, bits);
            ---> here : physaddr: 0x1019, bits:76 ?????
            ---> i'm thinking bits should be 36+12 == 48 ??
   }

    finally, call to alloc_heap_pages(...) returns frames too high, pg
    ptr is returned at : ffff82849df30000 (zone_hi == 0x27)


    On a side note, is my understanding correct that while a guest pfn must
    always be < 36 bits for a PAE 32 guest, the mfn doesn't have to be?


    thanks
    mukesh