On 9/2/2014 1:31 PM, Suravee Suthikulanit wrote:
> On 9/2/2014 1:15 PM, Stefano Stabellini wrote:
>> On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
>>> On 9/2/2014 1:06 PM, Stefano Stabellini wrote:
>>>> On Tue, 2 Sep 2014, Suravee Suthikulanit wrote:
>>>>> Hi all,
>>>>>
>>>>> I am running into an issue booting Xen on Seattle platform where dom0
>>>>> kernel
>>>>> failed to make the following hypercall in arch/arm/xen/enlighten.c:
>>>>> xen_percpu_init().
>>>>>
>>>>>    HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info, cpu, &info)
>>>>>
>>>>> After investigation, the hypercall failed in the xen/common/domain.c:
>>>>> map_vcpu_info(), where it check
>>>>>
>>>>>       if ( offset > (PAGE_SIZE - sizeof(vcpu_info_t)) )
>>>>>           return -EINVAL;
>>>>>
>>>>> Here:
>>>>>
>>>>> (XEN) DEBUG0: map_vcpu_info: offset = 0x6790
>>>>> (XEN) DEBUG0: map_vcpu_info: PAGE_SIZE = 0x1000
>>>>> (XEN) DEBUG0: map_vcpu_info: sizeof(vcpu_info_t) = 0x30
>>>>>
>>>>> If I understand correctly, it seems that the hypervisor is making
>>>>> assumption
>>>>> that the Dom0 guest is using 4K page size.  The PAGE_SHIFT in
>>>>> include/asm-arm/config.h is currently set to 12.  So, I have tried
>>>>> changing
>>>>> this to 16 (for 64K page). However, there is a restriction on the
>>>>> 16-bit
>>>>> value
>>>>> of the struct active_grant_entry { length:16; }. So, I cannot boot
>>>>> dom0
>>>>> with
>>>>> 64K page size. Is this accurate?
>>>>
>>>> Hello Suravee,
>>>> yes, that is correct: all pages used in hypercalls and shared with the
>>>> hypervisor need to be 4K pages.
>>>>
>>>
>>> Ok, the next question would be, would it make sense/possible to
>>> extend the
>>> support to allow 64K pages?
>>
>> It might make sense, but it is not trivial: all Xen hypercall interfaces
>> are built for 4K pages and would need a significant revamp.  Certainly
>> too late for the next release. It might be something to look at for Xen
>> 4.6.
>>
>> There is nothing preventing a guest kernel from using 64K pages anywhere
>> else though.
>>
> The alternative is to try booting with dom0 kernel configured with 4K
> pages. Currently, with 3.16 kernel, this is not working yet with Seattle
> since we would need level 4 page table support to boot with 4K page size.
>
> There is a patch from Catalin which should add support for level 4 page.
>   Let us experiment with that and get back with you guys.
>

Quick update:

After configure kernel w/ 4K page and Using 48-bit (w/ 4 level page 
table), Dom0 kernel seems to be coming along further at this point up 
until the following point:

.....
(XEN) DOM0: NET: Registered protocol family 10
(XEN) DOM0: sit: IPv6 over IPv4 tunneling driver
(XEN) DOM0: NET: Registered protocol family 17
(XEN) DOM0: Bridge firewalling registered
(XEN) DOM0: 9pnet: Installing 9P2000 support
(XEN) DOM0: NET: Registered protocol family 37
(XEN) DOM0: bootconsole [uart0] disabled

Please see attachment for full boot log. I am still working on trying to 
get to Dom0 console.

Suravee