Re: [PATCH V3 1/1] expand x86 arch_shared_info to support >3 level p2m tree

[Juergen Gross <jgross@suse.com>]

On 09/16/2014 12:14 PM, David Vrabel wrote:
> On 16/09/14 04:52, Juergen Gross wrote:
>> On 09/15/2014 04:30 PM, David Vrabel wrote:
>>> On 15/09/14 11:46, Juergen Gross wrote:
>>>> So you'd prefer:
>>>>
>>>> 1) >512GB pv-domains (including Dom0) will be supported only with new
>>>>      Xen (4.6?), no matter if the user requires migration to be supported
>>>
>>> Yes.  >512 GiB and not being able to migrate are not obviously related
>>> from the point of view of the end user (unlike assigning a PCI device).
>>>
>>> Failing at domain save time is most likely too late for the end user.
>>
>> What would you think about following compromise:
>>
>> We add a flag that indicates support of multi-level p2m. Additionally
>> the Linux kernel can ignore the flag not being set either if started as
>> Dom0 or if told so via kernel parameter.
>
> This sounds fine but this override should be via the command line
> parameter only.  Crash dump analysis tools may not understand the 4
> level p2m.
>
>>>> to:
>>>>
>>>> 2) >512GB pv-domains (especially Dom0 and VMs with direct hw access) can
>>>>      be started on current Xen versions, migration is possible only if
>>>> Xen
>>>>      is new (4.6?)
>>>
>>> There's also my preferred option:
>>>
>>> 3) >512 GiB PV domains are not supported.  Large guests must be PVH or
>>> PVHVM.
>>
>> In theory okay, but not right now, I think. PVH Dom0 is not production
>> ready.
>
> I'm not really seeing the need for such a large dom0.

Okay, then I'd come back to V1 of my patches. This is the minimum
required to be able to boot up a system with Xen and more than 512GB
memory without having to reduce the Dom0 memory via Xen boot parameter.

Otherwise the hypervisor built mfn_list mapped into the initial address
space will be too large.

And no, I don't think setting the boot parameter is the solution here.
Dom0 should be usable on a huge machine without special parameters.

>
> I remain unconvinced that there are sufficient use cases to justify
> extending the PV only ABI and increasing complexity of the current
> 3-level p2m code.
>
> I'm concerned that 4-level p2m support will impact the performance of
> guests that do not need the 4 levels.  It may be necessary to use the
> alternatives mechanism to select the correct low-level lookup function.

I'll try to get some numbers to post together with a patch.

> I also think a flat array for the p2m might be better (less complex).
> There's plenty of virtual address space in a 64-bit guest to allow for this.

Hmm, do you think we could reserve an area of many GBs for Xen in
virtual space? I suspect this would be rejected as another "Xen-ism".

BTW: the mfn_list_list will still be required to be built as a tree.


Juergen