Re: [Xen-devel] [Patch V3 14/15] xen: allow more than 512 GB of RAM for 64 bit pv-domains

[Juergen Gross <jgross@suse.com>]

On 05/27/2015 07:05 PM, David Vrabel wrote:
> On 27/05/15 17:25, David Vrabel wrote:
>> On 20/04/15 06:23, Juergen Gross wrote:
>>> 64 bit pv-domains under Xen are limited to 512 GB of RAM today. The
>>> main reason has been the 3 level p2m tree, which was replaced by the
>>> virtual mapped linear p2m list. Parallel to the p2m list which is
>>> being used by the kernel itself there is a 3 level mfn tree for usage
>>> by the Xen tools and eventually for crash dump analysis. For this tree
>>> the linear p2m list can serve as a replacement, too. As the kernel
>>> can't know whether the tools are capable of dealing with the p2m list
>>> instead of the mfn tree, the limit of 512 GB can't be dropped in all
>>> cases.
>>>
>>> This patch replaces the hard limit by a kernel parameter which tells
>>> the kernel to obey the 512 GB limit or not. The default is selected by
>>> a configuration parameter which specifies whether the 512 GB limit
>>> should be active per default for domUs (domain save/restore/migration
>>> and crash dump analysis are affected).
>>>
>>> Memory above the domain limit is returned to the hypervisor instead of
>>> being identity mapped, which was wrong anyway.
>>>
>>> The kernel configuration parameter to specify the maximum size of a
>>> domain can be deleted, as it is not relevant any more.
>>
>> Something in this patch breaks the hvc console in my test domU.
>>
>> kernel BUG at /local/davidvr/work/k.org/tip/drivers/tty/hvc/hvc_xen.c:153
>>
>> Which suggests the hvc driver mapped the wrong console ring frame.
>
> Sorry, it's patch #13 (xen: move p2m list if conflicting with e820 map)
> that seems to be bad.

I think I've found the reason: the console frame isn't being marked as
"reserved" any more. With moving the p2m list I had to change the call
of memblock_reserve() for it. Before that patch this call covered the
p2m list, the start_info page, the xenstore page and the console page.
I added a memblock_reserve() for start_info, but failed to do so for
xenstore and console.

I'll modify the patch and respin.

I have to check why I didn't hit this issue. Maybe my test machine was
too large and the memory in question didn't get reused until my test
was finished.


Thanks for testing,

Juergen