From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751555AbbFHFJi (ORCPT ); Mon, 8 Jun 2015 01:09:38 -0400 Received: from cantor2.suse.de ([195.135.220.15]:48826 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750769AbbFHFJ2 (ORCPT ); Mon, 8 Jun 2015 01:09:28 -0400 Message-ID: <55752386.5010806@suse.com> Date: Mon, 08 Jun 2015 07:09:26 +0200 From: Juergen Gross User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: David Vrabel , linux-kernel@vger.kernel.org, xen-devel@lists.xensource.com, konrad.wilk@oracle.com, boris.ostrovsky@oracle.com Subject: Re: [Xen-devel] [Patch V3 14/15] xen: allow more than 512 GB of RAM for 64 bit pv-domains References: <1429507420-18201-1-git-send-email-jgross@suse.com> <1429507420-18201-15-git-send-email-jgross@suse.com> <5565EFE2.7050308@citrix.com> <5565F977.102@citrix.com> In-Reply-To: <5565F977.102@citrix.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/27/2015 07:05 PM, David Vrabel wrote: > On 27/05/15 17:25, David Vrabel wrote: >> On 20/04/15 06:23, Juergen Gross wrote: >>> 64 bit pv-domains under Xen are limited to 512 GB of RAM today. The >>> main reason has been the 3 level p2m tree, which was replaced by the >>> virtual mapped linear p2m list. Parallel to the p2m list which is >>> being used by the kernel itself there is a 3 level mfn tree for usage >>> by the Xen tools and eventually for crash dump analysis. For this tree >>> the linear p2m list can serve as a replacement, too. As the kernel >>> can't know whether the tools are capable of dealing with the p2m list >>> instead of the mfn tree, the limit of 512 GB can't be dropped in all >>> cases. >>> >>> This patch replaces the hard limit by a kernel parameter which tells >>> the kernel to obey the 512 GB limit or not. The default is selected by >>> a configuration parameter which specifies whether the 512 GB limit >>> should be active per default for domUs (domain save/restore/migration >>> and crash dump analysis are affected). >>> >>> Memory above the domain limit is returned to the hypervisor instead of >>> being identity mapped, which was wrong anyway. >>> >>> The kernel configuration parameter to specify the maximum size of a >>> domain can be deleted, as it is not relevant any more. >> >> Something in this patch breaks the hvc console in my test domU. >> >> kernel BUG at /local/davidvr/work/k.org/tip/drivers/tty/hvc/hvc_xen.c:153 >> >> Which suggests the hvc driver mapped the wrong console ring frame. > > Sorry, it's patch #13 (xen: move p2m list if conflicting with e820 map) > that seems to be bad. I think I've found the reason: the console frame isn't being marked as "reserved" any more. With moving the p2m list I had to change the call of memblock_reserve() for it. Before that patch this call covered the p2m list, the start_info page, the xenstore page and the console page. I added a memblock_reserve() for start_info, but failed to do so for xenstore and console. I'll modify the patch and respin. I have to check why I didn't hit this issue. Maybe my test machine was too large and the memory in question didn't get reused until my test was finished. Thanks for testing, Juergen