From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42241) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XwA9C-0003UE-Pl for qemu-devel@nongnu.org; Wed, 03 Dec 2014 08:39:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XwA96-0004X5-Jj for qemu-devel@nongnu.org; Wed, 03 Dec 2014 08:39:14 -0500 Received: from fldsmtpe04.verizon.com ([140.108.26.143]:20685) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XwA96-0004Wr-0t for qemu-devel@nongnu.org; Wed, 03 Dec 2014 08:39:08 -0500 From: Don Slutz Message-ID: <547F1274.6090607@terremark.com> Date: Wed, 03 Dec 2014 08:39:00 -0500 MIME-Version: 1.0 References: <5474C96A.6090506@citrix.com> <54768F7F.2030602@terremark.com> <547C6E12.50502@terremark.com> <547CF6C6.4060106@terremark.com> <547E1FC1.70004@terremark.com> <20141203105458.GA4208@zion.uk.xensource.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [Xen-devel] [PATCH] increase maxmem before calling xc_domain_populate_physmap List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefano Stabellini Cc: xen-devel@lists.xensource.com, Wei Liu , Ian Campbell , Andrew Cooper , qemu-devel@nongnu.org, Don Slutz On 12/03/14 07:20, Stefano Stabellini wrote: > On Wed, 3 Dec 2014, Wei Liu wrote: >> On Tue, Dec 02, 2014 at 03:23:29PM -0500, Don Slutz wrote: >> [...] >>>>>>> hw_error("xc_domain_getinfo failed"); >>>>>>> } >>>>>>> - if (xc_domain_setmaxmem(xen_xc, xen_domid, info.max_memkb + >>>>>>> - (nr_pfn * XC_PAGE_SIZE / 1024)) < 0) { >>>>>>> + max_pages = info.max_memkb * 1024 / XC_PAGE_SIZE; >>>>>>> + free_pages = max_pages - info.nr_pages; >>>>>>> + real_free = free_pages; >>>>>>> + if (free_pages > VGA_HOLE_SIZE) { >>>>>>> + free_pages -= VGA_HOLE_SIZE; >>>>>>> + } else { >>>>>>> + free_pages = 0; >>>>>>> + } >>>> I don't think we need to subtract VGA_HOLE_SIZE. >>> If you do not use some slack (like 32 here), xen does report: >>> >>> >>> (d5) [2014-11-29 17:07:21] Loaded SeaBIOS >>> (d5) [2014-11-29 17:07:21] Creating MP tables ... >>> (d5) [2014-11-29 17:07:21] Loading ACPI ... >>> (XEN) [2014-11-29 17:07:21] page_alloc.c:1568:d5 Over-allocation for domain >>> 5: 1057417 > 1057416 >>> (XEN) [2014-11-29 17:07:21] memory.c:158:d5 Could not allocate order=0 >> This message is a bit red herring. >> >> It's hvmloader trying to populate ram for firmware data. The actual >> amount of extra pages needed depends on the firmware. >> >> In any case it's safe to disallow hvmloader from doing so, it will just >> relocate some pages from ram (hence shrinking *mem_end). > That looks like a better solution > I went with a "leave some slack" so that the error message above is not output. When a change to hvmloader is done so that the message does not appear during normal usage, the extra pages in QEMU can be dropped. >>> extent: id=5 memflags=0 (0 of 1) >>> (d5) [2014-11-29 17:07:21] vm86 TSS at 00098c00 >>> (d5) [2014-11-29 17:07:21] BIOS map: >>> >>> >>> However while QEMU startup ends with 32 "free" pages (maxmem - curmem) >>> this quickly changes to 23. I.E. there are 7 more places that act like a >>> call >>> to xc_domain_populate_physmap_exact() but do not log errors if there is >>> no free pages. And just to make sure I do not fully understand what is >>> happening here, after the message above, the domain appears to work >>> fine and ends up with 8 "free" pages (code I do not know about ends up >>> releasing populated pages). >>> >>> So my current thinking is to have QEMU leave a few (9 to 32 (64?)) pages >>> "free". >>> >> Unless we know before hand how many pages hvmloader needs this number is >> estimation at best. > > Right. It would be nice to get rid of any estimations by: > - making hvmloader use normal ram > - making qemu increase maxmem > - removing all the estimation from libxl Sounds like a plan for 4.6 -Don Slutz From mboxrd@z Thu Jan 1 00:00:00 1970 From: Don Slutz Subject: Re: [Qemu-devel] [PATCH] increase maxmem before calling xc_domain_populate_physmap Date: Wed, 03 Dec 2014 08:39:00 -0500 Message-ID: <547F1274.6090607@terremark.com> References: <5474C96A.6090506@citrix.com> <54768F7F.2030602@terremark.com> <547C6E12.50502@terremark.com> <547CF6C6.4060106@terremark.com> <547E1FC1.70004@terremark.com> <20141203105458.GA4208@zion.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Stefano Stabellini Cc: xen-devel@lists.xensource.com, Wei Liu , Ian Campbell , Andrew Cooper , qemu-devel@nongnu.org, Don Slutz List-Id: xen-devel@lists.xenproject.org On 12/03/14 07:20, Stefano Stabellini wrote: > On Wed, 3 Dec 2014, Wei Liu wrote: >> On Tue, Dec 02, 2014 at 03:23:29PM -0500, Don Slutz wrote: >> [...] >>>>>>> hw_error("xc_domain_getinfo failed"); >>>>>>> } >>>>>>> - if (xc_domain_setmaxmem(xen_xc, xen_domid, info.max_memkb + >>>>>>> - (nr_pfn * XC_PAGE_SIZE / 1024)) < 0) { >>>>>>> + max_pages = info.max_memkb * 1024 / XC_PAGE_SIZE; >>>>>>> + free_pages = max_pages - info.nr_pages; >>>>>>> + real_free = free_pages; >>>>>>> + if (free_pages > VGA_HOLE_SIZE) { >>>>>>> + free_pages -= VGA_HOLE_SIZE; >>>>>>> + } else { >>>>>>> + free_pages = 0; >>>>>>> + } >>>> I don't think we need to subtract VGA_HOLE_SIZE. >>> If you do not use some slack (like 32 here), xen does report: >>> >>> >>> (d5) [2014-11-29 17:07:21] Loaded SeaBIOS >>> (d5) [2014-11-29 17:07:21] Creating MP tables ... >>> (d5) [2014-11-29 17:07:21] Loading ACPI ... >>> (XEN) [2014-11-29 17:07:21] page_alloc.c:1568:d5 Over-allocation for domain >>> 5: 1057417 > 1057416 >>> (XEN) [2014-11-29 17:07:21] memory.c:158:d5 Could not allocate order=0 >> This message is a bit red herring. >> >> It's hvmloader trying to populate ram for firmware data. The actual >> amount of extra pages needed depends on the firmware. >> >> In any case it's safe to disallow hvmloader from doing so, it will just >> relocate some pages from ram (hence shrinking *mem_end). > That looks like a better solution > I went with a "leave some slack" so that the error message above is not output. When a change to hvmloader is done so that the message does not appear during normal usage, the extra pages in QEMU can be dropped. >>> extent: id=5 memflags=0 (0 of 1) >>> (d5) [2014-11-29 17:07:21] vm86 TSS at 00098c00 >>> (d5) [2014-11-29 17:07:21] BIOS map: >>> >>> >>> However while QEMU startup ends with 32 "free" pages (maxmem - curmem) >>> this quickly changes to 23. I.E. there are 7 more places that act like a >>> call >>> to xc_domain_populate_physmap_exact() but do not log errors if there is >>> no free pages. And just to make sure I do not fully understand what is >>> happening here, after the message above, the domain appears to work >>> fine and ends up with 8 "free" pages (code I do not know about ends up >>> releasing populated pages). >>> >>> So my current thinking is to have QEMU leave a few (9 to 32 (64?)) pages >>> "free". >>> >> Unless we know before hand how many pages hvmloader needs this number is >> estimation at best. > > Right. It would be nice to get rid of any estimations by: > - making hvmloader use normal ram > - making qemu increase maxmem > - removing all the estimation from libxl Sounds like a plan for 4.6 -Don Slutz