From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vladimir Zidar Subject: Re: hypervisor memory usage Date: Thu, 29 Oct 2009 12:07:32 +0100 Message-ID: <4AE97774.5020104@mindnever.org> References: <4AE828A4.4080601@mindnever.org> <20091028115800.GA1434@reaktio.net> <4AE832C6.8090309@mindnever.org> <4AE844A3.90300@redhat.com> <4AE8518B.2090101@mindnever.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4AE8518B.2090101@mindnever.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org I have nailed the problem down to RHEL version of compute_dom0_nr_pages() function. Vanilla xen uses something like this to reserve up to 128MB of ram for DMA etc. The same alg. is used in rhel <= 5.2 and also in official xen 3.4.1 if ( dom0_nrpages == 0 ) { dom0_nrpages = avail; dom0_nrpages = min(dom0_nrpages / 16, 128L << (20 - PAGE_SHIFT)); dom0_nrpages = -dom0_nrpages; } However, RHEL >= 5.3 uses this: /* * If domain 0 allocation isn't specified, reserve 1/16th of available * memory for things like DMA buffers. This reservation is clamped to * a maximum of 384MB. */ if ( dom0_nrpages == 0 ) { dom0_nrpages = avail; dom0_nrpages = min(dom0_nrpages / 8, 384L << (20 - PAGE_SHIFT)); dom0_nrpages = -dom0_nrpages; } else { /* User specified a dom0_size. Do not clamp the maximum. */ dom0_max_nrpages = LONG_MAX; } I do understand that they like the idea of reserving more ram, but additionally /8 would make 1/8th of memory instead of 1/16th? So this might be intended behavior, just not advertised anywhere, and as a kind of side effect, specifying dom0_mem would altogether skip this funny allocation scheme - at least in theory [ I've just put dom0_mem=64G (but I have 8G only) ] and it is not coming up, and I will not be able to t see the console for at least next couple of hours. Vladimir Zidar wrote: > Chris, > > good that you pointed to 5.2 vs 5.3 vs 5.4, > the difference in number of pages is noticed between these: > > xen.gz-2.6.18-92.1.22.el5 - last 5.2 update - all pages are ok, > xen.gz-2.6.18-128.el5 - first 5.3 release - ~80000 pages missing > on 8GB ram setup. > > Chris Lalancette wrote: >> Vladimir Zidar wrote: >> >>> Sounds possible. However it would be great if there was switch to >>> disable that feature in case hardware is not capable of VT-d, as I'd >>> rather use those 300mb than have software support for something that >>> I can't actually use. >>> >> >> In point of fact, VT-d is disabled by default; you need to explicitly >> enable it >> for it to use memory. However, it's possible that there's a bug, or >> some other >> change caused the memory difference, so it's worthwhile to try and >> track it down >> a little better. In particular, you jumped from the 5.2 kernel to >> the 5.4, so >> it would be worthwhile to try the 5.3 kernel and see what you get. >> >> > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel